Beam Weaver

Physics-informed n-step reinforcement learning for stochastic photon transport in water

Status: research prototype License: Apache-2.0 Domain: radiation transport & AI

Beam Weaver is a research software prototype for learning event-by-event photon transport in a homogeneous water phantom. It combines a custom Monte Carlo reference simulator, a physics-informed n-step Soft Actor-Critic reinforcement learning environment, and evaluation tools that compare learned particle histories against conventional transport outputs.

What Beam Weaver Does

Beam Weaver explores whether a Soft Actor-Critic (SAC) agent can learn the stochastic logic of photon transport in water at the event level, rather than only predicting final observables.

For each photon history, the code predicts:

The next interaction type
The distance to the next interaction
Scattering geometry
The outgoing photon state
The number, energy, and direction of secondary particles

This makes Beam Weaver a learned transport-engine prototype rather than a generic machine-learning regressor.

Photon Interactions Modeled

Rayleigh Scattering

Coherent scattering with energy-dependent form factors for water.

Compton Scattering

Incoherent scattering with recoil electron kinematics.

Photoelectric Absorption

Shell-specific absorption with electron emission.

Pair Production

Electron-positron pair creation with energy sharing.

Training Architecture

Beam Weaver is trained in two stages:

Stage I — Pretraining: Supervised learning of a multi-head physics branch on Monte Carlo-generated labels (interaction type, free path, scattering angles, secondary particles).
Stage II — Curriculum RL: A five-phase curriculum that progressively transitions from teacher-forced Monte Carlo supervision to autonomous policy learning using a hybrid n-step SAC framework.

The RL objective is physics-regularized, combining the standard SAC loss with auxiliary physics losses for energy, angle, and interaction-type consistency.

Quick Start

# Create environment
conda env create -f environment.yml
conda activate beam-weaver

# Or with pip
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

# Run Beam Weaver
python Beam_weaver_0.1.0.py

The script is interactive and supports: generating Monte Carlo data, pretraining the physics head, training a SAC model, evaluating trained models, and comparing MC vs. agent outputs.

Authors

Pedro Teles

Assistant Professor, Department of Physics and Astronomy, Faculty of Sciences, University of Porto

João Melo

MSc Student in Medical Physics, Department of Physics and Astronomy, Faculty of Sciences, University of Porto

References

F. Salvat. PENELOPE-2018: A Code System for Monte Carlo Simulation of Electron and Photon Transport. OECD/NEA, 2019.
T. Haarnoja et al. Soft Actor-Critic: Off-Policy Maximum Entropy Deep RL with a Stochastic Actor. ICML, 2018.
A. Raffin et al. Stable-Baselines3: Reliable RL Implementations. JMLR, 22:1–8, 2021.
J. H. Hubbell et al. Atomic Form Factors, Incoherent Scattering Functions, and Photon Scattering Cross Sections. J. Phys. Chem. Ref. Data, 4(3), 1975.