LION.optimizers.Noise2InverseSolver

Classes

Noise2InverseSolver(model, optimizer, loss_fn)

Noise2Inverse: the measurements are split into sino_split_count interleaved parts (for CT, every k-th angle), each reconstructed with recon_fn (by default the operator's default reconstruction, FDK for CT), and the model learns to map the mean of some parts' reconstructions to the mean of the others'.

---

class LION.optimizers.Noise2InverseSolver.Noise2InverseSolver(model, optimizer, loss_fn, operator_params=None, solver_params=None, model_regularization=None, scheduler=None, scheduler_step='epoch', verbose=1, device=None, seed=0, deterministic=False)

Bases: LIONsolver

Noise2Inverse: the measurements are split into sino_split_count interleaved parts (for CT, every k-th angle), each reconstructed with recon_fn (by default the operator’s default reconstruction, FDK for CT), and the model learns to map the mean of some parts’ reconstructions to the mean of the others’.

Works with any operator whose parameters can be split (OperatorParameters.split); others raise when the solver is made.

Every solver takes (model, optimizer, loss_fn, operator_params, solver_params, model_regularization, scheduler, scheduler_step, verbose, device, seed, deterministic), in that order (ARSolver has no loss_fn).

operator_params: the parameters of the operator (for CT, the geometry); the model’s, if it has them.

solver_params: the solver’s settings, from default_parameters().

model_regularization: None, or a term added to the training loss of every mini-batch, model_regularization(model, step): a penalty on the model’s weights (step unused) or on what it does with the mini-batch (e.g. LION.losses.jacobian_penalty). step has input (what the model was given), output, target (what the output is compared with; None for solvers without one) and forward (the model as the solver calls it, e.g. with the noise level). Recorded as loss_fn is.

scheduler: a torch learning-rate scheduler made on optimizer (e.g. torch.optim.lr_scheduler.StepLR(optimizer, 30)), or None. It steps after every epoch (scheduler_step="epoch", torch’s convention) or every mini-batch ("batch", e.g. for OneCycleLR). ReduceLROnPlateau steps with the validation value of the epochs that validate, or with the training loss when there is no validation. Its state is saved in checkpoints, and the training record has it and the learning rate of every epoch.

seed: the training seed. When training starts, torch (CPU and CUDA), numpy and Python’s random are seeded with it, so the same solver, model and data give the same training, and checkpoints keep the random state, so a resumed run continues the same way. The randomness within a run (shuffling, noise a solver adds) still changes every batch and epoch. Two runs with the same seed give the same model: use different seeds for an ensemble. None: unseeded.

deterministic: use only deterministic algorithms (torch’s use_deterministic_algorithms, deterministic cuDNN), so that on a GPU two runs with the same seed give exactly the same model, and a resumed run the uninterrupted one. Slower; a few operations then raise (torch names them). It applies to the whole program from here on. Off, GPU runs with the same seed differ slightly (e.g. ~1e-3 in the weights after a few epochs). On a GPU it needs CUBLAS_WORKSPACE_CONFIG=:4096:8 before cuBLAS starts: it is set here if it isn’t (exporting it before the script is surer).

verbose: how much progress is shown. 0: nothing; 1 (default): one progress bar for training (every mini-batch of every epoch, with the epoch, the loss, the last validation value and the time left) and one for testing; 2: also the validation bars and messages (each epoch’s losses, checkpoints, the seed). True is 1, False 0.

classmethod X_one_strategy(sino_split_count)

The X:1 strategy of the paper (the default, which the paper found better): each split in turn is the target, the mean of the others the input. The target sets cali_J (1-based): [[1], [2], ...].

classmethod one_X_strategy(sino_split_count)

The 1:X strategy of the paper: each split in turn is the input, the mean of the others the target. The target sets cali_J (1-based): [[2, 3, ...], [1, 3, ...], ...].

static default_parameters()

sino_split_count: the number of splits (4); cali_J: the sets of splits used as targets, 1-based (None: X_one_strategy; one_X_strategy(k) for the paper’s 1:X; any sets for others); recon_fn: how each split is reconstructed (“default”: the operator’s default reconstruction).

mini_batch_step(y, targets)

The loss of the model mapping the mean of some splits’ reconstructions to the mean of the others’, averaged over the target sets cali_J (targets aren’t used: self-supervised).

reconstruct(y)

The model’s reconstructions of the measurements y, as the paper does: the mean of its outputs on each target set’s input.

static cite(cite_format='bib')

Where Noise2InverseSolver comes from: its citation, BibTeX (“bib”) or “MLA”.