Parameters and specs

Every piece of LION is set by parameters: the geometry, a model’s layers, a solver’s settings, an experiment’s noise. They are LIONParameters, plain namespaces you read, change and save. And every object LION records (an optimizer, a loss, a model) is saved as a spec: its class and the arguments that build it again. Together they are how LION knows, from a file alone, how anything was made.

Previous: Operators: CT geometry, projections and reconstructions. Next: Data and experiments.

LIONParameter

A namespace of settings: set any attribute, nest them, print them.

import tempfile
from pathlib import Path

from LION.utils.parameter import LIONParameter

parameters = LIONParameter(learning_rate=1e-3, epochs=10)
parameters.optimizer = LIONParameter(name="Adam", betas=[0.9, 0.99])
parameters.epochs = 20
print(parameters)

learning_rate : 0.001
epochs : 20
optimizer : name : Adam
betas : [0.9, 0.99]

They are saved as JSON, and read back:

folder = Path(tempfile.mkdtemp())
parameters.save(folder / "parameters.json")
print((folder / "parameters.json").read_text())

again = LIONParameter()
again.load(folder / "parameters.json")
print(again.optimizer.betas)

{
    "learning_rate": 0.001,
    "epochs": 20,
    "optimizer": {
        "name": "Adam",
        "betas": [
            0.9,
            0.99
        ]
    }
}
[0.9, 0.99]

default_parameters()

Every class with settings has default_parameters(): every setting it has, with its default (for a method from a paper, the paper’s). Change what you need and give them back.

from LION.models.iterative_unrolled.LPD import LPD
from LION.optimizers.Noise2InverseSolver import Noise2InverseSolver
from LION.reconstructors.PnP import PnP

print(LPD.default_parameters())
print(Noise2InverseSolver.default_parameters())
print(PnP.default_parameters("ADMM"))

model_input_type : ModelInputType.SINOGRAM
initial_reconstruction : default
n_iters : 10
data_channels : [7, 32, 32, 5]
reg_channels : [6, 32, 32, 5]
learned_step : False
step_size : None
step_positive : False
instance_norm : False
conv_bias : True
dual_step_primal_channel : 1
sino_split_count : 4
cali_J : None
recon_fn : default
algorithm : ADMM
initial_reconstruction : default
eta : None
max_iter : 10
cg_max_iter : 100
cg_tol : 1e-07
noise_level : None

A model is made from its operator’s parameters (for CT, the geometry) and its own; it keeps them, so the model is always what its parameters say.

from LION.operators.ct import Geometry

lpd_parameters = LPD.default_parameters()
lpd_parameters.n_iters = 5
model = LPD(Geometry.default_parameters(), lpd_parameters)
print(model.model_parameters.n_iters, model.operator_params.dso)

5 575

The geometry is a LIONParameter too (an OperatorParameters: one that makes an operator), so it saves and loads the same way.

geometry = Geometry.default_parameters()
geometry.save(folder / "geometry.json")
loaded = Geometry()
loaded.load(folder / "geometry.json")
print(loaded.dsd, loaded.image_shape)

1050 [  1 512 512]

Specs

What LION saves of an object is its spec: a dict with the class (_target_) and the arguments that build it. to_spec makes one, build makes the object again.

import torch

from LION.utils import spec

optimizer = torch.optim.Adam(model.parameters(), lr=1e-4, betas=(0.9, 0.99))
print(spec.to_spec(optimizer))  # its hyperparameters
print(spec.to_spec(torch.nn.L1Loss(reduction="sum")))
print(spec.to_spec(geometry)["dso"])

{'_target_': 'torch.optim.adam.Adam', 'lr': 0.0001, 'betas': {'_target_': 'builtins.tuple', '_args_': [[0.9, 0.99]]}, 'eps': 1e-08, 'weight_decay': 0, 'amsgrad': False, 'maximize': False, 'foreach': None, 'capturable': False, 'differentiable': False, 'fused': None, 'decoupled_weight_decay': False}
{'_target_': 'torch.nn.modules.loss.L1Loss', 'reduction': 'sum'}
{'_target_': 'numpy.array', '_args_': [575], 'dtype': 'int64'}

An optimizer is built again for a model’s parameters, given when building:

new_optimizer = spec.build(spec.to_spec(optimizer), params=model.parameters())
print(new_optimizer.defaults["lr"])

0.0001

A model’s spec is its class and parameters: building it gives the same architecture with new random weights (the weights are saved separately).

model_spec = spec.to_spec(model)
print(model_spec["_target_"])
same_architecture = spec.build(model_spec)
print(type(same_architecture).__name__, same_architecture.model_parameters.n_iters)

LION.models.iterative_unrolled.LPD.LPD
LPD 5

Your own classes

LION reads an object’s arguments from its attributes of the same names (the “same-name rule”): keep every constructor argument as an attribute and your class is saved like LION’s. (If it can’t, define spec_kwargs(), which returns them.)

class WeightedL1(torch.nn.Module):
    def __init__(self, weight=1.0):
        super().__init__()
        self.weight = weight  # same name as the argument

    def forward(self, output, target):
        return self.weight * (output - target).abs().mean()


loss_spec = spec.to_spec(WeightedL1(weight=0.5))
print(loss_spec)

{'_target_': '__main__.WeightedL1', 'weight': 0.5}

Building only imports LION, torch, numpy and a few builtins: a file can’t run other code. Your own modules are allowed by name:

try:
    spec.build(loss_spec)
except Exception as error:
    print(type(error).__name__, ":", str(error)[:80], "...")
loss = spec.build(loss_spec, allow=["__main__."])
print(loss.weight)

SpecError : '__main__.WeightedL1' is not in the modules a spec may use (LION., torch.nn., to ...
0.5

Specs are what LION’s records are made of: the data a model saw, how it was trained, how it was tested. Saving, loading and records reads them.