LION.optimizers.training_record

The record of how a model was trained.

A solver writes it, as the training of every model file it saves (final, best validation, checkpoints), next to the record of the training data (dataset_params, LION.data_loaders.data_record):

• solver: the solver’s spec, without the model (the file’s model), the optimizer (below) and where it ran (device, verbose, save_folder); its solver_params include values it computed (e.g. E2I’s noise_std);

• optimizer: the optimizer’s class and hyperparameters; parameter_groups: its parameter groups (each one’s parameters, by name in the model, and settings, with the learning rate it started with), or None for one group of all the parameters with those hyperparameters; scheduler: the learning-rate scheduler’s class and arguments, or None (when it steps, scheduler_step, is the solver’s);

• seed: the training seed (None: unseeded), and initial_weights_sha256, a hash of the weights training started from;

• initial_weights: how those can be made again (LIONmodel.instantiate( file, initial_weights=True)): {"from": "random_state"} (the model made again from the random state saved with it), {"from": "file", "file": ...} (saved next to the results), or {"from": None}; for weights that weren’t made from the random state, origin: the model file they were loaded from and its weights hash (a fine-tuned model’s original), or None if they didn’t come from a LION model file;

• train: how the training data were fed (batch_size, shuffle, drop_last); the data themselves are the file’s dataset_params;

• validation: its data record, how they were fed, every (epochs), the validation function, and the reconstructor it reconstructs with (its spec: operator and parameters), if one was given;

• epochs: asked (train(n)) and done;

• weights: which weights the file holds: kind (“final”, “best_validation”, “checkpoint”), after which epoch (counted from 1), and for the best validation its value;

• history: the training loss, validation value (null when not validated) and learning rate (a list, one per parameter group, when there are several) of each epoch done.

What can’t be written as a spec (a lambda loss) is named, with a warning. A record is a LIONParameter (record.epochs.done) whose specs (record.solver, record.optimizer) stay dicts; it is saved as plain JSON and read back as a LIONParameter, like the data record.

Functions

history(values, done)	The first done values as JSON numbers (None for NaN).
is_training_record(value)	A training record, as a LIONParameter or as the dict read from JSON.
loader_settings(loader)	How a DataLoader feeds its data: batch_size, shuffle, drop_last.
read_training_record(source)	The training record in source: a record, a model's metadata holding one (as its training), or the path of a .json with either.
weights_sha256(model)	A hash of a model's weights (names and values, in order).

---

LION.optimizers.training_record.history(values, done)

The first done values as JSON numbers (None for NaN).

LION.optimizers.training_record.is_training_record(value)

A training record, as a LIONParameter or as the dict read from JSON.

LION.optimizers.training_record.loader_settings(loader)

How a DataLoader feeds its data: batch_size, shuffle, drop_last. A sampler other than torch’s sequential and random ones is named in sampler.

LION.optimizers.training_record.read_training_record(source)

The training record in source: a record, a model’s metadata holding one (as its training), or the path of a .json with either.

LION.optimizers.training_record.weights_sha256(model)

A hash of a model’s weights (names and values, in order).