LION.utils.math¶

Utility math functions.

Functions

power_method(op[, maxiter, tol, device, seed])

The norm of the operator op (its largest singular value), estimated by power iterations on AᵀA, as a tensor.

LION.utils.math.power_method(op, maxiter=100, tol=1e-06, device=None, seed=0)[source]¶

The norm of the operator op (its largest singular value), estimated by power iterations on AᵀA, as a tensor.

maxiter: the most iterations; tol: stop once ||A v - sigma u|| <= tol * sigma (relative, so independent of the operator’s scale); device: where to compute (the operator’s if None); seed: of the random starting vector, so that the estimate is the same on every call (e.g. step sizes computed from it; None: torch’s global random state).