Optimizers Supported

The table below contains a list of different optimzers that DESC is interfaced to. New optimizers can be added easily, see Adding new optimizers.

• Name : Name of the optimizer method. Pass this string to desc.optimize.Optimizer to use the method.

• Scalar : Whether the method assumes a scalar residual, or a vector of residuals for least squares.

• Equality constraints : Whether the method handles equality constraints.

• Inequality constraints : Whether the method handles inequality constraints.

• Stochastic : Whether the method can handle noisy objectives.

• Hessian : Whether the method requires calculation of the full hessian matrix.

• GPU : Whether the method supports running on GPU

• Description : Short description of the optimizer method.

In addition to the listed optimizers, DESC also includes proximal- prefix to them (i.e. proximal-lsq-exact), which allows for proximal optimization. This is useful for optimization problems where ForceBalance is in the constraints. Any optimizer with proximal- prefix solves the equilibrium for each optimization step to preserve the MHD force balance. Iterations of these optimizers will take longer than the original optimizers, but they will be more robust to large perturbations in the initial guess. For example, after an optimization with lsq-auglag optimizer, the final result will not be good force balance, hence we suggest solving equilibrium again (see Advanced QS Optimization) The proximal optimizers are not listed in the table below, but they can be used by passing proximal-OptimizerName, or this conversion will happen automatically if the ForceBalance constraint is used.

List of Optimizers

Name	Scalar	Equality Constraints	Inequality Constraints	Stochastic	Hessian	GPU	Description
fmin-auglag	True	True	True	False	True	True	Augmented Lagrangian trust region method for minimizing scalar valued multivariate function. See https://desc-docs.readthedocs.io/en/stable/_api/optimize/desc.optimize.fmin_auglag.html
fmin-auglag-bfgs	True	True	True	False	False	True	Augmented Lagrangian trust region method for minimizing scalar valued multivariate function. Uses BFGS to approximate Hessian. See https://desc-docs.readthedocs.io/en/stable/_api/optimize/desc.optimize.fmin_auglag.html
lsq-auglag	False	True	True	False	False	True	Least squares augmented Lagrangian for constrained optimizationSee https://desc-docs.readthedocs.io/en/stable/_api/optimize/desc.optimize.lsq_auglag.html
lsq-exact	False	False	False	False	False	True	Trust region least squares, similar to the trf method in scipySee https://desc-docs.readthedocs.io/en/stable/_api/optimize/desc.optimize.lsqtr.html
fmintr	True	False	False	False	True	True	Trust region method for minimizing scalar valued multivariate function. See https://desc-docs.readthedocs.io/en/stable/_api/optimize/desc.optimize.fmintr.html
fmintr-bfgs	True	False	False	False	False	True	Trust region method for minimizing scalar valued multivariate function. Uses BFGS to approximate the Hessian. See https://desc-docs.readthedocs.io/en/stable/_api/optimize/desc.optimize.fmintr.html
sgd	True	False	False	True	False	True	Stochastic gradient descent with Nesterov momentum. See https://desc-docs.readthedocs.io/en/stable/_api/optimize/desc.optimize.sgd.html
optax-custom	True	False	False	True	False	True	Wrapper for custom optax optimizer. See https://desc-docs.readthedocs.io/en/stable/_api/optimize/desc.optimize.sgd.html
optax-adabelief	True	False	False	True	False	True	optax wrapper for adabelief. See https://optax.readthedocs.io/en/latest/api/optimizers.html#optax.adabelief
optax-adadelta	True	False	False	True	False	True	optax wrapper for adadelta. See https://optax.readthedocs.io/en/latest/api/optimizers.html#optax.adadelta
optax-adafactor	True	False	False	True	False	True	optax wrapper for adafactor. See https://optax.readthedocs.io/en/latest/api/optimizers.html#optax.adafactor
optax-adagrad	True	False	False	True	False	True	optax wrapper for adagrad. See https://optax.readthedocs.io/en/latest/api/optimizers.html#optax.adagrad
optax-adam	True	False	False	True	False	True	optax wrapper for adam. See https://optax.readthedocs.io/en/latest/api/optimizers.html#optax.adam
optax-adamax	True	False	False	True	False	True	optax wrapper for adamax. See https://optax.readthedocs.io/en/latest/api/optimizers.html#optax.adamax
optax-adamaxw	True	False	False	True	False	True	optax wrapper for adamaxw. See https://optax.readthedocs.io/en/latest/api/optimizers.html#optax.adamaxw
optax-adamw	True	False	False	True	False	True	optax wrapper for adamw. See https://optax.readthedocs.io/en/latest/api/optimizers.html#optax.adamw
optax-adan	True	False	False	True	False	True	optax wrapper for adan. See https://optax.readthedocs.io/en/latest/api/optimizers.html#optax.adan
optax-amsgrad	True	False	False	True	False	True	optax wrapper for amsgrad. See https://optax.readthedocs.io/en/latest/api/optimizers.html#optax.amsgrad
optax-fromage	True	False	False	True	False	True	optax wrapper for fromage. See https://optax.readthedocs.io/en/latest/api/optimizers.html#optax.fromage
optax-lamb	True	False	False	True	False	True	optax wrapper for lamb. See https://optax.readthedocs.io/en/latest/api/optimizers.html#optax.lamb
optax-lars	True	False	False	True	False	True	optax wrapper for lars. See https://optax.readthedocs.io/en/latest/api/optimizers.html#optax.lars
optax-lbfgs	True	False	False	True	False	True	optax wrapper for lbfgs. See https://optax.readthedocs.io/en/latest/api/optimizers.html#optax.lbfgs
optax-lion	True	False	False	True	False	True	optax wrapper for lion. See https://optax.readthedocs.io/en/latest/api/optimizers.html#optax.lion
optax-nadam	True	False	False	True	False	True	optax wrapper for nadam. See https://optax.readthedocs.io/en/latest/api/optimizers.html#optax.nadam
optax-nadamw	True	False	False	True	False	True	optax wrapper for nadamw. See https://optax.readthedocs.io/en/latest/api/optimizers.html#optax.nadamw
optax-noisy_sgd	True	False	False	True	False	True	optax wrapper for noisy_sgd. See https://optax.readthedocs.io/en/latest/api/optimizers.html#optax.noisy_sgd
optax-novograd	True	False	False	True	False	True	optax wrapper for novograd. See https://optax.readthedocs.io/en/latest/api/optimizers.html#optax.novograd
optax-optimistic_adam_v2	True	False	False	True	False	True	optax wrapper for optimistic_adam_v2. See https://optax.readthedocs.io/en/latest/api/optimizers.html#optax.optimistic_adam_v2
optax-optimistic_gradient_descent	True	False	False	True	False	True	optax wrapper for optimistic_gradient_descent. See https://optax.readthedocs.io/en/latest/api/optimizers.html#optax.optimistic_gradient_descent
optax-polyak_sgd	True	False	False	True	False	True	optax wrapper for polyak_sgd. See https://optax.readthedocs.io/en/latest/api/optimizers.html#optax.polyak_sgd
optax-radam	True	False	False	True	False	True	optax wrapper for radam. See https://optax.readthedocs.io/en/latest/api/optimizers.html#optax.radam
optax-rmsprop	True	False	False	True	False	True	optax wrapper for rmsprop. See https://optax.readthedocs.io/en/latest/api/optimizers.html#optax.rmsprop
optax-rprop	True	False	False	True	False	True	optax wrapper for rprop. See https://optax.readthedocs.io/en/latest/api/optimizers.html#optax.rprop
optax-sgd	True	False	False	True	False	True	optax wrapper for sgd. See https://optax.readthedocs.io/en/latest/api/optimizers.html#optax.sgd
optax-sign_sgd	True	False	False	True	False	True	optax wrapper for sign_sgd. See https://optax.readthedocs.io/en/latest/api/optimizers.html#optax.sign_sgd
optax-sm3	True	False	False	True	False	True	optax wrapper for sm3. See https://optax.readthedocs.io/en/latest/api/optimizers.html#optax.sm3
optax-yogi	True	False	False	True	False	True	optax wrapper for yogi. See https://optax.readthedocs.io/en/latest/api/optimizers.html#optax.yogi
scipy-bfgs	True	False	False	False	False	False	BFGS quasi-newton method with line search. See https://docs.scipy.org/doc/scipy/reference/optimize.minimize-bfgs.html
scipy-l-bfgs-b	True	False	False	False	False	False	L-BFGS-B quasi-newton method with line search. See https://docs.scipy.org/doc/scipy/reference/optimize.minimize-bfgs.html
scipy-CG	True	False	False	False	False	False	Nonlinear conjugate gradient method. See https://docs.scipy.org/doc/scipy/reference/optimize.minimize-cg.html
scipy-Newton-CG	True	False	False	False	True	False	Newton conjugate gradient method. See https://docs.scipy.org/doc/scipy/reference/optimize.minimize-newtoncg.html
scipy-dogleg	True	False	False	False	True	False	Trust region method with dogleg step. Hessian must be positive definite. See https://docs.scipy.org/doc/scipy/reference/optimize.minimize-dogleg.html
scipy-trust-exact	True	False	False	False	True	False	Trust region method using ‘exact’ method to solve subproblem. See https://docs.scipy.org/doc/scipy/reference/optimize.minimize-trustexact.html
scipy-trust-ncg	True	False	False	False	True	False	Trust region method using conjugate gradient to solve subproblem. See https://docs.scipy.org/doc/scipy/reference/optimize.minimize-trustncg.html
scipy-trust-krylov	True	False	False	False	True	False	Trust region method using Krylov iterations to solve subproblem. See https://docs.scipy.org/doc/scipy/reference/optimize.minimize-trustkrylov.html
scipy-trf	False	False	False	False	False	False	Trust region least squares method. See https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.least_squares.html
scipy-lm	False	False	False	False	False	False	Levenberg-Marquardt implicit trust region method. See https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.least_squares.html
scipy-dogbox	False	False	False	False	False	False	Dogleg method with box shaped trust region. See https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.least_squares.html
scipy-trust-constr	True	True	True	False	True	False	Trust region interior point method. See https://docs.scipy.org/doc/scipy/reference/optimize.minimize-trustconstr.html
scipy-SLSQP	True	True	True	False	False	False	Sequential least squares programming method. See https://docs.scipy.org/doc/scipy/reference/optimize.minimize-slsqp.html