src.environment.problem.SOO.COCO_BBOB.kan.LBFGS

Module Contents

Classes

LBFGS

Implements L-BFGS algorithm.

Functions

_cubic_interpolate
_strong_wolfe

Data

__all__

API

src.environment.problem.SOO.COCO_BBOB.kan.LBFGS.__all__

[‘LBFGS’]

src.environment.problem.SOO.COCO_BBOB.kan.LBFGS._cubic_interpolate(x1, f1, g1, x2, f2, g2, bounds=None)

src.environment.problem.SOO.COCO_BBOB.kan.LBFGS._strong_wolfe(obj_func, x, t, d, f, g, gtd, c1=0.0001, c2=0.9, tolerance_change=1e-09, max_ls=25)

class src.environment.problem.SOO.COCO_BBOB.kan.LBFGS.LBFGS(params, lr=1, max_iter=20, max_eval=None, tolerance_grad=1e-07, tolerance_change=1e-09, tolerance_ys=1e-32, history_size=100, line_search_fn=None)

Bases: torch.optim.Optimizer

Implements L-BFGS algorithm.

Heavily inspired by minFunc <https://www.cs.ubc.ca/~schmidtm/Software/minFunc.html>_.

.. warning:: This optimizer doesn’t support per-parameter options and parameter groups (there can be only one).

.. warning:: Right now all parameters have to be on a single device. This will be improved in the future.

.. note:: This is a very memory intensive optimizer (it requires additional param_bytes * (history_size + 1) bytes). If it doesn’t fit in memory try reducing the history size, or use a different algorithm.

Args: lr (float): learning rate (default: 1) max_iter (int): maximal number of iterations per optimization step (default: 20) max_eval (int): maximal number of function evaluations per optimization step (default: max_iter * 1.25). tolerance_grad (float): termination tolerance on first order optimality (default: 1e-7). tolerance_change (float): termination tolerance on function value/parameter changes (default: 1e-9). history_size (int): update history size (default: 100). line_search_fn (str): either ‘strong_wolfe’ or None (default: None).

Initialization

_numel()

_gather_flat_grad()

_add_grad(step_size, update)

_clone_param()

_set_param(params_data)

_directional_evaluate(closure, x, t, d)

step(closure)

Perform a single optimization step.

Args: closure (Callable): A closure that reevaluates the model and returns the loss.