numopt — 168 numerical methods, each cited to its algorithm and equation, tested against an oracle, and replayable iterate by iterate in the browser.

Open the interactive labs →

16 labs, nothing to install
Methods · Research

_{Four runs of numopt.run on Himmelblau's function from x₀ = (−3.75, 2.5) with one stopping test: the first iterate with ‖∇f(xk)‖₂ ≤ 10⁻⁸. Other parameters are the registered defaults · reproduce}

A run returns more than an answer. It returns every iterate, the exact number of function, gradient and Hessian evaluations, and the stopping test that ended it. The Python package is the reference implementation.

Quick start

numopt needs Python 3.11 or later, and its only runtime dependency is NumPy.

pip install git+https://github.com/ML-Dev-Hub/numopt

Note

The distribution is named numopt-lab (the PyPI name numopt belongs to another project), so pip install numopt installs the wrong package. It imports as import numopt.

import numopt

rosen = numopt.problems.get("rosenbrock")
res = numopt.run("bfgs", rosen, x0=[-1.2, 1.0])

res.converged, res.n_iter  # (True, 38)
res.n_fev, res.n_gev, res.n_hev  # (56, 45, 0)
res.trace[1].info["alpha"]  # 0.00135 (strong Wolfe step at k = 1)

gd = numopt.run("gradient_descent", rosen, x0=[-1.2, 1.0])
gd.converged, gd.message  # (False, 'reached max_iter=5000 (‖∇f(x)‖ = 0.00117 > gtol)')

The command line reaches the same methods and problems:

numopt run bfgs rosenbrock --trace     # one row per iterate, k = 0 … 38
numopt list --family roots             # the 16 root finders with their convergence orders
numopt problems --kind lp              # the 10 linear programs in the problem library

What's inside

numopt list prints every method, and numopt problems lists the 103 test problems. Each family name below opens its lab.

Group	Families (methods)	For example
Equations	Root finding (16) · Nonlinear systems (2) · Linear systems (14)	Brent, Newton–Raphson, Broyden, GMRES
Optimization	1-D minimization (8) · Line search (5) · Unconstrained (42) · Nonlinear least squares (2) · Global (5) · Stochastic gradients (9)	BFGS, L-BFGS, Nelder–Mead, Adam, CMA-ES, SVRG
Constrained & discrete	Constrained (6) · Linear & integer programming (10) · Combinatorial (10)	SQP, simplex, branch and bound, Held–Karp
Numerical analysis & data	Quadrature (13) · Differentiation (7) · Interpolation (12) · Regression (7)	Gauss–Legendre, complex step, AAA, Huber

How each method is checked

• Cited. Each method names its source, down to the algorithm and equation (numopt.get_method("bfgs").references → Nocedal & Wright 2006, Algorithm 6.1, eqs. 6.17 and 6.20).
• Tested against oracles. Each method converges on at least two problems, has a failure-path test, and is checked against SciPy or NumPy where an equivalent exists.
• Parity-checked. The labs run a TypeScript port of each method. Each port must reproduce the first ten Python iterates within 10⁻⁸ and, for a deterministic method, the iteration count exactly. CI fails on stale fixtures.
• Honest about failure. converged=True only when the documented stopping test passed. A singular Hessian or an exhausted budget returns converged=False with a message that says what happened; no method raises on numerical breakdown.

Research

research/ holds nine reproducible studies. Eight test a newer method against the classics on equal budgets and state where it loses; one tests the benchmarking method itself. In one study, re-tuned Adam needs 9.2 times as many iterations on an ill-conditioned quadratic turned 45° from the coordinate axes as on the aligned one, while gradient descent needs exactly as many.

Documentation

Architecture (the Result and Step contract) · Contributing · Research protocol

Citing

GitHub's Cite this repository button reads CITATION.cff. When you cite a method, also cite its original source, which numopt.get_method(id).references lists.

numopt is released under the MIT license.