A programmable boosting foundation for researchers and AI agents.
Build boosting algorithms in Python by composing objectives, statistics, split rules, weak learners and update policies. OpenBoost provides a NumPy CPU reference and experimental CUDA execution with Python-authored kernels.
An algorithm change can require more than a new loss function. It may need extra statistics, a different split constraint, a custom leaf solver, coupled outputs, or a different acceptance rule. OpenBoost aims to make each of those decisions accessible through public components and ordinary Python training loops.
The design takes inspiration from PyTorch's composable building blocks and explicit execution. Here the building blocks are boosting operations: objective geometry, named row fields, histograms, candidate scoring, feasibility, routing, leaf solving, and immutable training transactions. A recipe is a working composition that a researcher or agent can inspect, modify and reuse.
Standard GBDT, NaturalBoost/NGBoost-style distributional methods, FormulaBoost-style structured models and training many models guide the abstraction boundaries. Applications matter equally: classification, regression, ranking, quantiles, multi-output prediction, counts, positive and aggregate targets, survival, distributional modeling and model selection all remain required v1 scope.
Success means making a correct algorithm change easier, then demonstrating useful quality and execution cost on real workloads. CPU is the semantic reference and usable development path; CUDA is the route toward efficient execution. Comparative agent-authoring studies are currently deferred while foundation construction and validation continue. Author productivity and adoption benefits remain hypotheses.
Experimental v1, under construction. The current foundation is merged through PR #25. It is not a drop-in replacement for XGBoost, LightGBM or CatBoost, and full v1 acceptance remains open.
Public CPU components include typed numeric/categorical data, fitted preparation, weights and offsets, named statistics, composable split/routing/leaf operations, and depthwise, best-first and symmetric growers. Scalar/vector leaves and mapped outputs share explicit proposal/accept/reject state. All twelve CPU recipes support independent validation patience; saved models retain their required inference metadata.
The CUDA implementation uses CuPy-owned storage and streams with Python kernels
compiled by numba-cuda. Public operations expose storage, fields, histograms,
scores, feasibility masks, routing, scalar leaves, depthwise trees and resident
transactions. Training uses explicit device interfaces; the CPU recipe API does
not automatically dispatch to CUDA. Current CUDA tree growth covers numeric and
missing features, with scalar trees and mapped multi-parameter updates.
| Use case | CPU implementation | Verified CUDA scope |
|---|---|---|
| Regression | Squared error | Resident squared recipe |
| Classification | Binary, multiclass | Binary recipe; multiclass pending |
| Counts and positive/aggregate targets | Poisson with exposure, Gamma, fixed-power Tweedie, frequency–severity composition | Poisson recipe; other cells pending |
| Ranking and quantiles | Query-local pairwise/lambda ranking, quantile and penalized leaves | Pending |
| Survival | Fixed-scale log-normal AFT with events/right censoring | Pending |
| Distributional and structured models | Normal ordinary/Fisher updates, saturation Formula/full-GGN updates | Bounded Normal joint/ordered recipes; Formula pending |
| Multi-output regression | Independent/shared trees, projected splits and target scaling | Vector topology pending |
| Train-many | Shared preparation and independent sequential runs, verified at M=1/8/32 | Compatible resident execution pending |
CUDA entries describe bounded correctness evidence, not complete feature coverage or a speed guarantee. Categorical CUDA growth, broader vector learners and fused train-many remain unverified. See the CPU component guide, tree contracts, stopping semantics and explicit CUDA interfaces for supported inputs and limits.
- Latest CUDA validation: run 12 passes 571/571 real T4 cases: 153 binary/Poisson checks and 418 regressions. All 77 JSON artifacts are retained. The offline audit verifies 246 numerical loss-change comparisons and replays 32 final/best models from saved input bytes.
- Reliable Normal decisions: comparison and revalidation evidence covers all 529 revised requirements across two executions. Earlier failed verdicts remain preserved. This is bounded coverage, not full Normal conformance.
- Measured internal improvement: run 11 passes 474 T4 checks and three cost gates. Parallel field validation reduces median warm fit time for synthetic squared boosting at 100,000 rows from 13.513 to 8.947 seconds, with unchanged model/prediction bytes. That workload uses 16 features, depth three and 20 rounds; the reduction is 33.79% against the earlier OpenBoost implementation on the same T4.
- CPU and packaging: the merged checkpoint has 2,292 local CPU tests passing. Hosted CI passes Linux/macOS on Python 3.10/3.12, including offline audits and package builds; strict documentation checks also pass. Historical tests are explicitly separated from current conformance.
These results do not establish competitive speed or predictive quality against mature boosting libraries. Real application evaluations and the formal end-to-end quality/cost gate remain open. The earlier incomplete performance checkpoint is retained alongside the later complete measurements.
Use Python 3.10+ and install from this checkout:
uv sync --extra testThis example supplies a custom learner through the public growth and feasibility operations, fits a squared-error recipe, and saves its best validation model.
from functools import partial
from openboost import NumericData, Problem, RunContext
from openboost.artifacts import Model
from openboost.ops import feasible
from openboost.recipes import squared
from openboost.tree import depthwise
train_x = NumericData([[0], [1], [2], [3]], [10, 11, 12, 13], ("x",))
valid_x = NumericData([[0.5], [2.5]], [20, 21], ("x",))
train = Problem(train_x, [[-3], [-1], [1], [3]], train_x.row_ids)
valid = Problem(valid_x, [[-2], [2]], valid_x.row_ids)
def learner(binned, fields):
return depthwise(
binned, fields, max_depth=1,
legality=partial(feasible, min_child_h=2),
)
fit = squared(
train, valid, context=RunContext("example", seed=7),
learner=learner, rounds=3, learning_rate=0.5, bins=4,
)
model = fit.state.best_model
prediction = model.predict(valid_x) # Shape: (2, 1)
model.save("model.json")
restored = Model.load("model.json")For deeper changes, compose objective/statistics operations
and run transactions directly. CUDA users need real NVIDIA
hardware and the optional dependencies (uv sync --extra cuda); start with the
separate device execution guide.
- Complete the required CUDA recipes, starting with multiclass, then AFT and vector topology, with independent mathematics, CPU/CUDA checks and persisted inference for each declared scope.
- Establish compatible train-many execution, preserving independent state while reusing preparation and device resources.
- Measure real-workload quality and complete execution cost with fair baselines, then stabilize the public contracts supported by that evidence.
All R1–R9 / C1–C7 / A1–A13 requirements remain in scope. Each application family needs its own implementation and evaluation. Multi-GPU, Ray and out-of-core expansion are outside the active plan.
uv run pytest tests/ -m "not gpu and not benchmark" -n 0 -q
uv run ruff check src/openboost tests/v1 tests/conftest.py
uv run mkdocs build --strict
uv buildDefault discovery runs tests/v1/; see test scope. Run GPU-marked
tests only on real hardware. The current documentation lives in docs/v1/.
The retired implementation remains at revision 50acfc6 for reproducing old APIs,
examples and experiments. There is no compatibility layer in v1. Historical
packages and benchmarks describe their recorded revisions; all new claims must
link to reproducible evidence for the current foundation.