train: the exact walk, one optimizer step per window on the gradient of its whole loss (opt-in) by joelteply · Pull Request #47 · CambrianTech/llama.cpp

train: the exact walk carries a recurrent state's gradient across chunks (hybrid models)

joelteply

and

claude

committed

ed292df

train: the exact walk's host memory is counted before a window runs and refused by name over its budget (Fable's ask 4)

joelteply

and

claude

committed

8b5f095

train: the exact walk checkpoints a recurrent state every few chunks to fit its host budget

joelteply

and

claude

committed

5a06f8b

train: the exact walk's gradient horizon shrinks to fit the device, and the run reports the one it used

joelteply

and

claude

committed

1ef1a29

tests: test-grad-leaf and test-opt-dynamic-accum take the CPU backend by type (CI's backend-DL build)

joelteply

and

claude

committed

9f7e575

examples: the finetune examples name the exact walk's fields (off) in their llama_opt_params

joelteply

and

claude

committed

477e7cd

train: the exact walk resets its horizon to the request each window; measured at the served cache types (Fable on #47)

joelteply

and

claude

committed

7b90bd1

Merge remote-tracking branch 'origin/feat/props-weight-residency' into train/walk-exact-gradient

joelteply

committed

10292b5

Navigation Menu

Repository navigation

train: the exact walk, one optimizer step per window on the gradient of its whole loss (opt-in) - #47

train: the exact walk, one optimizer step per window on the gradient of its whole loss (opt-in)#47
joelteply merged 14 commits into
feat/props-weight-residencyfrom
train/walk-exact-gradient

Commits on Oct 6, 2026

ggml: GGML_TENSOR_FLAG_GRAD, a leaf whose gradient the backward computes and no optimizer updates

ggml-opt: graphs built per step start every optimizer period from zero gradients

ggml-opt: accumulators and momenta belong to a parameter, never to a node index

ggml-opt: a caller-driven optimizer period, a weighted loss with an extra term, and GRAD-leaf gradients read back

ggml: CONCAT backward slices a contiguous gradient (a transposed one read the wrong elements)

train: the exact walk, one optimizer step per window on the gradient of its whole loss (opt-in)

Commits on Oct 7, 2026

train: the exact walk carries a recurrent state's gradient across chunks (hybrid models)

train: the exact walk's host memory is counted before a window runs and refused by name over its budget (Fable's ask 4)

train: the exact walk checkpoints a recurrent state every few chunks to fit its host budget

train: the exact walk's gradient horizon shrinks to fit the device, and the run reports the one it used

tests: test-grad-leaf and test-opt-dynamic-accum take the CPU backend by type (CI's backend-DL build)

examples: the finetune examples name the exact walk's fields (off) in their llama_opt_params

train: the exact walk resets its horizon to the request each window; measured at the served cache types (Fable on #47)

Merge remote-tracking branch 'origin/feat/props-weight-residency' into train/walk-exact-gradient