Repository navigation
Tune Presage: deltas, user words, speller ranking and bigger n-gram databases - #501
Merged
Merged
Conversation
Set the n-gram DELTAS to 0.01 0.08 0.3 0.6. A sweep with eval_ngram.py on held-out HPLT text gives +0.23 (en), +0.13 (pl) and +0.22 (de) percentage points of KSR. Top-1 does not drop. The deltas now sum to less than 1, so an n-gram score stays below the 1.0 cap of the combiner and below an exact text expansion. Scale the Hunspell, Aspell, user dictionary and recency scores by the same factor as the unigram delta (x0.08). They keep their place among the rarest n-gram words. Rebuild libpresage with the DictionaryPredictor fix from bartekplus/presage#2: a user word that starts with the typed prefix is a completion, so it shows from its first letters. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Rebuild libpresage with two more changes from bartekplus/presage#2: - A user word that does not start with the typed prefix is offered only when it is a typo of the word's start. Before, unrelated user words filled empty slots ("fluent" gave "kubernetes"). - The n-gram model also scores the speller corrections, times the new CORRECTION_WEIGHT (0.000001). On 800 synthetic typos per language, top-1 goes up by 2.1-2.6 points for en, pl and de. KSR does not change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The old pruning capped each n-gram order at 75% of the order below. For
en_US it dropped about 95% of the n-grams seen 10 times or more. Now all
words are kept, and the most frequent longer n-grams (orders 2 to 4
together) are kept, up to 4.5 for each word. The databases are 2.00 times
the old size (29.8 MB -> 60.0 MB). KSR goes up in all 10 languages, by
0.88 (fr) to 2.99 (hr) points, and top-1 by 0.27 to 1.24 points.
Regenerate the Review lexicons from the new databases and fix what the
bigger databases exposed:
- German gender: contrary cues under 1/20 of a word's cues are noise
("jede gerät", "was das Kunden"). The weak-noun -n filter also covers
m/M ("einen Nachbarn"), and nForm no longer cuts two letters from a
word that does not end in -en.
- German runtime: a feminine noun in -e keeps its plural -n
("anhand ihrer Unterlagen").
- German supplement: author "altgold", which the new counts do not hold.
- Spanish: author "física" as a noun. The generator also probes
"c" + every verb for Bloom false positives; "cterminar" was one.
- e2e: "Where wa it?" now ranks "water" second after "was".
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bartekplus
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three commits. They tune the Presage configuration, rebuild
libpresage.wasmfrom bartekplus/presage#2 and bartekplus/presage#3 (both merged), and rebuild the n-gram databases.1. N-gram deltas and user words from their first letters
DELTAS0.125 0.25 0.5 0.75→0.01 0.08 0.3 0.6. A sweep of 182 ratios withscripts/eval_ngram.pyfound a flat plateau near 1 : 8 : 32–64 : 32–128. KSR: en +0.23, pl +0.13, de +0.22. The deltas now sum to less than 1, so an n-gram score stays below the combiner's 1.0 cap and below an exact text expansion.2. Speller ranking and unrelated user words
CORRECTION_WEIGHT0.000001: the n-gram model also scores speller corrections. On 800 synthetic typos per language, top-1 goes up by 2.1–2.6 points (en, pl, de), and KSR does not change. "i went to teh" → "the" first (before: "tehran").3. Bigger n-gram databases (2×)
gen_ngram.prune: the old rule capped each order at 75% of the order below and dropped about 95% of the en n-grams seen 10+ times. Now all words are kept, plus the most frequent longer n-grams of all orders together, up to 4.5 per word..datafiles: 160.5 MB → 190.6 MB.The Review lexicons were regenerated from the new DBs. The bigger DBs exposed these issues, and this commit fixes them:
nFormbug is fixed.prepositionCase.ts): a feminine noun in "-e" keeps its plural "-n" ("anhand ihrer Unterlagen").Test plan
bun run checkbun run test: 97697 + 42 pass, 0 fail (live Presage and Review tests included)bun run test:e2e,bun run test:e2e:full(all 6 shards),bun run check:e2e:coveragescripts/test_*.py)🤖 Generated with Claude Code