Repository navigation
Add sweep_tunable_mask_size/recommend_target_impact_share - #59
Merged
Merged
Conversation
select_tunable_mask's target_impact_share picks a comparable *share* of leverage on any model, but the same share does not sit at a comparable point on the *fit* curve -- two models with similarly shaped leverage curves can have fit curves that diverge sharply. The only way to tell is to tune at a few candidate sizes and look at where the improvement stops. sweep_tunable_mask_size tunes at each of a list of target_impact_shares (and seeds) and reports the resulting fit per point. recommend_target_impact_share picks the smallest share past which further tuning stops paying for itself, using two independent checks against a later point's best result: a relative-improvement tolerance, and, where at least one of the two points compared has more than one seed, a check against their own local seed spread (not a sweep-wide pooled one, since how noisy a comparison is can differ across the sweep). MaskSizePoint.rmse takes min(rmse_trace) per seed rather than the final entry, since rmse_trace only advances at a step that improved the (possibly prior-penalised) objective, per cmaes_kcat_tuning's own docstring -- with the default nonzero prior_penalty_weight the last entry is not guaranteed to be the best one recorded. sweep_tunable_mask_size's default target_impact_shares now starts at 0.3 (previously 0.5) and its default seeds is (0, 1) (previously a single seed), so a default call's noise-floor check has something to compare against instead of silently degrading to the relative check alone.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
select_tunable_mask'starget_impact_sharepicks a comparable shareof leverage on any model, but the same share does not sit at a
comparable point on the fit curve -- two models with similarly shaped
leverage curves can have fit curves that diverge sharply. The only way
to tell is to tune at a few candidate sizes and look at where the
improvement stops.
This adds two functions to
geckopy.kcat_tuning.evotune:sweep_tunable_mask_sizetunes at each of a list oftarget_impact_shares (and seeds) and reports the resulting fit perpoint, reusing one screen across all of them.
recommend_target_impact_sharepicks the smallest share past whichfurther tuning stops paying for itself, using two independent
checks against a later point's best result: a relative-improvement
tolerance, and, where at least one of the two points compared has
more than one seed, a check against their own local seed spread
(not a sweep-wide pooled one, since how noisy a comparison is can
differ across the sweep).
Along the way:
MaskSizePoint.rmsetakesmin(rmse_trace)per seedrather than the final entry, since
rmse_traceonly advances at astep that improved the (possibly prior-penalised) objective, per
cmaes_kcat_tuning's own docstring -- with the default nonzeroprior_penalty_weightthe last entry is not guaranteed to be the bestone recorded.
sweep_tunable_mask_size's defaulttarget_impact_sharesnow startsat 0.3 (not 0.5) and its default
seedsis(0, 1)(not a singleseed), so a default call's noise-floor check has something to compare
against instead of silently degrading to the relative check alone.
Built and validated against real sweeps on both the E. coli and yeast
models worked with in this branch's originating session; matches
manual findings on both.
New tests in
tests/test_mask_size.py; full suite passes(1448 passed, 1 deselected, 1 xfailed).