Repository navigation
Consolidate the tool showcases, rebuild four packages, write out five folders - #21
Merged
Merged
Conversation
… folders An audit found twenty-one of thirty folders with fewer than six files and nine with no code, several a README and one image presented on the landing page as a module. This is the long tail, done properly. Seven tool showcases consolidated into tool_notes/. Excalidraw, Kumu, Observable, RawGraphs, Flourish, Power BI and Miro live on their own sites; what a repository can hold about each is a page of judgement, when it is worth the setup and what goes wrong, plus one worked file. That is what each now contains, and the seven are one landing-page row. foundation_tools/ deleted. It carried a second, different implementation of data_validation, label_variables, replication and survey_to_codebook, in five-line functions overlapping with the top-level folders. The two unique .sps companions moved up; the rest went. Four folders rebuilt as tested Python packages, 39 tests in CI. data_validation: rule-driven from a data dictionary; duplicates, required, ranges, allowed values, types, name style, column set, cross-file ids; one issue row per problem, by identifier. Blanks in skip-pattern questions are not flagged and sentinel codes count as missing, because the opposite policy, which the old scripts had, teaches people to ignore the report. label_variables: labels from the dictionary written into .dta and .sav, where they survive; round-trips both ways. The old script attached labels to df.attrs and wrote a CSV on the next line, which discards them. survey_to_codebook: XLSForm to codebook, resolving groups, repeats and choice lists and reporting form defects. The bundled sample form named a list `hobby` that the form referenced as `hobbies`; the old notebook printed an empty list without comment, the new tool reported it on first run, and the sample is fixed. replication: one entry point, a recorded result, a Python-against-R cross-check that agrees to six places, and a test that corrupts a value and confirms verification fails. A verification that cannot fail is not verifying anything. The three packages share one dictionary format, so a codebook generated from an XLSForm can be applied as labels and used as validation rules without retyping. Five folders built out as written content. annotated_research: five illustrative briefs under one eight-tag annotation scheme, weakest design to strongest, invented programmes and no real citations, said so on every page. eval_docs: logframe template, indicator reference sheet, MEL framework outline, indicator guidance. writing_guides: results chain, evaluation report structure, policy brief, writing about uncertainty, around the idea that the verb is decided by the design. learning_layers: learning as scheduled decisions at three tempos. KM_tools: folder structure and a project README template. SignalStack is dead and is gone from the family table; PolicyStack is in it. Emoji stripped from the notes that are kept. One bug caught on the way. pandas 3 gives text columns a str dtype, so a blank check keyed on dtype == object silently skipped every text column. CI installs unpinned pandas and would have shipped it. A test now pins the behaviour on string dtype. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DMYhECx2Dzbqad6KJxWZ7t
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
An audit found twenty-one of thirty folders with fewer than six files and nine with no code, several of them a README and one image presented on the landing page as a module. This is the long tail, done properly, per the direction: consolidate the seven tool showcases, build out the rest.
Consolidated:
tool_notes/Excalidraw, Kumu, Observable, RawGraphs, Flourish, Power BI and Miro live on their own sites. What a repository can hold about each is a page of judgement (when it is worth the setup, when it is not, what goes wrong) plus one worked file. Each folder now contains exactly that, and the seven are one landing-page row rather than seven.
Deleted:
foundation_tools/A second, different implementation of
data_validation,label_variables,replicationandsurvey_to_codebook, in five-line functions overlapping with the top-level folders. Two unique.spscompanions moved up; the rest is gone.Rebuilt as tested packages (39 tests, all in CI)
data_validation/label_variables/.dtaand.sav, where they survive; round-trips both ways. The old script attached labels todf.attrsand wrote a CSV on the next linesurvey_to_codebook/replication/The three share one dictionary format (
variable,label,values,min,max,allowed,required), so a codebook generated from an XLSForm can be applied as labels and used as validation rules without retyping. Stata, R and SPSS companions are kept beside the Python.Written out
annotated_research/: five illustrative briefs under one eight-tag annotation scheme, ordered weakest design to strongest. Invented programmes, no real citations, stated on every page. ~5,000 words.eval_docs/: logframe template (md + csv), indicator reference sheet, MEL framework outline with a reviewer's question per section, indicator guidance.writing_guides/: results chain, evaluation report structure (with the table of which verb each design permits), policy brief, writing about uncertainty.learning_layers/: learning as scheduled decisions at three tempos.KM_tools/: folder structure template, project README template.Also
SignalStack is dead and is out of the family table; PolicyStack is in it. Emoji stripped from the notes that were kept.
Two things caught on the way
pandas 3 gives text columns a
strdtype, so a blank check keyed ondtype == objectsilently skipped every text column. CI here installs unpinned pandas, so it would have shipped. A test now pins the behaviour on string dtype.The bundled sample XLSForm was defective. It referenced a choice list
hobbiesand defined it ashobby. The old notebook printed an empty choice list without comment; the new tool reported it on first run, and the sample is fixed.Verified locally: all 12 suites pass in Python and R; landing page has no horizontal overflow at 390×844.
🤖 Generated with Claude Code
https://claude.ai/code/session_01DMYhECx2Dzbqad6KJxWZ7t
Generated by Claude Code