Repository navigation
datasets(traces): add memory_read_write trace — derived summary outlives column entitlement - #102
Conversation
…ing column entitlement First entry in the memory_read_write category. Models an agent writing a summary derived from a restricted column to a workspace-shared memory namespace without source lineage, and a non-entitled user later receiving it via semantic retrieval. Maps to DSGAI01 (primary), DSGAI11, DSGAI07, DSGAI15, DSGAI18. Tier: derived_from_public_documentation. Regenerated index.csv and updated the index.csv sha256 in croissant.json.
emmanuelgjr
left a comment
There was a problem hiding this comment.
Thanks @sangaraju1988, this is a strong first entry for memory_read_write. That category is in the README but had no entries until now, and this pattern is new to the dataset: every access check passes, and the leak happens only at the step where the summary is derived and stored. The span-level findings are well placed. The mitigations are concrete. Both evidence sources check out, and the PostgreSQL quote matches the docs word for word.
I ran the checks locally on your branch. run_all_checks.py passes, validate.py passes, and index.csv matches what build_index.py generates. Two changes are needed before merge:
1. Croissant metadata drift (CI will fail)
The index.csv sha256 you updated by hand is correct. But python data_validation/croissant/build_croissant.py --check still fails because dateModified wasn't updated:
- "dateModified": "2026-09-15",
+ "dateModified": "2026-10-12",You can edit that line in datasets/agentdataflow_toolexchange_traces/croissant.json directly or rerun the script. It's the only difference.
2. Remove the self-reference from notes
Thanks for disclosing your related work, and for not citing it as evidence. We'd like the disclosure to stay in the PR description and out of the merged file. Our rule is that merged dataset files credit the contributor only through the contributor field and don't point to the contributor's own work. Please drop the final sentence of notes, starting with "Mitigation 2 follows the general derivation-aware access-control pattern; the contributor has published…". The scope and limitations text before it is good and should stay.
Once those two changes are in, I'll merge. We'd also welcome the adversarial variant you mentioned, where a user deliberately phrases questions to pull restricted items from a shared namespace, as a follow-up PR. It would pair well with this one.
…fresh croissant dateModified
e861fb2 to
c794798
Compare
|
Thanks @emmanuelgjr for running the checks locally and for the clear review. Both changes are in (c794798): I removed the final sentence of notes (the disclosure stays in the PR description) and regenerated croissant.json, so dateModified is now 2026-10-12. Locally, build_croissant.py --check, run_all_checks.py and validate.py all pass. The workflows are waiting on maintainer approval since this comes from a fork. I’ll follow up with the adversarial shared-namespace variant as a separate PR. |
emmanuelgjr
left a comment
There was a problem hiding this comment.
Both changes verified, and CI is green. Thanks, merging.
Summary
Adds the first entry in the
memory_read_writecategory ofdatasets/agentdataflow_toolexchange_traces:entries/DSGAI-TRACE-derived-summary-memory-outlives-column-entitlement.jsonAn analytics agent answers an entitled user's question through a governed warehouse tool that enforces column-level access. The agent then saves its summary to a workspace-shared long-term memory namespace. The memory item records namespace, key and value but drops the source lineage the tool returned. Four days later, a workspace member without the column entitlement asks a related question. Semantic search returns the stored summary, and the only check is namespace membership, so the restricted figures reach that user. Some of them are small-group aggregates that come close to individual values.
Every guard passes. The failure only exists across the derivation step.
unintentional_failurederived_from_public_documentation, backed by two independently checkable sources:security_invoker) is opt-in. This is the established precedent that derived artifacts aren't automatically bound to the reader's rights on their sources.notes.Not covered here: the adversarial variant, where a user deliberately phrases questions to pull high-sensitivity items from a shared namespace. That should be a separate entry, and I'm happy to follow up with it.
DSGAI mapping
DSGAI01 (primary), DSGAI11, DSGAI07, DSGAI15, DSGAI18.
Also LLM02:2025 / LLM02:2026 and ASI03:2026.
Checklist
python data_validation/run_all_checks.py). The dataset'svalidate.pyalso passes.Note: I regenerated
index.csvwithbuild_index.pyand updated theindex.csvsha256 incroissant.jsonby hand, using the same LF-normalized hashing asbuild_croissant.py. I couldn't installmlcroissantlocally to runbuild_croissant.py --check. If CI flags drift, I'll rerun it.