Skip to content

un_fao historical delivery carries wire vocabulary (lr_ged_*) — the consumer rename reached the forecast and not the historical file #305

Description

@Polichinel

What

The delivered historical artefact for the un_fao consumer carries its three value columns as
lr_ged_sb, lr_ged_ns, lr_ged_os — the wire vocabulary. The forecast artefacts carry
consumer-facing names (sb_map, ns_hdi90_upper, …). An FAO engineer joining the two files gets
matching keys and mismatched value-column conventions.

Verified against the newest delivery — historical_dataset_20260813_080043.parquet, the 13 August
re-delivery after the empty-store incident: 28,421,738 rows, 14 columns, lr_ged_* present. Also
against a copy downloaded 2026-09-09. Both carry the wire names.

This is not legacy drift — the wire name is deliberate

The register's ADR-013 adoption record states: target vocabulary decided: lr_ged_sb/ns/os,
producers rename at publish (models#146)
. views-datafactory supplies ged_sb_best; the
un_fao queryset config maps it deliberately
(views-models/postprocessors/un_fao/configs/config_queryset.py:57-59). The producer side is
correct and should not change
— the name flows into the run manifest and every shard filename.

What is actually missing

The consumer-side rename at the historical write boundary.

D-06 (RESOLVED) settled the principle: the internal → consumer rename "belongs in views-faoapi as
a response-formatting step, coordinated with FAO."
That is built and shipped for the forecast —
json_contract.to_consumer_columns is applied in forecast/serialize/bulk_parquet.py,
forecast/serialize/grid_parquet.py, and the hdi-map route. It is applied on no historical
path
. wire_reader.py records the intended design: "the served target keeps the wire vocabulary;
mapping to the consumer name happens at the boundary"
— and the historical boundary has no mapping.

Write site: views_postprocessing/unfao/managers/unfao.py:440 writes the artefact with the frame's
columns unchanged.

Why it matters beyond tidiness

FAO Release Note 06 (drafted, not yet transmitted) commits in Topic B that consumer-facing
columns "omit the internal VIEWS pipeline prefixes — source (ged_), scale/transform (lr_, and
the deprecated ln_), and model-output (pred_)"
, and states the deprecated prefix "must not be
read as log space."
The forecast honours that commitment. The historical file does not.

Pre-Release Note 07 (2026-09-14) now names the three columns explicitly so an FAO engineer is not
surprised. That is honest, but it leaves the release note's commitment wider than the implementation.

Decision needed before implementation

  1. Rename at this boundary, mirroring what faoapi does for the forecast. Small; makes RN 06's
    commitment true; changes the delivered historical schema, so it wants doing deliberately between
    deliveries and announcing to FAO.
  2. Scope the commitment instead — keep the historical file on wire vocabulary and narrow RN 06's
    naming paragraph to the forecast columns. No code; costs a paragraph of explanation in every
    future note that touches the schema.

Either is defensible. Option 1 is the smaller long-run cost if the historical schema is going to be
described to a partner repeatedly.

Not in scope

  • Changing the producer-side name or the wire vocabulary (deliberate, ADR-013).
  • The forecast rename (built and shipped).
  • Identity/geography column naming — that is D-06's original subject and is separately resolved.

Register: C-242 (Tier 3) in reports/technical_risk_register.md.
Cross-refs: D-06 (resolved principle), C-24 (consumer-facing contract divergence),
views-faoapi forecast/serialize/json_contract.py (the implemented half).
Found while reviewing FAO Release Note 06 against the delivered artefacts.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs-decisionRequires a human decision before proceedingstoryA single reviewable unit of an epic

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions