Repository navigation
OrionBelt converter drops a field's logical datatype in both directions #409
Description
Summary
Neither conversion direction carries a field's logical type through the
canonical spec datatype field:
- Ossie->OBML ignores the declared
datatypeand derives OBML
abstractTypefrom a column-name heuristic, so aStringfield can be
emitted as numeric and vice-versa (wrong type). - OBML->Ossie never writes the spec
datatypefield at all - the type is
stashed only inside an OrionBeltcustom_extensionsblob (missing type).
A round-trip appears lossless only because the converter is self-consistent
with its own non-spec key; any other spec-compliant tool sees wrong or absent
types.
Root cause
Ossie->OBML (ossie_to_obml.py):
ossie_type = field.get("data_type", "") # (1) wrong key
if ossie_type and ossie_type in OSSIE_TO_OBML_TYPE: # (2) wrong-cased map
abstract_type = OSSIE_TO_OBML_TYPE[ossie_type]
else:
abstract_type = self._infer_obml_type(field)- Wrong key. The spec field is
datatype, no underscore, so the branch is dead - Wrong-cased map. The spec enum is capitalized, but
OSSIE_TO_OBML_TYPEis keyed lowercase with different terms.
Even after fixing the key, case-normalizing is not enough:
DateTime/DateTimeTz/Decimal/Floathave no key - so case-normalizing is not enough
OBML->Ossie: canonical field never written (obml_to_ossie.py).
The type goes only into a vendor extension, under the non-spec key
data_type with a lowercase value; field["datatype"] is not set.
Illustrative Repro
# Ossie -> OBML: declared datatype ignored
for f, want in [({"name": "total_amount", "datatype": "String"}, "string"),
({"name": "customer_id", "datatype": "Integer"}, "int")]:
_, col = OssietoOBML(ossie={})._convert_field(dict(f))
got = col["abstractType"]
print(f"fwd {f['datatype']:8} -> {got:8} (want {want}) [{'OK' if got==want else 'BUG'}]")
# OBML -> Ossie: canonical datatype never emitted
field = OBMLtoOssie(obml={})._convert_column("customer_id", {"code": "customer_id",
"abstractType": "int"}, "Orders", {})
print("rev canonical 'datatype' present?", "datatype" in field, "[BUG]" if "datatype" not in field else "")Why this survived
The existing fixtures (test_ossie_converter_vendors, test_ossie_metric_no_silent_loss)
use the same wrong key data_type, so the suite stays green with the bug present.
Drop-in test (currently fails). Suggested home:
tests/test_ossie_metric_no_silent_loss.py, which already guards this "no
silent loss on field conversion" path.
def test_explicit_datatype_roundtrips():
_, col = OssietoOBML(ossie={})._convert_field({"name": "total_amount", "datatype": "String"})
assert col["abstractType"] == "string" # fwd: currently 'float'
field = OBMLtoOssie(obml={})._convert_column("total_amount",
{"code": "total_amount", "abstractType": "string"}, "Orders", {})
assert field.get("datatype") == "String" # rev: currently missingSuggested fix
- Ossie->OBML: read
field.get("datatype")(fix the comment); rebuild
OSSIE_TO_OBML_TYPEon the spec enum - OBML->Ossie: set the canonical
field["datatype"](capitalized enum value)
in addition to any extension bookkeeping.
Activity
christianeu-db commentedon Sep 17, 2026
cc @ralfbecher if you'd like to a take a first look. otherwise, I'm happy to take a pass
ralfbecher commentedon Sep 17, 2026
Thanks for the detailed report, confirmed on
main. This is already fixed in the downstreamosi-orionbeltpackage (reads/emits the canonicaldatatypeon fields and metrics, maps the fullDataTypeenum, keeps the extension only for the lossless OBML round trip); it just was never ported back here. I'll open a PR porting it with your drop-in test. Since #396 touches the same files, happy to either land first or rebase onto it, whichever the reviewers prefer.christianeu-db commentedon Sep 17, 2026
Wow - thanks for the very speedy turnaround! I'd bias towards getting in this change early since it is a data loss issue but defer to @jbonofre, esp since JB is reviewing the large refactor pr & is working on other converters
fix(orionbelt): carry field and metric datatype in both directions
fix(orionbelt): carry field and metric datatype in both directions (a…