Repository navigation
Conversation
Offline conversion between a SAP CSN document (Core Schema Notation) and an Apache Ossie semantic model, in both directions. CSN to Ossie: entities become datasets, elements become fields, key and @assert.unique become keys, associations and compositions become relationships (managed ones also yield their generated foreign-key fields), calculated elements become SQL expressions, and @Aggregation.default becomes a metric. Everything with no Ossie equivalent is kept in an SAP_CSN custom extension so the document exports back unchanged. Ossie to CSN: relationships become managed or on-condition associations, and a metric that aggregates one dataset's columns (optionally reaching others through relationships) becomes a measure element. Metrics CSN cannot express travel in meta.ossie.metrics, AI context and other vendors' extensions in @ossie.* annotations, and each such case is reported as a warning. Also adds the CI workflow and the labeler entry, and lists SAP_CSN in the converters README.
| raise UnsupportedExpression("bare $self reference") | ||
| if ref[0].startswith("$"): | ||
| raise UnsupportedExpression(f"{ref[0]!r} has no SQL rendering") | ||
| if len(ref) == 1: |
There was a problem hiding this comment.
Identifiers are rendered unquoted, so an element whose name is a SQL keyword produces an Ossie expression that is not valid SQL, with no issue reported. A CSN entity with elements from (plain) and a calculated element from + 1:
$ uv run validation/validate.py out.yaml
Validation FAILED with 2 error(s):
[SQL] Field 'Orders.from' in model 'sap_csn_model' (ANSI_SQL): Expected table name but got
<Token token_type: TokenType.SENTINEL, ...>. Line 1, Col: 11.
[SQL] Field 'Orders.plus' in model 'sap_csn_model' (ANSI_SQL): Expected table name but got
<Token token_type: TokenType.PLUS, text: +, ...>. Line 1, Col: 13.
A calculated element select + 1 is the worse case: it raises no validator error at all, because it re-parses as SELECT 1 — a different expression. Rendering references through sqlglot, or quoting them, would fix both.
| shorts = [n.rsplit(".", 1)[-1] for n in self.csn_names] | ||
| unique = len(set(shorts)) == len(shorts) | ||
| self.dataset_name = { | ||
| n: (s if unique else n.replace(".", "_")) for n, s in zip(self.csn_names, shorts) |
There was a problem hiding this comment.
The flattened form can itself collide, and this is an import bug rather than only an export one: the document produced already fails the validator.
definitions: a.Item, b.Item, foo.bar, foo_bar
dataset names: ['a_Item', 'b_Item', 'foo_bar', 'foo_bar'] import issues: none
$ uv run validation/validate.py out.yaml
[Unique] Duplicate dataset name 'foo_bar' in model 'sap_csn_model'
Export then raises ConverterError: the document declares two datasets with the same name. Import should disambiguate until the dataset names are unique.
| if isinstance(value, bool): | ||
| return "TRUE" if value else "FALSE" | ||
| if isinstance(value, (int, float)): | ||
| return repr(value) |
There was a problem hiding this comment.
json.loads accepts the non-standard NaN, Infinity and -Infinity, so a CSN document holding one yields 'inf' / 'nan' as SQL text. A math.isfinite guard raising UnsupportedExpression keeps it with the other refusals.
| entity_name: dict[str, str], | ||
| issues: list[ConverterIssue], | ||
| ) -> dict[str, Any]: | ||
| entity: dict[str, Any] = {"kind": "entity"} |
There was a problem hiding this comment.
dataset.source is never read; @cds.persistence.name survives only because the entity extension preserved the annotation. Two consequences:
no SAP_CSN extension, source=tpcds.public.store_sales
-> @cds.persistence.name ABSENT, issues: none (lost)
CSN import, then source edited to warehouse.other_name
-> @cds.persistence.name: tpcds.public.store_sales (silently stale)
The second matters more: the export contradicts the document it was given. The TPC-DS round-trip test stays green because it doesn't assert source.
| annotate: dict[tuple[str, str], tuple[OssieMetric, str]] = {} # (dataset, field) -> (metric, aggregation) | ||
| derived: dict[str, list[tuple[OssieMetric, _Metric]]] = {n: [] for n in datasets} | ||
| in_meta: list[OssieMetric] = [] | ||
| element_names = {n: {f.name for f in (d.fields or [])} for n, d in datasets.items()} |
There was a problem hiding this comment.
Field-name collisions route a metric to meta.ossie.metrics with a warning, but planned association names aren't in this set, and associations are written into the same elements dict first. A metric orders_cust with SUM(orders.cust_id) alongside a relationship of that name:
orders_cust: {"type":"cds.Integer","value":{"ref":["cust_id"]},"@Aggregation.default":{"#":"SUM"},"@Ossie.metric":true}
still an association? False issues: none
Including {p["name"] for p in plans[n]} here treats them like field names.
Quote identifiers that are SQL keywords, treat a path through several associations to one target as ambiguous, keep flattened dataset names unique, strip ANSI escapes from parse errors, refuse non-finite literals, keep key on managed associations, export dataset.source as the persistence name, and stop measures taking an association's name.
|
@ayushtkn thank you so much for the review! I think I've addressed most of your comments. |
2 participants
Summary
Adds
converters/sap-csn: an offline, bidirectional converter between a SAP CSN document (Core Schema Notation, as written bycds compile --to csnand exchanged by SAP Datasphere) and an Ossie semantic model.keyand@assert.uniquebecome keys. Associations and compositions become relationships. Calculated elements become SQL expressions.@Aggregation.defaultbecomes a metric. Whatever Ossie has no home for is kept in anSAP_CSNcustom extension, so a CSN to Ossie to CSN round trip is lossless.on-condition associations. A metric that aggregates one dataset's columns, optionally reaching others through relationships, becomes a measure element. Metrics CSN cannot express go tometa.ossie.metrics. AI context and other vendors' extensions go to@Ossie.*annotations. Every such case is reported as a warning.Limitations (Datasphere analytic-model objects, views' queries, structured elements,
$selfbacklinks) are listed in the converter README. The exporter does not run the CDS compiler.Also adds the CI workflow and the
labeler.ymlentry, and listsSAP_CSNinconverters/README.md.Note:
converters/README.mdsays new converters belong inapache/ossie-converters, but recent ones (e.g. Holistics, #538) were still added here. This PR touches nothing outsideconverters/sap-csn/apart from the labeler, the workflow and one README row, so it can move to that repository if preferred.Related Issues
None.
Checklist
.github/labeler.ymlis updated with the new path