Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/labeler.yml
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,10 @@ converter/wisdom:
- changed-files:
- any-glob-to-any-file: 'converters/wisdom/**'

converter/sap-csn:
- changed-files:
- any-glob-to-any-file: 'converters/sap-csn/**'

converter/sigma:
- changed-files:
- any-glob-to-any-file: 'converters/sigma/**'
Expand Down
73 changes: 73 additions & 0 deletions .github/workflows/converter-sap-csn-ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
#

name: Converters SAP CSN CI

on:
push:
branches: [ "main" ]
paths:
- 'converters/sap-csn/**'
- 'core-spec/**'
- 'examples/**'
- 'python/**'
- '.github/workflows/converter-sap-csn-ci.yml'
pull_request:
branches: [ "main" ]
paths:
- 'converters/sap-csn/**'
- 'core-spec/**'
- 'examples/**'
- 'python/**'
- '.github/workflows/converter-sap-csn-ci.yml'

permissions:
contents: read

jobs:
build:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: ["3.11", "3.12", "3.13", "3.14"]

steps:
- name: Checkout project
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: ${{ matrix.python-version }}

- name: Install uv
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
with:
version: "0.12.19"

- name: Sync dependencies
working-directory: converters/sap-csn
run: |
uv sync

- name: Unit Tests
working-directory: converters/sap-csn
run: |
uv run pytest
1 change: 1 addition & 0 deletions converters/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,7 @@ The Ossie specification currently defines extensions for the following vendors:
| `CUBE` | Cube data model |
| `HEX` | Hex semantic project |
| `HOLISTICS` | Holistics AML dataset |
| `SAP_CSN` | SAP CSN (Core Schema Notation) document |

[`wisdom-semantic-view`](wisdom-semantic-view/README.md) is a separate, fixture-scoped spoke. It reads one Snowflake semantic view, writes one Ossie document, writes a Wisdom domain-export JSON (format `1.0`), and reverses that path. It keeps metric synonyms and `module_custom_instructions.sql_generation`. It does not call `converters/snowflake` (Cortex Analyst export only) or `converters/wisdom` (that spoke drops synonyms and field or metric `ai_context`).

Expand Down
113 changes: 113 additions & 0 deletions converters/sap-csn/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

# Ossie ⇄ SAP CSN converter

Converts between a SAP **CSN** document (Core Schema Notation, the JSON that
`cds compile --to csn` writes and SAP Datasphere exchanges) and an Apache Ossie
semantic model, in both directions. It works offline on files: it does not call
SAP.

```
ossie-sap-csn csn-to-ossie -i model.csn.json -o semantic_model.yaml [--name NAME]
ossie-sap-csn ossie-to-csn -i semantic_model.yaml -o model.csn.json
```

Warnings (information the target cannot hold, or that was set aside) go to stderr,
one per line, as `[WARNING] <TYPE>: <element> — <detail>`. Input that cannot be
converted at all exits with status 1 and a message.

```
uv sync
uv run pytest
```

## CSN → Ossie

| CSN | Ossie |
|-----|-------|
| `entity` (table or view) | dataset; `source` is `@cds.persistence.name`, else the CSN name; the name drops the namespace unless two entities would collide |
| element | field; `type` resolves through user-defined types to a CDS built-in, then to an Ossie `datatype` |
| `key: true` | `primary_key` |
| `@assert.unique.*` | `unique_keys` |
| calculated element (`value`) | field expression, translated from CXN to ANSI SQL |
| `Association` / `Composition` with a target in the document | relationship, from the owning entity to the target |
| managed association | relationship, plus a field for each generated foreign-key column (`<assoc>_<key>`) |
| `@Aggregation.default` (`SUM`, `MIN`, `MAX`, `AVG`, `COUNT`, `COUNT_DISTINCT`) | metric over the element |
| `@Aggregation.default` + `@Ossie.metric: true` | metric over the element's expression; no field |
| `@EndUserText.label`, `@Common.Label` | `label` |
| `doc`, `@EndUserText.quickInfo`, `@Common.QuickInfo` | `description` |
| a temporal type, or `@Semantics.businessDate` / `calendar` / `fiscal` | `dimension.is_time: true` |

Everything else — remaining annotations, `length`/`precision`/`scale`, the `query`
or `projection` of a view, types, services, actions and the root keys — is kept in a
`SAP_CSN` custom extension so the document exports back unchanged.

## Ossie → CSN

| Ossie | CSN |
|-------|-----|
| dataset | `entity` named by the `SAP_CSN` extension, else the dataset name |
| field | element; `datatype` → `cds.String`, `Integer`, `Decimal`, `Double`, `Boolean`, `Date`, `Time`, `Timestamp` |
| field expression other than its own column | calculated element (`value`), translated from SQL to CXN |
| relationship | association on the `from` entity: managed (`keys`) when its foreign-key fields were generated by one, otherwise an `on` condition. A composite key becomes an `and` of equalities |
| metric, one aggregate over a column of the same name | `@Aggregation.default` on that element |
| metric, one aggregate over an expression | derived measure element (`value` + `@Aggregation.default` + `@Ossie.metric`) |
| `unique_keys` | `@assert.unique.<name>` |

A derived measure may reach other datasets through relationships (`SUM(orders.qty * book.price)`),
because a relationship runs from the many side to the one side and so cannot fan out the rows
being aggregated.

### What CSN has no place for

* **Metrics CSN cannot express** — `COUNT(*)`, ratios of two aggregates, window functions,
expressions over datasets no relationship connects — are written to `meta.ossie.metrics` on the
CSN root and reported as `METRIC_NOT_MODELED`. CSN consumers ignore `meta.ossie`; this converter
reads it back.
* **AI context and other vendors' custom extensions** ride in `@Ossie.aiContext` and
`@Ossie.customExtensions` annotations. Model-level ones go to `meta.ossie.model`.
* **A field expression CXN cannot hold** (a window function, a subquery) is exported as a plain
column with the original text in `@Ossie.sql`, and reported as `EXPRESSION_NOT_TRANSLATABLE`.
* **A key naming a column the dataset does not declare** is carried whole in `@Ossie.primaryKey`
or `@Ossie.uniqueKeys`, and reported.
* **A field with no `datatype`** is exported as `cds.String`, because CSN requires a type, and
reported as `ASSUMED_DATATYPE`.

Only the `ANSI_SQL` dialect is read. A field or metric that has only another dialect is reported
as `DIALECT_UNAVAILABLE` rather than guessed at.

## Limitations

* **CAP and Datasphere CSN.** The converter reads the annotations both write for analytics
(`@Aggregation.default`, `@Analytics.dataCategory`, `@EndUserText.label`). It does not interpret
Datasphere's analytic-model objects (fact sources, attribute and measure groups, variables) as
such; they arrive as plain entities with their annotations preserved.
* **Not translated:** structured and array-typed elements, virtual elements, `$self` backlink
associations (the forward association already is the relationship), associations whose target is
not in the document, `on` conditions other than a conjunction of column equalities, and
expressions with subqueries, `$session` or `$user`. Each is kept in the `SAP_CSN` extension and
reported, never dropped silently.
* **Views.** A view's elements become fields and its `query` is preserved, but the converter does
not interpret the query; the dataset `source` is the view's own name.
* **Export validity.** The converter does not run the CDS compiler. A CSN it writes follows the
CSN schema but has not been compiled with `cds`.
* **Conversion is not a CDS or SQL migration.** Expressions are translated between SQL and CXN
only for the constructs both express: references, literals, arithmetic, comparison, `AND`/`OR`/`NOT`,
`IN`, `LIKE`, `BETWEEN`, `IS NULL`, `CASE`, `CAST` and plain function calls.
75 changes: 75 additions & 0 deletions converters/sap-csn/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[dependency-groups]
dev = [
"pytest>=8.0",
# So the core-spec schema validation in the tests runs rather than skipping;
# the converter itself does not need it.
"jsonschema>=4.0",
]

[project]
name = "apache-ossie-sap-csn"
version = "0.2.0.dev0"
description = "SAP CSN (Core Schema Notation) <> Apache Ossie converter"
authors = [{ name = "Apache Software Foundation", email = "dev@ossie.apache.org" }]
requires-python = ">=3.11"
readme = "README.md"
license = "Apache-2.0"
keywords = [
"Apache Ossie",
"Ossie",
"SAP",
"CSN",
"CDS",
"Datasphere",
]
dependencies = [
"apache-ossie>=0.2.0.dev0",
"PyYAML>=6.0",
# Translating calculated-element and measure expressions between SQL and CXN.
"sqlglot>=20.0",
]

[project.scripts]
ossie-sap-csn = "ossie_sap_csn.cli:main"

[project.urls]
homepage = "https://ossie.apache.org/"
repository = "https://github.com/apache/ossie/"

[tool.hatch.build.targets.wheel]
packages = ["src/ossie_sap_csn"]

[tool.pytest.ini_options]
testpaths = ["tests"]

[tool.uv]
required-version = ">=0.9.0"
default-groups = [
"dev"
]

# apache-ossie is not yet published to PyPI; resolve it from the in-repo
# package for now. Remove this block once apache-ossie is published to PyPI.
[tool.uv.sources]
apache-ossie = { path = "../../python", editable = true }
21 changes: 21 additions & 0 deletions converters/sap-csn/src/ossie_sap_csn/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

from ossie_sap_csn.csn_to_ossie import CsnToOssieConverter
from ossie_sap_csn.ossie_to_csn import OssieToCsnConverter

__all__ = ["CsnToOssieConverter", "OssieToCsnConverter"]
Loading
Loading