Skip to content

fix(harvester): allow sandbox create of missing CDS DOI records - #987

Open
TahaKhan998 wants to merge 1 commit into
CERNDocumentServer:masterfrom
TahaKhan998:fix/harvester-sandbox-allow-cds-doi-create
Open

TahaKhan998 wants to merge 1 commit into
CERNDocumentServer:masterfrom
TahaKhan998:fix/harvester-sandbox-allow-cds-doi-create

Conversation

@TahaKhan998

Copy link
Copy Markdown

closes #978

in sandbox the matcher finds nothing for records that already exist on prod (they have a CDS DOI). create then gets blocked.
added CDS_HARVESTER_ALLOW_CDS_DOI_CREATE (off by default). when on, allow create and keep the same CDS DOI. sandbox config sets this to True.

@TahaKhan998
TahaKhan998 force-pushed the fix/harvester-sandbox-allow-cds-doi-create branch 2 times, most recently from a71bcdf to 5d07379 Compare September 21, 2026 11:17

def check(self, stream_entry, *, record=None, record_pid=None, matcher=None):
"""Return an error if the entry DOI uses the CDS DataCite prefix."""
if current_app.config["CDS_HARVESTER_ALLOW_CDS_DOI_CREATE"]:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't this also apply for cds-rdm identifier as we discussed, not only the DOI?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we have CDS-RDM only, no doi, is the new record created with the same PID assigned to the record and the parent(this might come later if we don't receive parent pids from inspire yet)?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the config covers cdsrdm now too. if inspire sends a cdsrdm id and the config is true, we reuse it as the parent recid on create. the version still gets a new id because parent and version cant be the same recid.
if its cdsrdm only with no doi, create already went through before. with the config true we also keep that same parent id instead of minting a new one.

renamed the config to CDS_HARVESTER_ALLOW_MISSING_CDS_CREATE so it is not doi only.

@TahaKhan998
TahaKhan998 force-pushed the fix/harvester-sandbox-allow-cds-doi-create branch 2 times, most recently from cb21b9d to a67df73 Compare September 22, 2026 13:41
minting a new parent id (sandbox create of a prod record).
"""
draft = current_rdm_records_service.create(self.identity, data=entry)
if parent_pid:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are you 100% sure that what we receive from inspire is the parent pid? or this is the record pid?
we can't do this create operation in production though, if there is any discrepancy between CDS and INSPIRE (wrong PID for example), this will start creating wrong records in prod, it should be only in non-prod environments.
in addition, please use the unit of work to encapsulate these operations as one atomic one

@TahaKhan998
TahaKhan998 force-pushed the fix/harvester-sandbox-allow-cds-doi-create branch 4 times, most recently from 2d550ff to f7f5e19 Compare September 23, 2026 13:51
@TahaKhan998
TahaKhan998 force-pushed the fix/harvester-sandbox-allow-cds-doi-create branch from f7f5e19 to 7af18ff Compare September 24, 2026 12:17

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

harvester: allow create in sandbox when the CDS record is not in the local DB

2 participants