feat(cmdb-import): offer the missing-records choice and an Archived quick filter

CHANGELOG.md

@@ -7,6 +7,15 @@
   sheets becomes or updates a module, its vendor, a usage for the municipality
   and a contact person for its owner; a repeat import matches on APPID, so it
   updates instead of duplicating. See `docs/features/cmdb-import.md`.
+- CMDB import, repeat imports: applications that left the CMDB sheets are
+  archived when the sheet "Gearchiveerde Applicaties" lists them and moved to
+  OpenRegister's trash when no sheet does; one that returns is unarchived or
+  restored, never duplicated. The new option `missingRecords` (`archive`, the
+  default, or `keep`) is a choice in the CMDB import section, and the report
+  counts archived, unarchived, deleted and restored applications. The module
+  (0.3.9) and usage (1.5.7) schemas opt into OpenRegister's archive state, and
+  the Applications and Applications in use pages get an "Archived" quick
+  filter. (cmdb-import-archive-reconciliation)
 
 ### Changed
 - EOL feed: the register and schema slugs the feature reads from are corrected.

docs/features/README.md

@@ -158,7 +158,7 @@ The ArchiMate integration maps GEMMA Softwarecatalogus objects to ArchiMate appl
 
 ## CMDB Import
 
-Import a TOPdesk CMDB export (`.xlsx`) for one municipality from the **CMDB import** section of the admin settings. Every application row of the two CMDB sheets ("Onbeh Applicaties CMDB" and "Beheerde Applicaties CMDB") becomes or updates a module, its vendor as a Supplier organisation, a usage that links it to the municipality and records whether maintenance is arranged, and a contact person for its owner (identity in Nextcloud Contacts, never public). A repeat import matches on APPID per municipality, so it updates instead of duplicating, and leaves applications missing from the newer export as they are. The column mapping is declarative JSON executed by OpenRegister's mapping engine.
+Import a TOPdesk CMDB export (`.xlsx`) for one municipality from the **CMDB import** section of the admin settings. Every application row of the two CMDB sheets ("Onbeh Applicaties CMDB" and "Beheerde Applicaties CMDB") becomes or updates a module, its vendor as a Supplier organisation, a usage that links it to the municipality and records whether maintenance is arranged, and a contact person for its owner (identity in Nextcloud Contacts, never public). A repeat import matches on APPID per municipality, so it updates instead of duplicating. By default it archives the applications that moved to the sheet "Gearchiveerde Applicaties" and moves those on no sheet any more to OpenRegister's trash, and brings either back when it returns; the admin can choose to keep them instead. The Applications and Applications in use pages list the archive under the quick filter "Archived". The column mapping is declarative JSON executed by OpenRegister's mapping engine.
 
 See [CMDB import](cmdb-import.md) for the steps, the expected file structure, the error codes and how to adjust the mapping.
 

l10n/en.js

@@ -982,7 +982,7 @@ OC.L10N.register(
         "Import for {name} cancelled after {read} rows.": "Import for {name} cancelled after {read} rows.",
         "Import for {name} finished. {read} rows read: {created} created, {updated} updated, {unchanged} unchanged.": "Import for {name} finished. {read} rows read: {created} created, {updated} updated, {unchanged} unchanged.",
         "Import progress": "Import progress",
-        "Importing a newer export again updates the same applications, matched on APPID per municipality. Applications missing from it are left as they are.": "Importing a newer export again updates the same applications, matched on APPID per municipality. Applications missing from it are left as they are.",
+        "Importing a newer export again updates the same applications, matched on APPID per municipality. Applications missing from it are archived or deleted, unless you choose to keep them.": "Importing a newer export again updates the same applications, matched on APPID per municipality. Applications missing from it are archived or deleted, unless you choose to keep them.",
         "Importing the export…": "Importing the export…",
         "Importing…": "Importing…",
         "APPID": "APPID",
@@ -999,7 +999,7 @@ OC.L10N.register(
         "Reasons and warnings": "Reasons and warnings",
         "Records missing from the export can only be kept.": "Records missing from the export can only be kept.",
         "Reload the page and try again.": "Reload the page and try again.",
-        "Remove sheets the import does not read, or split the export, and try again.": "Remove sheets the import does not read, or split the export, and try again.",
+        "Remove sheets the import does not read and try again. Or split the export and import every part with \"Keep them as they are\".": "Remove sheets the import does not read and try again. Or split the export and import every part with \"Keep them as they are\".",
         "Row": "Row",
         "Row 1 holds the column names. \"APPID\" and \"Applicatie Naam\" are required; column order does not matter.": "Row 1 holds the column names. \"APPID\" and \"Applicatie Naam\" are required; column order does not matter.",
         "Rows": "Rows",
@@ -1017,7 +1017,7 @@ OC.L10N.register(
         "The import failed unexpectedly.": "The import failed unexpectedly.",
         "The import mapping cannot run.": "The import mapping cannot run.",
         "The import reads workbooks with the spreadsheet library that ships with OpenRegister. Make sure OpenRegister is installed and enabled.": "The import reads workbooks with the spreadsheet library that ships with OpenRegister. Make sure OpenRegister is installed and enabled.",
-        "The import was cancelled. The rows processed before it stopped are kept.": "The import was cancelled. The rows processed before it stopped are kept.",
+        "The import was cancelled. The rows and applications processed before it stopped are kept.": "The import was cancelled. The rows and applications processed before it stopped are kept.",
         "The organisation register is not configured, so existing municipalities cannot be listed. You can still type the name of a municipality.": "The organisation register is not configured, so existing municipalities cannot be listed. You can still type the name of a municipality.",
         "The server could not be reached.": "The server could not be reached.",
         "The sheet \"{sheet}\" has no column \"{column}\".": "The sheet \"{sheet}\" has no column \"{column}\".",
@@ -1051,10 +1051,10 @@ OC.L10N.register(
         "Applications missing from the export": "Applications missing from the export",
         "Archive or delete them": "Archive or delete them",
         "Keep them as they are": "Keep them as they are",
-        "An application of this municipality that is on neither CMDB sheet any more is archived when the sheet \"Gearchiveerde Applicaties\" lists it, and moved to the trash when no sheet does. Both disappear from the lists, from OpenCatalogi and from Portaliq; one that returns in a later export is brought back. Without that sheet in the file, applications are only archived. Choose Keep to leave them as they are.": "An application of this municipality that is on neither CMDB sheet any more is archived when the sheet \"Gearchiveerde Applicaties\" lists it, and moved to the trash when no sheet does. Both disappear from the lists, from OpenCatalogi and from Portaliq; one that returns in a later export is brought back. Without that sheet in the file, applications are only archived. Choose Keep to leave them as they are.",
+        "An application of this municipality that is on neither CMDB sheet any more is archived when the sheet \"Gearchiveerde Applicaties\" lists it, and moved to the trash when no sheet does. Both disappear from the lists, from OpenCatalogi and from Portaliq; one that returns in a later export is brought back. When that sheet is missing, has no APPID column, or has an APPID without a cached value, applications are only archived and the result says so. Choose Keep to leave them as they are.": "An application of this municipality that is on neither CMDB sheet any more is archived when the sheet \"Gearchiveerde Applicaties\" lists it, and moved to the trash when no sheet does. Both disappear from the lists, from OpenCatalogi and from Portaliq; one that returns in a later export is brought back. When that sheet is missing, has no APPID column, or has an APPID without a cached value, applications are only archived and the result says so. Choose Keep to leave them as they are.",
         "Applications missing from the export cannot be archived.": "Applications missing from the export cannot be archived.",
         "OpenRegister is missing or too old to archive objects. Choose \"Keep them as they are\" and import again, or update OpenRegister. Nothing was imported.": "OpenRegister is missing or too old to archive objects. Choose \"Keep them as they are\" and import again, or update OpenRegister. Nothing was imported.",
-        "Checking the applications missing from the export: {processed} of {total}": "Checking the applications missing from the export: {processed} of {total}",
+        "Checking the applications of this municipality against the export": "Checking the applications of this municipality against the export",
         "Sheet \"%1$s\" has more than %2$s rows.": "Sheet \"%1$s\" has more than %2$s rows.",
         "Sheet \"%1$s\" has no column \"%2$s\".": "Sheet \"%1$s\" has no column \"%2$s\".",
         "Stackiq is not configured: the register or its schemas cannot be found.": "Stackiq is not configured: the register or its schemas cannot be found.",
@@ -1102,15 +1102,15 @@ OC.L10N.register(
         "The server could not store the uploaded file. The details are in the Nextcloud log.": "The server could not store the uploaded file. The details are in the Nextcloud log.",
         "The file is larger than {size}, the most the import accepts.": "The file is larger than {size}, the most the import accepts.",
         "The file is larger than the server accepts.": "The file is larger than the server accepts.",
-        "A source sheet may hold at most {limit} rows. Split the export and import the parts one after the other.": "A source sheet may hold at most {limit} rows. Split the export and import the parts one after the other.",
-        "Split the export and import the parts one after the other.": "Split the export and import the parts one after the other.",
+        "A source sheet may hold at most {limit} rows. Split the export and import the parts one after the other with \"Keep them as they are\".": "A source sheet may hold at most {limit} rows. Split the export and import the parts one after the other with \"Keep them as they are\".",
+        "Split the export and import the parts one after the other with \"Keep them as they are\".": "Split the export and import the parts one after the other with \"Keep them as they are\".",
         "Excel workbook (.xlsx) with the sheet \"{first}\" or \"{second}\". By default the file may be at most {size}.": "Excel workbook (.xlsx) with the sheet \"{first}\" or \"{second}\". By default the file may be at most {size}.",
-        "The sheets \"{first}\" (applications without arranged maintenance) and \"{second}\" (with arranged maintenance) are read; other sheets, including the \"Invoer\" sheets, are ignored.": "The sheets \"{first}\" (applications without arranged maintenance) and \"{second}\" (with arranged maintenance) are read; other sheets, including the \"Invoer\" sheets, are ignored.",
+        "The sheets \"{first}\" (applications without arranged maintenance) and \"{second}\" (with arranged maintenance) are read, and of the sheet \"{archive}\" only the APPID column; other sheets, including the \"Invoer\" sheets, are ignored.": "The sheets \"{first}\" (applications without arranged maintenance) and \"{second}\" (with arranged maintenance) are read, and of the sheet \"{archive}\" only the APPID column; other sheets, including the \"Invoer\" sheets, are ignored.",
         "The page got no answer from the import.": "The page got no answer from the import.",
         "The connection was cut off before the import answered, so it may still be running or may have finished. Wait a few minutes and check the applications of the municipality before you import again. Importing the same file again creates no duplicates.": "The connection was cut off before the import answered, so it may still be running or may have finished. Wait a few minutes and check the applications of the municipality before you import again. Importing the same file again creates no duplicates.",
         "The import cannot be cancelled now: the server has not started its rows yet, or has already finished them. If the import keeps running, press Cancel import again in a moment.": "The import cannot be cancelled now: the server has not started its rows yet, or has already finished them. If the import keeps running, press Cancel import again in a moment.",
         "The import could not be cancelled: {reason}": "The import could not be cancelled: {reason}",
-        "Cancelling the import. It stops before the next row; the rows already processed stay imported.": "Cancelling the import. It stops before the next row; the rows already processed stay imported.",
+        "Cancelling the import. It stops before the next row or, while the applications are compared with the export, before the next application; what was already processed stays.": "Cancelling the import. It stops before the next row or, while the applications are compared with the export, before the next application; what was already processed stays.",
         "Showing {shown} of {total} rows.": "Showing {shown} of {total} rows.",
         "Show {count} more rows": "Show {count} more rows",
         "The request field \"{field}\" has a value the import does not accept.": "The request field \"{field}\" has a value the import does not accept.",
@@ -1141,14 +1141,21 @@ OC.L10N.register(
         "conflict: the application with this import key is used by another organisation, so it is not changed": "conflict: the application with this import key is used by another organisation, so it is not changed",
         "When on, a re-import overwrites the application's name, descriptions, application type, hosting model, BBN level, source fields and supplier, and the usage's status, phase-out date and business owner, with the values from the export. The usage's TIME classification and internal note are only set when the usage is created or the field is empty, so changes made in stackiq stay.": "When on, a re-import overwrites the application's name, descriptions, application type, hosting model, BBN level, source fields and supplier, and the usage's status, phase-out date and business owner, with the values from the export. The usage's TIME classification and internal note are only set when the usage is created or the field is empty, so changes made in stackiq stay.",
         "An .xlsx file is compressed, so a file of a few MB can be much larger once unpacked. Nothing was imported.": "An .xlsx file is compressed, so a file of a few MB can be much larger once unpacked. Nothing was imported.",
-        "Split the rows over two copies of the export and import them one after the other.": "Split the rows over two copies of the export and import them one after the other.",
+        "Split the rows over two copies of the export and import both with \"Keep them as they are\": an import that archives treats the applications of the other copy as missing.": "Split the rows over two copies of the export and import both with \"Keep them as they are\": an import that archives treats the applications of the other copy as missing.",
         "The sheet \"{sheet}\" is too large to import: unpacked it is {size}, and the import reads at most {limit} of one sheet.": "The sheet \"{sheet}\" is too large to import: unpacked it is {size}, and the import reads at most {limit} of one sheet.",
         "The part {part} of the workbook is too large to import: unpacked it is {size}, and the import reads at most {limit} of one part.": "The part {part} of the workbook is too large to import: unpacked it is {size}, and the import reads at most {limit} of one part.",
-        "Delete the sheets the import does not read from a copy of the export (it reads only \"{first}\" and \"{second}\"), and import that copy.": "Delete the sheets the import does not read from a copy of the export (it reads only \"{first}\" and \"{second}\"), and import that copy.",
+        "Delete the sheets the import does not read from a copy of the export (it reads \"{first}\" and \"{second}\", and the APPID column of \"{archive}\"), and import that copy.": "Delete the sheets the import does not read from a copy of the export (it reads \"{first}\" and \"{second}\", and the APPID column of \"{archive}\"), and import that copy.",
         "The workbook holds more than {count} different texts, the most the import reads.": "The workbook holds more than {count} different texts, the most the import reads.",
         "Together, the cells of the workbook reference more than {size} of shared text, the most the import reads.": "Together, the cells of the workbook reference more than {size} of shared text, the most the import reads.",
         "In development since": "In development since",
-        "In use since": "In use since"
+        "In use since": "In use since",
+        "{count} archived": "{count} archived",
+        "{count} unarchived": "{count} unarchived",
+        "{count} deleted": "{count} deleted",
+        "{count} restored": "{count} restored",
+        "Applications: {changes}.": "Applications: {changes}.",
+        "This tab lists the whole archive. It ignores the GEMMA filters and the search term.": "This tab lists the whole archive. It ignores the GEMMA filters and the search term.",
+        "Applications no longer in the export are archived or moved to the trash. The (i) button above explains when.": "Applications no longer in the export are archived or moved to the trash. The (i) button above explains when."
     },
     "nplurals=2; plural=(n != 1);"
 )