Skip to content

fix(sync): keep UI edits of adopted configs and encrypt secrets before the first push write - #839

Merged
soustruh merged 1 commit into
mainfrom
fix/792-e-f-push-adopted-and-retry
Oct 11, 2026
Merged

soustruh merged 1 commit into
mainfrom
fix/792-e-f-push-adopted-and-retry

Conversation

@soustruh

Copy link
Copy Markdown
Contributor

Refs #792

What was wrong

  • E: sync push overwrote an edit made in the UI. This happened for a config file that has a _keboola.config_id but no manifest entry, for example a config new --push --output-dir scaffold. Sync adopted such a file by its id. It had no base state, so the diff could not see which side changed. It reported modified and push wrote the local file over the remote.
  • E: sync pull also overwrote such a file. A local edit was lost without a message.
  • F: sync push encrypted secrets one config at a time, just before each write. An ENCRYPTION_FAILED stopped the push after some configs were already created. The manifest did not record them. The next sync push created them again.

What changed

  • config new --push --output-dir writes _keboola.base_config_hash into the _config.yml footer. The value is the hash of the created config as the API returned it.
  • The diff uses this hash as the base of an adopted file. An adopted file without it is conflict when it differs from the remote. It is never modified. Two files that adopt the same id are both conflict.
  • Push skips these conflicts, like all other conflicts. After a successful push the manifest holds the base, and push removes base_config_hash from the file.
  • sync pull keeps an adopted file with local changes and reports it as skipped. sync pull --theirs overwrites it.
  • sync push encrypts all secrets of the changes it will write before the first write. It makes one Encryption API call per component. The writes use the encrypted values from that call. If encryption fails, push writes nothing, and the error message ends with Nothing was pushed.
  • --allow-plaintext-on-encrypt-failure keeps its old behavior. Push does not encrypt in advance with this flag.
  • The encrypt-first step reads every added or modified config and row in the branch tree. This includes keboola.sandboxes workspace configs in a syncWorkspaces tree.
  • Name-drift detection moved from sync_service.py to _sync_push_ops.py to stay in the file-size budget. Its behavior did not change.
  • Docs: commands-reference.md, sync-workflow.md, gotchas.md, scaffold-workflow.md, the kbagent context text and formal/sync/README.md describe the new behavior with vNEXT tags.

Open questions

  • E has no "local wins" option. To push an adopted file that is in conflict, the user must run sync pull, which keeps the local file, and then resolve the conflict by hand, or use sync pull --theirs to take the remote. Do we need a flag that pushes the local file anyway?
  • The base state is in the file (_keboola.base_config_hash), not in the manifest. config new --push --output-dir can write into a directory that has no manifest, or into a tree that does not track the config yet. The file is the only place that always exists. After the first push the base moves to the manifest. Should config new add a manifest entry instead when a manifest exists?
  • A conflict from E goes to skipped. It is not in errors[], so the push exits 0, like every other skipped conflict since fix(cli): partial failures set the exit code and the headline (CLI-6) #747. Is this correct for E?

Tests

  • tests/test_sync_formal_counterexamples.py: the replay tests for E and F were xfail. They now pass without the marker.
  • tests/test_sync_adopted_files.py: diff, push and pull of adopted files with and without a base, a remote edit, edits on both sides, a placeholder scaffold, and two files with one id.
  • tests/test_sync_push_encrypt_first.py: an encryption failure stops the push before any write. A retry applies each change once. Each secret is encrypted once, and the writes send the ciphertext. Rows are covered. A value the API leaves in plaintext is not cached. The plaintext fallback flag skips the step.
  • Updated tests in test_sync_diff_engine.py, test_sync_service.py, test_sync_reconcile.py, test_component_service.py and test_config_create_cli.py.
  • make check passes.

@keboola-pr-reviewer-bot keboola-pr-reviewer-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: needs_human (risk 3/5) · profile keboola-mcp-server

Well-tested sync bug-fix, but it reshapes the push/pull write and encryption-ordering paths, which warrants a human owner.

Impact flags: possible rollback re-introduction — see Check Run summary.

Concerns:

  • src/keboola_agent_cli/services/_sync_push_ops.py: Encrypt-first changes push failure semantics on the config write path
  • src/keboola_agent_cli/sync/diff_engine.py: Adopted files without baseline now classified conflict; author questions exit-0 behavior
  • src/keboola_agent_cli/services/sync_service.py: pull now skips untracked local-edited files; verify no legitimate overwrite regressed

@soustruh soustruh left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No findings. Checked:

  • E fix: an adopted file without _keboola.base_config_hash is a conflict in diff. A scaffold from config new --push records the hash and gets a 3-way diff. Two files with one id are both conflicts. Pull keeps an adopted file with local changes, and --theirs overwrites it. Push removes the key after the first write. Targeted tests in tests/test_sync_adopted_files.py and tests/test_sync_diff_engine.py pass.
  • F fix: encrypt_secrets runs before the first write and covers configs and rows. The deletes in push --force (including syncWorkspaces deletes) run after it, so a failure writes nothing. --allow-plaintext-on-encrypt-failure skips the pre-encryption, as the docs say. ENCRYPTION_FAILED still maps to exit code 1, so the #747 exit codes do not change. tests/test_sync_push_encrypt_first.py asserts an empty API log and an unchanged manifest and files on failure.
  • Docs (CLAUDE.md, context.py, gotchas.md, sync and scaffold workflows, commands reference, expert agent) match the code and carry vNEXT tags.
  • Ran the changed test files locally with a temporary config dir: 188 passed, 2 xfailed. I did not run the full make check.

@soustruh
soustruh merged commit e0dd40c into main Oct 11, 2026
5 checks passed
@soustruh
soustruh deleted the fix/792-e-f-push-adopted-and-retry branch October 11, 2026 22:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants