Skip to content

fix(toolkit): keep db mv placement when lite merge replaces archive dbs - #155

Open
halibobo1205 wants to merge 1 commit into
release_v4.8.3from
feature/dblite-merge-moved-db
Open

halibobo1205 wants to merge 1 commit into
release_v4.8.3from
feature/dblite-merge-moved-db

Conversation

@halibobo1205

@halibobo1205 halibobo1205 commented Oct 12, 2026 •

Copy link
Copy Markdown
Owner

Follow-up to tronprotocol/java-tron#6946 (comment).

The linked review reported that db lite merge follows the database symlinks created by db mv and clears the relocated directories. This PR addresses that combined workflow by preserving the relocated storage locations during merge and adding regression coverage for the command sequence.

What does this PR do?

Preserve the database placement established by db mv when db lite merge restores archive data.

For relocated archive databases, merge replaces the contents of their actual storage directories while preserving the original symlinks and destination directories. The merged databases remain at the locations configured in storage.properties.

Failures from the new relocation helpers propagate to the CLI as a nonzero exit code. Archive databases that were not relocated retain their existing handling.

Why are these changes required?

A valid operational sequence is to relocate a LiteNode’s archive databases to a larger disk before restoring full history. Currently, db lite merge can undo that relocation while leaving the node configuration unchanged.

Reproduction scenario

Consider a LiteNode whose database is initially stored at /ssd/tron/output-directory/database. The operator wants to restore its block and transaction history while keeping the growing archive databases on HDD.

The example assumes:

  • storage.db.directory is database.
  • The LiteNode has its snapshot metadata and existing block and trans databases.
  • /data/history contains a compatible history dataset generated from FullNode data, with matching genesis and sufficient overlap with the LiteNode’s recent block range.
  • The destination database directories do not already exist.
  1. Stop the node and configure the storage locations in node.conf:

    storage.properties = [
      { name = "block", path = "/hdd/tron" },
      { name = "trans", path = "/hdd/tron" }
    ]
  2. Relocate the existing databases:

    java -jar Toolkit.jar db mv \
      -c node.conf \
      -d /ssd/tron/output-directory

    This moves their contents to HDD and creates links at the original paths:

    /ssd/tron/output-directory/database/block -> /hdd/tron/database/block
    /ssd/tron/output-directory/database/trans -> /hdd/tron/database/trans
    
  3. Merge the history dataset into the LiteNode using its original database path:

    java -jar Toolkit.jar db lite -o merge \
      --fn-data-path /ssd/tron/output-directory/database \
      --dataset-path /data/history
  4. Inspect the database locations after merge completes, then restart the node with the same configuration.

Behavior before this fix

Merge first backs up the existing archive databases. Its subsequent deletion follows the database symlinks, clears the HDD targets, and removes the original SSD links. History is then copied into newly created ordinary directories on SSD, and the required recent data is merged from the backup.

block-beta
    columns 5
    A["1. db mv<br/>Move to HDD"] space B["2. SSD links<br/>Config: HDD"] space C["3. Start merge<br/>Back up data"]
    space:5
    F["6. Restart<br/>Read empty HDD"] space E["5. Merge on SSD<br/>Delete backup"] space D["4. Clear HDD<br/>Delete SSD links"]
    A --> B
    B --> C
    C --> D
    D --> E
    E --> F
    style D fill:#fee2e2,stroke:#dc2626,color:#7f1d1d
    style E fill:#fee2e2,stroke:#dc2626,color:#7f1d1d
    style F fill:#fee2e2,stroke:#dc2626,color:#7f1d1d
Loading

Consequences

The storage placement established by db mv is effectively undone for the affected archive databases:

Location Expected after merge Actual before this fix
Original SSD database paths Symlinks to HDD Ordinary directories containing the merged data
Relocated HDD directories Merged archive data Existing contents cleared
Node configuration Reads the populated HDD databases Still points to the emptied HDD databases

After a successful merge, the merged archive data still exists on SSD. This is a recoverable storage placement and configuration mismatch: the node reads a location that no longer contains the expected block or transaction data.

Correcting the affected configuration paths can restore access to the SSD databases. Restoring the intended HDD placement requires another migration. Merge should preserve that placement without requiring these recovery steps.

Behavior with this fix

Merge identifies relocated archive databases before modifying them and resolves their actual storage directories. After backing up the existing data, it clears only the destination contents, preserving both the original symlinks and the destination root directories.

History is copied into those destinations. The existing history trimming and backup merge steps then operate through the preserved links.

block-beta
    columns 5
    A["1. db mv<br/>Move to HDD"] space B["2. SSD links<br/>Config: HDD"] space C["3. Start merge<br/>Resolve links<br/>Back up data"]
    space:5
    F["6. Restart<br/>Read HDD DBs"] space E["5. Merge on HDD<br/>Delete backup"] space D["4. Clear HDD<br/>Keep SSD links<br/>and HDD roots"]
    A --> B
    B --> C
    C --> D
    D --> E
    E --> F
    style D fill:#dcfce7,stroke:#16a34a,color:#14532d
    style E fill:#dcfce7,stroke:#16a34a,color:#14532d
    style F fill:#dcfce7,stroke:#16a34a,color:#14532d
Loading

This PR has been tested by:

  • Helper unit tests covering dangling links and propagated cleanup/copy failures.
  • Actual DbMove → DbLite command tests covering all five archive databases, with history heights below, equal to, and above the snapshot height. These verify preserved links and merged data in the relocated directories.
  • Injected backup, cleanup, and history-copy failures. These verify exit code 1, retained links and snapshot metadata, intact original data after backup failure, and readable backups after cleanup or copy failure.
  • RocksDB and LevelDB integration tests with relocated block and trans databases, covering checkpoint versions 1 and 2.
  • Lint and main/test Checkstyle checks.

@codeant-ai

codeant-ai Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

🤖 CodeAnt AI — Review Status

Status Commit Started (UTC) Finished (UTC)
✅ Reviewed your PR 10edfc8 Oct 12, 2026 · 11:42 11:47

@codeant-ai

codeant-ai Bot commented Oct 12, 2026

Copy link
Copy Markdown

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-12T11:45:25.993095Z 10edfc8 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@codeant-ai codeant-ai Bot added the size:XL This PR changes 500-999 lines, ignoring generated files label Oct 12, 2026
@codeant-ai

codeant-ai Bot commented Oct 12, 2026

Copy link
Copy Markdown

CodeAnt Nitpicks

1 code suggestion

1. FileUtils.deleteDir can return false after a failed or partial deletion, but the result is ignored; history files are then copied over leftovers, leaving obsolete database files.

Missing cleanup · plugins/src/main/java/common/org/tron/plugins/DbLite.java:590

@codeant-ai

codeant-ai Bot commented Oct 12, 2026

Copy link
Copy Markdown

CodeAnt PR Risk: Low Risk

  • The PR appears safe to merge: moved archive databases are backed up, repopulated through their resolved targets, and left linked in place.
  • Tests cover successful merges and injected backup, clearing, and copy failures, including preservation of recoverable backups.

Assessed commit: 10edfc885526

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XL This PR changes 500-999 lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant