From 62fafdbf61a3bda1785aec7bda230e9d5340040a Mon Sep 17 00:00:00 2001 From: Illia Aihistov Date: Tue, 1 Sep 2026 20:17:48 +0300 Subject: [PATCH 1/2] docs: add dedupe package documentation and comparison table to notable packages guide --- .../docs/3_other_notable_packages.md | 30 +++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/doc/docusaurus/docs/3_other_notable_packages.md b/doc/docusaurus/docs/3_other_notable_packages.md index a749d515..342db05b 100644 --- a/doc/docusaurus/docs/3_other_notable_packages.md +++ b/doc/docusaurus/docs/3_other_notable_packages.md @@ -36,3 +36,33 @@ providing an objective way to find and fix overly complex logic and routines. - **Links**: [pub.dev](https://pub.dev/packages/cognitive_complexity) · [GitHub](https://github.com/kevmoo/analytica.dart/tree/main/packages/cognitive_complexity) + +--- + +## [dedupe](https://pub.dev/packages/dedupe) + +[![pub package](https://img.shields.io/pub/v/dedupe.svg)](https://pub.dev/packages/dedupe) + +High-performance code duplication and clone detection engine and CLI tool for +Dart and Flutter. + +It uses a polynomial rolling hash engine with maximal bidirectional expansion to +detect token-level, structural, and parameterized code clones across packages. + +- **Links**: [pub.dev](https://pub.dev/packages/dedupe) · + [GitHub](https://github.com/kevmoo/analytica.dart/tree/main/packages/dedupe) + +### Comparison: `avoid_duplicate_code` vs `dedupe` + +| Feature | `avoid_duplicate_code` | `dedupe` | +| :--- | :--- | :--- | +| **Tool type** | Dart Analyzer / Linter plugin rule | Standalone CLI tool & Dart library/API | +| **Primary workflow** | Real-time in-IDE feedback and `dart analyze` | Repository auditing, CI/CD checks, PR gating, batch analysis | +| **Core engine** | **AST Subtree Fingerprinting:** AST traversal and Jenkins One-at-a-time hashing | **Hybrid AST & Token Rolling Hash:** Rabin-Karp k-gram indexing and MinHash for gapped matches | +| **Code scope** | Syntactic blocks (functions, methods, closures, control-flow bodies) | Arbitrary token/line sequences (can detect clones across expression boundaries) | +| **Supported clone types** | • **Type 1** (exact copies)
• **Type 2** (syntactic: renamed variables, differing literals) | • **Type 1** (exact copies)
• **Type 2** (syntactic: renamed variables, differing literals)
• **Type 3** (near-miss: copies with insertions, deletions, or statement changes) | +| **Differing literals diffing** | ✅ Detailed diff analysis reporting specific differing literal values | ❌ None (literals are normalized during extraction without per-value diffing) | +| **Git / PR delta integration** | ❌ None (analyzes the current state of workspace files) | ✅ Built-in for PR delta and modified lines scanning | +| **Report formats** | Dart Analysis Server diagnostics (IDE Problems, Related Locations) | Markdown, JSON, GitHub Actions step annotations, plain text | +| **Metrics & statistics** | Duplicate locations and differing literal values | Duplication percentages per file/package, cluster inventories, estimated lines saved | +| **Disk cache usage** | ✅ | ✅ | From c55868dc7d5c4396f543163cb3b5957bd00c9a7d Mon Sep 17 00:00:00 2001 From: Illia Aihistov Date: Wed, 2 Sep 2026 09:48:51 +0300 Subject: [PATCH 2/2] docs: update dedupe description and improve comparison table with avoid_duplicate_code --- .../docs/3_other_notable_packages.md | 27 +++++++++++-------- 1 file changed, 16 insertions(+), 11 deletions(-) diff --git a/doc/docusaurus/docs/3_other_notable_packages.md b/doc/docusaurus/docs/3_other_notable_packages.md index 342db05b..db0a8b94 100644 --- a/doc/docusaurus/docs/3_other_notable_packages.md +++ b/doc/docusaurus/docs/3_other_notable_packages.md @@ -46,23 +46,28 @@ providing an objective way to find and fix overly complex logic and routines. High-performance code duplication and clone detection engine and CLI tool for Dart and Flutter. -It uses a polynomial rolling hash engine with maximal bidirectional expansion to -detect token-level, structural, and parameterized code clones across packages. +It scans codebases to detect token-level, structural, and near-miss code clones +across files and packages with fast incremental analysis. - **Links**: [pub.dev](https://pub.dev/packages/dedupe) · [GitHub](https://github.com/kevmoo/analytica.dart/tree/main/packages/dedupe) -### Comparison: `avoid_duplicate_code` vs `dedupe` +### Comparison: [`avoid_duplicate_code`](2_custom_lints/avoid_duplicate_code.md) vs `dedupe` -| Feature | `avoid_duplicate_code` | `dedupe` | +| Feature / Scenario | `avoid_duplicate_code` | `dedupe` | | :--- | :--- | :--- | | **Tool type** | Dart Analyzer / Linter plugin rule | Standalone CLI tool & Dart library/API | | **Primary workflow** | Real-time in-IDE feedback and `dart analyze` | Repository auditing, CI/CD checks, PR gating, batch analysis | -| **Core engine** | **AST Subtree Fingerprinting:** AST traversal and Jenkins One-at-a-time hashing | **Hybrid AST & Token Rolling Hash:** Rabin-Karp k-gram indexing and MinHash for gapped matches | -| **Code scope** | Syntactic blocks (functions, methods, closures, control-flow bodies) | Arbitrary token/line sequences (can detect clones across expression boundaries) | -| **Supported clone types** | • **Type 1** (exact copies)
• **Type 2** (syntactic: renamed variables, differing literals) | • **Type 1** (exact copies)
• **Type 2** (syntactic: renamed variables, differing literals)
• **Type 3** (near-miss: copies with insertions, deletions, or statement changes) | -| **Differing literals diffing** | ✅ Detailed diff analysis reporting specific differing literal values | ❌ None (literals are normalized during extraction without per-value diffing) | -| **Git / PR delta integration** | ❌ None (analyzes the current state of workspace files) | ✅ Built-in for PR delta and modified lines scanning | -| **Report formats** | Dart Analysis Server diagnostics (IDE Problems, Related Locations) | Markdown, JSON, GitHub Actions step annotations, plain text | -| **Metrics & statistics** | Duplicate locations and differing literal values | Duplication percentages per file/package, cluster inventories, estimated lines saved | +| **Detection approach** | **Block-level (AST):** Analyzes complete syntactic structures (functions, methods, closures, `if`/`for` bodies) | **Sequence-level (Tokens):** Analyzes continuous sequences of code anywhere across files | +| **Whole functions & blocks**
*(e.g., duplicated methods with renamed variables)* | ✅ **Detected** | ✅ **Detected** | +| **Sub-method fragments**
*(e.g., 5–10 copied lines inside a 50-line method)* | ❌ **Skipped** (only evaluates complete blocks/methods) | ✅ **Detected** (flags duplicate snippets regardless of block boundaries) | +| **Near-miss / modified clones**
*(e.g., copy-paste with an extra line or minor edit)* | ❌ **Skipped** (requires matching syntactic block structure) | ✅ **Detected** (fuzzy matching detects clones with insertions/deletions) | +| **Differing literals diffing** | ✅ **Detailed in-IDE diff** (pinpoints exact differing values, e.g. `'email'` vs `'sms'`) | ❌ **No per-value diff** (flags clone locations without a detailed literal breakdown) | +| **Reporting & diagnostics** | IDE Problems & Related Locations | Markdown, JSON, GitHub Actions annotations, % duplication metrics | | **Disk cache usage** | ✅ | ✅ | + +> **💡 Best Together:** Use `avoid_duplicate_code` for instant IDE feedback, +> and `dedupe` for CI/CD quality gates and repository-wide code clone audits. + + +