Story: Data Collection Infrastructure for Dashboard Stats #146

Branch: feat/stats-aggregation
Priority: Highest
Parent Epic: #145
Type: Story

Description

Implement the data collection infrastructure needed to support dashboard visualizations. This includes extending the IndexStats interface, creating a StatsAggregator class, and updating scanners to track detailed metadata about languages, component types, and package structure.

This is foundational work that must be completed before CLI enhancements or web dashboard can display rich statistics.

Acceptance Criteria

• Extended IndexStats interface includes language breakdown
• Extended IndexStats interface includes component type counts
• Extended IndexStats interface includes package-level stats (for monorepos)
• StatsAggregator class efficiently collects stats during indexing
• Scanners (TypeScript, Go) report language and component metadata
• Indexer aggregates stats with <5% performance overhead
• Stats are persisted to state file for retrieval
• Backward compatibility maintained (old state files still work)

Technical Requirements

Storage: Extend existing indexer-state.json file

New Types:

interface DetailedIndexStats extends IndexStats {
  byLanguage: Record<SupportedLanguage, LanguageStats>;
  byComponentType: Record<ComponentType, number>;
  byPackage?: Record<string, PackageStats>;
}

New Classes:

• StatsAggregator class in packages/core/src/indexer/stats-aggregator.ts
• Methods: addFile(), addComponent(), getAggregatedStats()
• Efficient incremental aggregation (no post-processing)

Scanner Updates:

• TypeScript scanner: Report language (TS vs JS) per file
• Go scanner: Report Go language per file
• Both: Include component type in metadata

Testing:

• Unit tests for StatsAggregator
• Integration tests with scanners
• Performance benchmarks (<5% overhead)
• Backward compatibility tests

Files to Create:

• packages/core/src/indexer/stats-aggregator.ts
• packages/core/src/indexer/__tests__/stats-aggregator.test.ts

Files to Modify:

• packages/core/src/indexer/types.ts
• packages/core/src/indexer/index.ts
• packages/core/src/scanner/typescript.ts
• packages/core/src/scanner/go.ts