Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 94 additions & 0 deletions docs/COMPETITIVE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
# graycode-cli vs Top 20 OSS — Competitive Analysis

Status: Proposed (branch `feat/competitive-analysis-top20`, 2026-09-08)
Scope: graycode-cli (this repo) vs 10 AI coding CLIs + 6 dev CLIs + 4 terminals (incl. `herdr` multiplexer)
Related: `docs/plans/toolbench-comparison-vs-top20.md` (tool-count parity), `docs/SECURITY-DEVELOPER.md` (sandbox model), `docs/plans/pi-adoption-plan.md` (Kitty graphics already proposed)

## Methodology (no assumptions)

Verified from source in this repo:

- Tools: `cmd/chat_tools.go:41-192` — 34 essential + ~90 lazy-loaded optional (126 unique `tool.*Tool` refs; prior plan counted 69 — surface grew, mostly `spec_*`).
- Browser/Screenshot: `tool.BrowserTool{}`, `tool.ScreenshotTool{}` in essential set (`cmd/chat_tools.go:75-76`); headless Chrome via chromedp per prior plan.
- Sandbox: mandatory Docker, fail-closed, never host fallback — `docs/SECURITY-DEVELOPER.md:71-73`, `internal/sandbox/container.go:72-74`.
- Creds: OS secret store only, no `.env`/env read — `docs/SECURITY-DEVELOPER.md:7-12`.
- Share: local deeplink only — `internal/session/export.go:801-819` returns `graycode://share/<hash[:16]>`, no hosted URL.
- Custom providers: supported — `internal/config/settings.go:50` (`custom_providers`), `internal/config/graycode_router_engine.go:32-50`.
- Unwired backends: `internal/tool/computer_use.go:67-94` (`SetComputerBackend`, nil default), `internal/tool/media_generation.go:69-71` (`SetMediaEngine`, nil default).
- Terminal detect covers kitty/ghostty/wezterm/alacritty names (`internal/ui/icons/detect_test.go:56`); Kitty graphics protocol not implemented (see `docs/plans/pi-adoption-plan.md:25`).
- Bench infra exists (`internal/bench/suite.go`, `internal/feature/eval/`, `make bench`) but README publishes no numbers.

External star counts below are approximate web-search snapshots (2026-09-08), not repo-verified. Treat as order-of-magnitude traction, not exact rankings. graycode-cli is pre-release (`VERSION`: `0.0.1`, `README.md:40-44` source-build primary) — it competes on architecture, not stars.

## The 20

### A. Direct AI coding CLIs

| # | Repo | Stars~ | Lang / Lic | Provider story | Sandbox | Memory | Multi-agent | Distro |
|---|---|---|---|---|---|---|---|---|
| A1 | `anomalyco/opencode` | ~200k | TS/Bun, MIT | 75+ via Models.dev, BYOK + Copilot/Plus login | allow/ask/deny globs, `--dangerously-skip-permissions` | AGENTS.md + @-imports + /init | primary/plan/subagent + custom agents | curl/npm/brew/scoop/Desktop |
| A2 | `openclaw/openclaw` | ~388k* | TS, MIT | any + fallbacks/aliases | Docker modes off/non-main/all + approvals | SOUL.md + MEMORY.md + wiki | agents.entries routing | curl/npm/Docker/Nix |
| A3 | `NousResearch/hermes-agent` | ~240k* | Py+TS, MIT | Portal + OpenRouter/OpenAI/custom | 7 backends (local/Docker/SSH/Modal/Daytona/…) | auto-skill creation + FTS5 + SOUL.md | subagents + worktree parallel | install.sh/Desktop |
| A4 | `openai/codex` | ~121k | Rust/Ratatui, Apache-2.0 | OpenAI-first + ChatGPT sub login + custom base_url | Seatbelt/Landlock/AppContainer; read-only/workspace-write/full; net-off default | AGENTS.md + Memories + /compress | subagent delegation | npm/brew/binary/Docker |
| A5 | `google-gemini/gemini-cli` | ~105k | TS, Apache-2.0 | Gemini-first + Anthropic/OpenAI/OpenRouter | Docker/Podman + gVisor + macOS sandbox-exec profiles | GEMINI.md + save_memory + checkpoints | subagents + policy engine | npm/npx/brew/Docker |
| A6 | `earendil-works/pi` | ~100k | TS, MIT | unified OpenAI/Anthropic/Google + Ollama | none by design (trust.json; run in container yourself) | AGENTS.md/CLAUDE.md + session persist | via Extension only | npm |
| A7 | `OpenHands/OpenHands` | ~86k | Py+TS, MIT | LiteLLM any + SaaS | DockerWorkspace (rec.) / Process / Remote | events + Condenser summarizer + skills | SDK delegation | pip/Docker |
| A8 | `cline/cline` | ~65k | TS, Apache-2.0 | BYOK shared config | approvals + shadow-git checkpoints | .clinerules-bank + skills | Plan/Act + SDK subagents | npm/VSCode/binaries |
| A9 | `block/goose` | ~53k | Rust, Apache-2.0 | registry + declarative custom | prompt/allow/deny + env strip + per-ext isolation | memory MCP (store/retrieve) | orchestrator + subagents | curl/Desktop/cargo |
| A10 | `Aider-AI/aider` | ~48k | Py, Apache-2.0 | LiteLLM any + Ollama | none; auto-commit + diff/undo + lint/test fix | RepoMap (graph-ranked defs) | none (wrappable as MCP tool) | pip/pipx/Docker |

*OpenClaw/Hermes counts volatile (mirrors/forks); directionally >100k.

### B. Dev CLIs (substrate + UX bar)

| # | Repo | Stars~ | Lang / Lic | Lesson for graycode |
|---|---|---|---|---|
| B1 | `junegunn/fzf` | ~82k | Go, MIT | Pipe-first Unix design; zero-config speed |
| B2 | `jesseduffield/lazygit` | ~82k | Go, MIT | Keyboard TUI that makes hard git trivial; closest Go-TUI comp |
| B3 | `BurntSushi/ripgrep` | ~68k | Rust, MIT/Unlicense | Benchmarks in README; respects .gitignore; SIMD+parallel |
| B4 | `alacritty/alacritty` | ~65k | Rust, Apache-2.0 | Minimal fast core; delegate tabs to mux |
| B5 | `cli/cli` (`gh`) | ~46k | Go, MIT | Official CLI wins via scripting (`gh api`) + extensions |
| B6 | `herdrdev/herdr` | ~36k | Rust, Apache-2.0 | Agent multiplexer: persistent terms, detach/SSH, socket API; runs any agent as-is |

### C. Terminals + modern CLI wave (runtime layer)

| # | Repo | Stars~ | Lang / Lic | Note |
|---|---|---|---|---|
| C1 | `ghostty-org/ghostty` | ~60k | Zig, MIT | Native Metal/GL, libghostty, Kitty-graphics compat |
| C2 | `sharkdp/bat` / `starship/starship` | ~59k each | Rust, MIT/ISC | Drop-in replacements, single binary, sane defaults |
| C3 | `kovidgoyal/kitty` | ~34k | Py+C, GPL-3.0 | Image protocol others copy; GPL limits embedding |
| C4 | `wez/wezterm` | ~28k | Rust, MIT | Built-in mux + SSH mux, Lua config |

## Deep dimensions

1. **Traction.** graycode has no star-moat (pre-release). Leaders won via day-1 provider-agnostic + one-liner install + Web/Desktop alongside TUI. graycode already ships script/brew/npm paths (`README.md:48-59`) — keep, don't add Desktop.
2. **Language/distro.** Go+MIT+zero-CGO (`Makefile:54`, `go.mod:3`) matches `gh/fzf/lazygit` enterprise-safe profile. Avoid GPL/EUPL patterns (kitty/eza). Rust wave wins on published benchmarks — graycode has `make bench` but publishes none (Gap-04).
3. **Providers.** graycode routes only via `graycode-router/engine` facade (`docs/SECURITY-DEVELOPER.md:51-56`, `ecosystem.yaml:29-30`); custom OpenAI-compat supported (`internal/config/settings.go:50`). Count messaging ("28 first-class" per README) trails OpenCode 75+ / Hermes 300+ — fix by exposing catalog count dynamically, not by forking providers into CLI (ownership lives in router per AGENTS.md).
4. **TUI/UX.** Bubble Tea v2 + vim keys + `/autonomy` + `/spec` + watch `AI!`/`AI?` + visual diff is competitive. Missing vs field: hosted share-link (ours is local `graycode://` deeplink), multi-session grid (we have `mission` worktrees + daemon — unsurfaced like herdr/cmux). Gap-02.
5. **Sandbox.** Docker-only fail-closed is strictest default alongside Codex net-off and Gemini gVisor. Tradeoff is onboarding friction without Docker. Must not add host-exec fallback (violates `docs/SECURITY-DEVELOPER.md:71-73`); fix with preflight/path/doctor messaging + image pull/build guidance. Gap-01.
6. **Memory/context.** AST repomap + Harrier graph + compaction segments + relevance-prune + conversation-arc + 80% tool-result clearing exceeds most. Missing: Hermes-style auto-skill learning loop (we have curator archive + harness — surface it).
7. **Multi-agent.** `mission` worktrees + family messenger + path reservations + budgets + portable `mission-graph.json` + `graph export` (hashes only) is unique verifiable-execution story. Surface it; no new runtime needed.
8. **MCP/skills/plugins.** MCP stdio/HTTP/SSE/WS + LSP + skills search/install/audit + curator matches Goose/Gemini/Codex. Contracts live in `internal/contracts` (no `shared/types`) — extensions vendor DTOs. Correct; don't regress.
9. **Media/computer-use.** Tools exist (`Browser/Screenshot/CodeMatch/SearchX/AppVerify/GenerateMedia/ComputerUse`) but media/computer/STT backends are nil-by-default seams. README notes router ships `ImageClient`/`AudioClient`; host wiring is the gap. Gap-05. Kitty graphics (image display) still missing despite terminal detection. Gap-03.
10. **Ops/determinism.** Daemon `:4590` health/ready/chat-SSE + cron + `exec --fanout N` + replay cache + circuit breaker + smart routing + harness eval is ahead of Pi minimalism and Aider single-agent. Keep; add published eval numbers (Gap-04).

## Verdict

- **Wins to keep:** fail-closed Docker + dual `/autonomy`+`/spec` gates; portable execution graph; Go zero-CGO MIT; router-facade-only provider access; 120+ tool surface (see `docs/plans/toolbench-comparison-vs-top20.md` for category parity).
- **Loses to fix (filed as plans):** Gap-01 onboarding friction; Gap-02 share/multi-session; Gap-03 Kitty graphics; Gap-04 published benchmarks; Gap-05 default backend wiring.

## Gap plans (this branch)

- `docs/plans/competitive-gap-01-docker-onboarding.md`
- `docs/plans/competitive-gap-02-share-multisession.md`
- `docs/plans/competitive-gap-03-kitty-graphics.md`
- `docs/plans/competitive-gap-04-published-benchmarks.md`
- `docs/plans/competitive-gap-05-backend-wiring.md`

Each follows the adoption-plan format (Status/Source/Existing/Decision/Priority) and respects developer-first + router-ownership + fail-closed constraints.

## Verification

- Source cites above re-checked 2026-09-08 on branch `feat/competitive-analysis-top20`.
- External stars: web-search snapshots, approximate — re-verify via GitHub API/badges before publishing.
- Docs-only change: run markdownlint + `make vet` (fast); full `make ci` before PR per `CONTRIBUTING.md:13-17`.
39 changes: 39 additions & 0 deletions docs/plans/competitive-gap-01-docker-onboarding.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Gap-01: Docker-Onboarding Friction (Docs/UX Only)

Status: Proposed
Source: field comparison vs Codex (net-off workspace-write), Gemini (gVisor/sandbox-exec profiles), Pi (no sandbox)

Constraint (non-negotiable): mandatory Docker isolation, fail-closed, never host fallback.
See `docs/SECURITY-DEVELOPER.md:71-73` and `internal/sandbox/container.go:72-74`.
This plan adds zero execution paths. It only improves messaging/docs.

## Existing graycode capabilities (verified)

- `graycode path` / `preflight` / `doctor` / `ecosystem` commands (`README.md:346-356`).
- `scripts/verify-developer-path.sh` (`make path`) and `scripts/smoke-graycode.sh` (`make smoke`).
- Sandbox image auto-pull (`graycodeai/graycode-sandbox`) with local Dockerfile build fallback (`docs/SECURITY-DEVELOPER.md:75-79`).

## Decision

Adopt: clearer failure copy + ordered remediation when Docker is missing.

Do not adopt: workspace/host execution tier, `--yolo`-style host bypass, silent fallback.

## Priority model

- P0: `path`/`preflight`/`doctor` emit the same ordered checklist (daemon running? image cached? registry reachable? local build available?).
- P1: README quick-start callout that Docker is required before first run (already stated; tighten wording + link to checklist).
- P2: `smoke` output pastes the failing step with the exact fix command.

## Steps

1. Audit current `path`, `preflight`, `doctor` outputs for divergent Docker messages.
2. Unify copy: state fail-closed explicitly, then ordered steps (start daemon → pull → local build).
3. Update `README.md` install section link to `docs/SECURITY-DEVELOPER.md` checklist.
4. No changes to `internal/sandbox`, `internal/engine`, permissions.

## Verification

- `make path` with Docker stopped prints ordered checklist (manual).
- `go test ./cmd/ -run 'TestPath|TestPreflight|TestDoctor' -count=1`.
- `make vet`.
37 changes: 37 additions & 0 deletions docs/plans/competitive-gap-02-share-multisession.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Gap-02: Share Links + Multi-Session Visibility (Local-First)

Status: Proposed
Source: field comparison vs OpenCode share-links/multi-session, herdr multiplexer, Cline checkpoints

Constraint: developer-first, local by default, no cloud account required
(per `.github/ISSUE_TEMPLATE/feature_request.yml:60-63`).

## Existing graycode capabilities (verified)

- `GenerateShareLink` returns local deeplink `graycode://share/<hash[:16]>` (`internal/session/export.go:801-819`); deterministic, no hosted URL.
- Session export (`session_export.go`), mission graph export (`cmd/execution_graph.go`, `mission-graph.json`), daemon sessions, `mission` worktrees.
- Completion list includes `exec`, `daemon`, `mission`, `sessions`, `tools`, `skills` (`cmd/completions_test.go:50`).

## Decision

Adopt: local share bundle + session inventory that works offline.

Do not adopt: hosted share URLs, cloud account,/Desktop app.

## Priority model

- P0: `sessions` list shows id/model/updated + export path; document `graph export` bundle as the share unit.
- P1: TUI session picker surfaces the `graycode://share/<id>` deeplink + export file path for copy-paste.
- P2: Mission watchdog read-only overview (already in HUD panel) exposed via `mission --dry-run`/status; no new runtime.

## Steps

1. Confirm `sessions` command output covers the P0 fields; extend only display, not storage format.
2. Document share flow in `docs/COMPETITIVE.md` + user-guide: export file → send → `graph export` validate.
3. Keep `internal/session` format stable; no contract break.

## Verification

- `go test ./internal/session/ -run TestGenerateShareLink -count=1`.
- `go test ./cmd/ -run TestSession -count=1` (or nearest session-picker test).
- `make vet`.
34 changes: 34 additions & 0 deletions docs/plans/competitive-gap-03-kitty-graphics.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Gap-03: Kitty Graphics Protocol for Terminal Images

Status: Proposed
Source: `https://github.com/kovidgoyal/kitty` (GPL-3.0; protocol only, no code copy), Ghostty compat; extends `docs/plans/pi-adoption-plan.md:25` (already proposed there — this file scopes the TUI work, it does not re-propose).

## Existing graycode capabilities (verified)

- Vision input path: `internal/engine/vision.go`; image command: `cmd/image.go`.
- Terminal detection covers kitty/ghostty/wezterm/alacritty names (`internal/ui/icons/detect_test.go:56`).
- No Kitty graphics emit path found in source audit 2026-09-08.

## Decision

Adopt: Kitty graphics emit for image display with capability detection + text fallback.

Do not adopt: kitty source, GPL code, Ghostty/Zig code, breaking Bubble Tea v2 rendering.

## Priority model

- P0: capability probe (env `KITTY_PID`/terminfo/`TERM_PROGRAM` + query) with safe fallback to current rendering.
- P1: wire probe into image/screenshot display path (`cmd/image.go`, vision output).
- P2: chunked transmit + resize policy for large PNGs.

## Steps

1. Add `internal/tui/graphics.go` (new, isolated): probe + encode + emit + fallback. No imports from kitty.
2. Gate behind explicit detection; default behavior unchanged on non-capable terminals.
3. Tests: unit probe/encode tests with golden byte prefixes; no live terminal required.

## Verification

- `go test ./internal/tui/ -run TestGraphics -count=1` (new).
- `go test ./cmd/ -run TestImage -count=1` (existing path unbroken).
- `make vet`.
37 changes: 37 additions & 0 deletions docs/plans/competitive-gap-04-published-benchmarks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Gap-04: Published Benchmarks (Docs from Existing Infra)

Status: Proposed
Source: field comparison vs ripgrep/fzf/alacritty (numbers in README), Aider RepoMap token budgets

Constraint: docs-only. No new benchmark framework; infra already exists.

## Existing graycode capabilities (verified)

- `make bench` (`go test -bench=. -benchmem -count=3`) in `Makefile:100-101`.
- `internal/bench/suite.go` (eco suite runner + report formatter).
- `internal/feature/eval/` (model benchmark tasks, runner, CSV export).
- Session load/save benchmarks (`internal/session/benchmark_test.go`).

## Decision

Adopt: a repeatable report workflow + published table in README/docs.

Do not adopt: new eval harness, SWE-bench claims, provider-funded comparisons.

## Priority model

- P0: fixed command + env (`make bench` subset: session save/load 100/1000, repomap size/tokens) recorded with machine + commit.
- P1: `docs/BENCHMARKS.md` table (TUI-independent, no latency theater): session save/load, repomap tokens, tool-catalog size before/after `GRAYCODE_TOOL_SHRINK=1`.
- P2: CI artifact (optional): nightly `bench` JSON upload; never gate releases on it.

## Steps

1. Run the P0 subset locally; capture `go test -bench` output + commit SHA.
2. Write `docs/BENCHMARKS.md` with method, hardware, commit, raw output link.
3. Link from README performance-adjacent section; keep claims to measured numbers only.

## Verification

- Commands used are existing `make`/`go test` targets (no code change).
- Markdown passes markdownlint config (line-length disabled).
- `make vet` clean (no code touched).
41 changes: 41 additions & 0 deletions docs/plans/competitive-gap-05-backend-wiring.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Gap-05: Default Wiring for Media / Computer-Use / STT Backends

Status: Proposed
Source: field comparison vs Qwen `computer_use`, Codex browser/screenshot, Goose extensions; README notes router ships `ImageClient`/`AudioClient`

Constraints (non-negotiable):

- Provider ownership lives in `../graycode-router`; graycode consumes only the stable engine facade (`AGENTS.md`, `docs/SECURITY-DEVELOPER.md:51-56`).
- Boundary guards must stay green: `make boundaries` (incl. `graycode-router-client-guard`, `graycode-router-engine-guard`).
- Tools fail safe today with explicit errors when unwired — preserve that behavior when disabled.

## Existing graycode capabilities (verified)

- `ComputerUseTool` reports "no computer backend installed" without `SetComputerBackend` (`internal/tool/computer_use.go:67-94`).
- `GenerateMediaTool` backend nil by default via `SetMediaEngine` (`internal/tool/media_generation.go:69-71`).
- `internal/stt` package exists (`stt.go`); Telegram voice path documented as backend-seamed.
- Custom providers supported via settings (`internal/config/settings.go:50,115`).

## Decision

Adopt: opt-in host wiring through the router facade, env-gated, off by default.

Do not adopt: direct `graycode-router/client` production imports, new secrets paths, always-on media/computer-use.

## Priority model

- P0: design note mapping each tool seam → facade method (media/image, STT/audio, computer backend) with guard-safe import path.
- P1: env-gated wiring (e.g. `GRAYCODE_MEDIA=1`) + docs; unwired default error text unchanged.
- P2: `graycode doctor` reports backend status (wired/unwired) without leaking secrets.

## Steps

1. Confirm facade methods exist in sibling `../graycode-router/engine`; if missing, file the change there first (router repo owns providers).
2. Implement wiring in graycode behind env gates; keep `Set*` seams for tests.
3. Run `make boundaries` + `make vet` + targeted `go test ./internal/tool/ -run 'TestComputer|TestMedia'`.

## Verification

- `make boundaries` green (no client-boundary violation).
- `go test ./internal/tool/ -run 'TestComputerUse|TestMediaGeneration' -count=1`.
- `make vet`.
Loading