From b157aaa44557dd979137ad8638d4fb82a6c99e60 Mon Sep 17 00:00:00 2001 From: Lakshman Patel Date: Tue, 8 Sep 2026 17:43:16 +0000 Subject: [PATCH] docs: add CLI vs top-20 OSS competitive analysis and gap plans --- docs/COMPETITIVE.md | 94 +++++++++++++++++++ .../competitive-gap-01-docker-onboarding.md | 39 ++++++++ .../competitive-gap-02-share-multisession.md | 37 ++++++++ .../competitive-gap-03-kitty-graphics.md | 34 +++++++ ...competitive-gap-04-published-benchmarks.md | 37 ++++++++ .../competitive-gap-05-backend-wiring.md | 41 ++++++++ 6 files changed, 282 insertions(+) create mode 100644 docs/COMPETITIVE.md create mode 100644 docs/plans/competitive-gap-01-docker-onboarding.md create mode 100644 docs/plans/competitive-gap-02-share-multisession.md create mode 100644 docs/plans/competitive-gap-03-kitty-graphics.md create mode 100644 docs/plans/competitive-gap-04-published-benchmarks.md create mode 100644 docs/plans/competitive-gap-05-backend-wiring.md diff --git a/docs/COMPETITIVE.md b/docs/COMPETITIVE.md new file mode 100644 index 00000000..de538090 --- /dev/null +++ b/docs/COMPETITIVE.md @@ -0,0 +1,94 @@ +# graycode-cli vs Top 20 OSS — Competitive Analysis + +Status: Proposed (branch `feat/competitive-analysis-top20`, 2026-09-08) +Scope: graycode-cli (this repo) vs 10 AI coding CLIs + 6 dev CLIs + 4 terminals (incl. `herdr` multiplexer) +Related: `docs/plans/toolbench-comparison-vs-top20.md` (tool-count parity), `docs/SECURITY-DEVELOPER.md` (sandbox model), `docs/plans/pi-adoption-plan.md` (Kitty graphics already proposed) + +## Methodology (no assumptions) + +Verified from source in this repo: + +- Tools: `cmd/chat_tools.go:41-192` — 34 essential + ~90 lazy-loaded optional (126 unique `tool.*Tool` refs; prior plan counted 69 — surface grew, mostly `spec_*`). +- Browser/Screenshot: `tool.BrowserTool{}`, `tool.ScreenshotTool{}` in essential set (`cmd/chat_tools.go:75-76`); headless Chrome via chromedp per prior plan. +- Sandbox: mandatory Docker, fail-closed, never host fallback — `docs/SECURITY-DEVELOPER.md:71-73`, `internal/sandbox/container.go:72-74`. +- Creds: OS secret store only, no `.env`/env read — `docs/SECURITY-DEVELOPER.md:7-12`. +- Share: local deeplink only — `internal/session/export.go:801-819` returns `graycode://share/`, no hosted URL. +- Custom providers: supported — `internal/config/settings.go:50` (`custom_providers`), `internal/config/graycode_router_engine.go:32-50`. +- Unwired backends: `internal/tool/computer_use.go:67-94` (`SetComputerBackend`, nil default), `internal/tool/media_generation.go:69-71` (`SetMediaEngine`, nil default). +- Terminal detect covers kitty/ghostty/wezterm/alacritty names (`internal/ui/icons/detect_test.go:56`); Kitty graphics protocol not implemented (see `docs/plans/pi-adoption-plan.md:25`). +- Bench infra exists (`internal/bench/suite.go`, `internal/feature/eval/`, `make bench`) but README publishes no numbers. + +External star counts below are approximate web-search snapshots (2026-09-08), not repo-verified. Treat as order-of-magnitude traction, not exact rankings. graycode-cli is pre-release (`VERSION`: `0.0.1`, `README.md:40-44` source-build primary) — it competes on architecture, not stars. + +## The 20 + +### A. Direct AI coding CLIs + +| # | Repo | Stars~ | Lang / Lic | Provider story | Sandbox | Memory | Multi-agent | Distro | +|---|---|---|---|---|---|---|---|---| +| A1 | `anomalyco/opencode` | ~200k | TS/Bun, MIT | 75+ via Models.dev, BYOK + Copilot/Plus login | allow/ask/deny globs, `--dangerously-skip-permissions` | AGENTS.md + @-imports + /init | primary/plan/subagent + custom agents | curl/npm/brew/scoop/Desktop | +| A2 | `openclaw/openclaw` | ~388k* | TS, MIT | any + fallbacks/aliases | Docker modes off/non-main/all + approvals | SOUL.md + MEMORY.md + wiki | agents.entries routing | curl/npm/Docker/Nix | +| A3 | `NousResearch/hermes-agent` | ~240k* | Py+TS, MIT | Portal + OpenRouter/OpenAI/custom | 7 backends (local/Docker/SSH/Modal/Daytona/…) | auto-skill creation + FTS5 + SOUL.md | subagents + worktree parallel | install.sh/Desktop | +| A4 | `openai/codex` | ~121k | Rust/Ratatui, Apache-2.0 | OpenAI-first + ChatGPT sub login + custom base_url | Seatbelt/Landlock/AppContainer; read-only/workspace-write/full; net-off default | AGENTS.md + Memories + /compress | subagent delegation | npm/brew/binary/Docker | +| A5 | `google-gemini/gemini-cli` | ~105k | TS, Apache-2.0 | Gemini-first + Anthropic/OpenAI/OpenRouter | Docker/Podman + gVisor + macOS sandbox-exec profiles | GEMINI.md + save_memory + checkpoints | subagents + policy engine | npm/npx/brew/Docker | +| A6 | `earendil-works/pi` | ~100k | TS, MIT | unified OpenAI/Anthropic/Google + Ollama | none by design (trust.json; run in container yourself) | AGENTS.md/CLAUDE.md + session persist | via Extension only | npm | +| A7 | `OpenHands/OpenHands` | ~86k | Py+TS, MIT | LiteLLM any + SaaS | DockerWorkspace (rec.) / Process / Remote | events + Condenser summarizer + skills | SDK delegation | pip/Docker | +| A8 | `cline/cline` | ~65k | TS, Apache-2.0 | BYOK shared config | approvals + shadow-git checkpoints | .clinerules-bank + skills | Plan/Act + SDK subagents | npm/VSCode/binaries | +| A9 | `block/goose` | ~53k | Rust, Apache-2.0 | registry + declarative custom | prompt/allow/deny + env strip + per-ext isolation | memory MCP (store/retrieve) | orchestrator + subagents | curl/Desktop/cargo | +| A10 | `Aider-AI/aider` | ~48k | Py, Apache-2.0 | LiteLLM any + Ollama | none; auto-commit + diff/undo + lint/test fix | RepoMap (graph-ranked defs) | none (wrappable as MCP tool) | pip/pipx/Docker | + +*OpenClaw/Hermes counts volatile (mirrors/forks); directionally >100k. + +### B. Dev CLIs (substrate + UX bar) + +| # | Repo | Stars~ | Lang / Lic | Lesson for graycode | +|---|---|---|---|---| +| B1 | `junegunn/fzf` | ~82k | Go, MIT | Pipe-first Unix design; zero-config speed | +| B2 | `jesseduffield/lazygit` | ~82k | Go, MIT | Keyboard TUI that makes hard git trivial; closest Go-TUI comp | +| B3 | `BurntSushi/ripgrep` | ~68k | Rust, MIT/Unlicense | Benchmarks in README; respects .gitignore; SIMD+parallel | +| B4 | `alacritty/alacritty` | ~65k | Rust, Apache-2.0 | Minimal fast core; delegate tabs to mux | +| B5 | `cli/cli` (`gh`) | ~46k | Go, MIT | Official CLI wins via scripting (`gh api`) + extensions | +| B6 | `herdrdev/herdr` | ~36k | Rust, Apache-2.0 | Agent multiplexer: persistent terms, detach/SSH, socket API; runs any agent as-is | + +### C. Terminals + modern CLI wave (runtime layer) + +| # | Repo | Stars~ | Lang / Lic | Note | +|---|---|---|---|---| +| C1 | `ghostty-org/ghostty` | ~60k | Zig, MIT | Native Metal/GL, libghostty, Kitty-graphics compat | +| C2 | `sharkdp/bat` / `starship/starship` | ~59k each | Rust, MIT/ISC | Drop-in replacements, single binary, sane defaults | +| C3 | `kovidgoyal/kitty` | ~34k | Py+C, GPL-3.0 | Image protocol others copy; GPL limits embedding | +| C4 | `wez/wezterm` | ~28k | Rust, MIT | Built-in mux + SSH mux, Lua config | + +## Deep dimensions + +1. **Traction.** graycode has no star-moat (pre-release). Leaders won via day-1 provider-agnostic + one-liner install + Web/Desktop alongside TUI. graycode already ships script/brew/npm paths (`README.md:48-59`) — keep, don't add Desktop. +2. **Language/distro.** Go+MIT+zero-CGO (`Makefile:54`, `go.mod:3`) matches `gh/fzf/lazygit` enterprise-safe profile. Avoid GPL/EUPL patterns (kitty/eza). Rust wave wins on published benchmarks — graycode has `make bench` but publishes none (Gap-04). +3. **Providers.** graycode routes only via `graycode-router/engine` facade (`docs/SECURITY-DEVELOPER.md:51-56`, `ecosystem.yaml:29-30`); custom OpenAI-compat supported (`internal/config/settings.go:50`). Count messaging ("28 first-class" per README) trails OpenCode 75+ / Hermes 300+ — fix by exposing catalog count dynamically, not by forking providers into CLI (ownership lives in router per AGENTS.md). +4. **TUI/UX.** Bubble Tea v2 + vim keys + `/autonomy` + `/spec` + watch `AI!`/`AI?` + visual diff is competitive. Missing vs field: hosted share-link (ours is local `graycode://` deeplink), multi-session grid (we have `mission` worktrees + daemon — unsurfaced like herdr/cmux). Gap-02. +5. **Sandbox.** Docker-only fail-closed is strictest default alongside Codex net-off and Gemini gVisor. Tradeoff is onboarding friction without Docker. Must not add host-exec fallback (violates `docs/SECURITY-DEVELOPER.md:71-73`); fix with preflight/path/doctor messaging + image pull/build guidance. Gap-01. +6. **Memory/context.** AST repomap + Harrier graph + compaction segments + relevance-prune + conversation-arc + 80% tool-result clearing exceeds most. Missing: Hermes-style auto-skill learning loop (we have curator archive + harness — surface it). +7. **Multi-agent.** `mission` worktrees + family messenger + path reservations + budgets + portable `mission-graph.json` + `graph export` (hashes only) is unique verifiable-execution story. Surface it; no new runtime needed. +8. **MCP/skills/plugins.** MCP stdio/HTTP/SSE/WS + LSP + skills search/install/audit + curator matches Goose/Gemini/Codex. Contracts live in `internal/contracts` (no `shared/types`) — extensions vendor DTOs. Correct; don't regress. +9. **Media/computer-use.** Tools exist (`Browser/Screenshot/CodeMatch/SearchX/AppVerify/GenerateMedia/ComputerUse`) but media/computer/STT backends are nil-by-default seams. README notes router ships `ImageClient`/`AudioClient`; host wiring is the gap. Gap-05. Kitty graphics (image display) still missing despite terminal detection. Gap-03. +10. **Ops/determinism.** Daemon `:4590` health/ready/chat-SSE + cron + `exec --fanout N` + replay cache + circuit breaker + smart routing + harness eval is ahead of Pi minimalism and Aider single-agent. Keep; add published eval numbers (Gap-04). + +## Verdict + +- **Wins to keep:** fail-closed Docker + dual `/autonomy`+`/spec` gates; portable execution graph; Go zero-CGO MIT; router-facade-only provider access; 120+ tool surface (see `docs/plans/toolbench-comparison-vs-top20.md` for category parity). +- **Loses to fix (filed as plans):** Gap-01 onboarding friction; Gap-02 share/multi-session; Gap-03 Kitty graphics; Gap-04 published benchmarks; Gap-05 default backend wiring. + +## Gap plans (this branch) + +- `docs/plans/competitive-gap-01-docker-onboarding.md` +- `docs/plans/competitive-gap-02-share-multisession.md` +- `docs/plans/competitive-gap-03-kitty-graphics.md` +- `docs/plans/competitive-gap-04-published-benchmarks.md` +- `docs/plans/competitive-gap-05-backend-wiring.md` + +Each follows the adoption-plan format (Status/Source/Existing/Decision/Priority) and respects developer-first + router-ownership + fail-closed constraints. + +## Verification + +- Source cites above re-checked 2026-09-08 on branch `feat/competitive-analysis-top20`. +- External stars: web-search snapshots, approximate — re-verify via GitHub API/badges before publishing. +- Docs-only change: run markdownlint + `make vet` (fast); full `make ci` before PR per `CONTRIBUTING.md:13-17`. diff --git a/docs/plans/competitive-gap-01-docker-onboarding.md b/docs/plans/competitive-gap-01-docker-onboarding.md new file mode 100644 index 00000000..eeccff75 --- /dev/null +++ b/docs/plans/competitive-gap-01-docker-onboarding.md @@ -0,0 +1,39 @@ +# Gap-01: Docker-Onboarding Friction (Docs/UX Only) + +Status: Proposed +Source: field comparison vs Codex (net-off workspace-write), Gemini (gVisor/sandbox-exec profiles), Pi (no sandbox) + +Constraint (non-negotiable): mandatory Docker isolation, fail-closed, never host fallback. +See `docs/SECURITY-DEVELOPER.md:71-73` and `internal/sandbox/container.go:72-74`. +This plan adds zero execution paths. It only improves messaging/docs. + +## Existing graycode capabilities (verified) + +- `graycode path` / `preflight` / `doctor` / `ecosystem` commands (`README.md:346-356`). +- `scripts/verify-developer-path.sh` (`make path`) and `scripts/smoke-graycode.sh` (`make smoke`). +- Sandbox image auto-pull (`graycodeai/graycode-sandbox`) with local Dockerfile build fallback (`docs/SECURITY-DEVELOPER.md:75-79`). + +## Decision + +Adopt: clearer failure copy + ordered remediation when Docker is missing. + +Do not adopt: workspace/host execution tier, `--yolo`-style host bypass, silent fallback. + +## Priority model + +- P0: `path`/`preflight`/`doctor` emit the same ordered checklist (daemon running? image cached? registry reachable? local build available?). +- P1: README quick-start callout that Docker is required before first run (already stated; tighten wording + link to checklist). +- P2: `smoke` output pastes the failing step with the exact fix command. + +## Steps + +1. Audit current `path`, `preflight`, `doctor` outputs for divergent Docker messages. +2. Unify copy: state fail-closed explicitly, then ordered steps (start daemon → pull → local build). +3. Update `README.md` install section link to `docs/SECURITY-DEVELOPER.md` checklist. +4. No changes to `internal/sandbox`, `internal/engine`, permissions. + +## Verification + +- `make path` with Docker stopped prints ordered checklist (manual). +- `go test ./cmd/ -run 'TestPath|TestPreflight|TestDoctor' -count=1`. +- `make vet`. diff --git a/docs/plans/competitive-gap-02-share-multisession.md b/docs/plans/competitive-gap-02-share-multisession.md new file mode 100644 index 00000000..e150191a --- /dev/null +++ b/docs/plans/competitive-gap-02-share-multisession.md @@ -0,0 +1,37 @@ +# Gap-02: Share Links + Multi-Session Visibility (Local-First) + +Status: Proposed +Source: field comparison vs OpenCode share-links/multi-session, herdr multiplexer, Cline checkpoints + +Constraint: developer-first, local by default, no cloud account required +(per `.github/ISSUE_TEMPLATE/feature_request.yml:60-63`). + +## Existing graycode capabilities (verified) + +- `GenerateShareLink` returns local deeplink `graycode://share/` (`internal/session/export.go:801-819`); deterministic, no hosted URL. +- Session export (`session_export.go`), mission graph export (`cmd/execution_graph.go`, `mission-graph.json`), daemon sessions, `mission` worktrees. +- Completion list includes `exec`, `daemon`, `mission`, `sessions`, `tools`, `skills` (`cmd/completions_test.go:50`). + +## Decision + +Adopt: local share bundle + session inventory that works offline. + +Do not adopt: hosted share URLs, cloud account,/Desktop app. + +## Priority model + +- P0: `sessions` list shows id/model/updated + export path; document `graph export` bundle as the share unit. +- P1: TUI session picker surfaces the `graycode://share/` deeplink + export file path for copy-paste. +- P2: Mission watchdog read-only overview (already in HUD panel) exposed via `mission --dry-run`/status; no new runtime. + +## Steps + +1. Confirm `sessions` command output covers the P0 fields; extend only display, not storage format. +2. Document share flow in `docs/COMPETITIVE.md` + user-guide: export file → send → `graph export` validate. +3. Keep `internal/session` format stable; no contract break. + +## Verification + +- `go test ./internal/session/ -run TestGenerateShareLink -count=1`. +- `go test ./cmd/ -run TestSession -count=1` (or nearest session-picker test). +- `make vet`. diff --git a/docs/plans/competitive-gap-03-kitty-graphics.md b/docs/plans/competitive-gap-03-kitty-graphics.md new file mode 100644 index 00000000..970e80db --- /dev/null +++ b/docs/plans/competitive-gap-03-kitty-graphics.md @@ -0,0 +1,34 @@ +# Gap-03: Kitty Graphics Protocol for Terminal Images + +Status: Proposed +Source: `https://github.com/kovidgoyal/kitty` (GPL-3.0; protocol only, no code copy), Ghostty compat; extends `docs/plans/pi-adoption-plan.md:25` (already proposed there — this file scopes the TUI work, it does not re-propose). + +## Existing graycode capabilities (verified) + +- Vision input path: `internal/engine/vision.go`; image command: `cmd/image.go`. +- Terminal detection covers kitty/ghostty/wezterm/alacritty names (`internal/ui/icons/detect_test.go:56`). +- No Kitty graphics emit path found in source audit 2026-09-08. + +## Decision + +Adopt: Kitty graphics emit for image display with capability detection + text fallback. + +Do not adopt: kitty source, GPL code, Ghostty/Zig code, breaking Bubble Tea v2 rendering. + +## Priority model + +- P0: capability probe (env `KITTY_PID`/terminfo/`TERM_PROGRAM` + query) with safe fallback to current rendering. +- P1: wire probe into image/screenshot display path (`cmd/image.go`, vision output). +- P2: chunked transmit + resize policy for large PNGs. + +## Steps + +1. Add `internal/tui/graphics.go` (new, isolated): probe + encode + emit + fallback. No imports from kitty. +2. Gate behind explicit detection; default behavior unchanged on non-capable terminals. +3. Tests: unit probe/encode tests with golden byte prefixes; no live terminal required. + +## Verification + +- `go test ./internal/tui/ -run TestGraphics -count=1` (new). +- `go test ./cmd/ -run TestImage -count=1` (existing path unbroken). +- `make vet`. diff --git a/docs/plans/competitive-gap-04-published-benchmarks.md b/docs/plans/competitive-gap-04-published-benchmarks.md new file mode 100644 index 00000000..b8b3790b --- /dev/null +++ b/docs/plans/competitive-gap-04-published-benchmarks.md @@ -0,0 +1,37 @@ +# Gap-04: Published Benchmarks (Docs from Existing Infra) + +Status: Proposed +Source: field comparison vs ripgrep/fzf/alacritty (numbers in README), Aider RepoMap token budgets + +Constraint: docs-only. No new benchmark framework; infra already exists. + +## Existing graycode capabilities (verified) + +- `make bench` (`go test -bench=. -benchmem -count=3`) in `Makefile:100-101`. +- `internal/bench/suite.go` (eco suite runner + report formatter). +- `internal/feature/eval/` (model benchmark tasks, runner, CSV export). +- Session load/save benchmarks (`internal/session/benchmark_test.go`). + +## Decision + +Adopt: a repeatable report workflow + published table in README/docs. + +Do not adopt: new eval harness, SWE-bench claims, provider-funded comparisons. + +## Priority model + +- P0: fixed command + env (`make bench` subset: session save/load 100/1000, repomap size/tokens) recorded with machine + commit. +- P1: `docs/BENCHMARKS.md` table (TUI-independent, no latency theater): session save/load, repomap tokens, tool-catalog size before/after `GRAYCODE_TOOL_SHRINK=1`. +- P2: CI artifact (optional): nightly `bench` JSON upload; never gate releases on it. + +## Steps + +1. Run the P0 subset locally; capture `go test -bench` output + commit SHA. +2. Write `docs/BENCHMARKS.md` with method, hardware, commit, raw output link. +3. Link from README performance-adjacent section; keep claims to measured numbers only. + +## Verification + +- Commands used are existing `make`/`go test` targets (no code change). +- Markdown passes markdownlint config (line-length disabled). +- `make vet` clean (no code touched). diff --git a/docs/plans/competitive-gap-05-backend-wiring.md b/docs/plans/competitive-gap-05-backend-wiring.md new file mode 100644 index 00000000..efce787f --- /dev/null +++ b/docs/plans/competitive-gap-05-backend-wiring.md @@ -0,0 +1,41 @@ +# Gap-05: Default Wiring for Media / Computer-Use / STT Backends + +Status: Proposed +Source: field comparison vs Qwen `computer_use`, Codex browser/screenshot, Goose extensions; README notes router ships `ImageClient`/`AudioClient` + +Constraints (non-negotiable): + +- Provider ownership lives in `../graycode-router`; graycode consumes only the stable engine facade (`AGENTS.md`, `docs/SECURITY-DEVELOPER.md:51-56`). +- Boundary guards must stay green: `make boundaries` (incl. `graycode-router-client-guard`, `graycode-router-engine-guard`). +- Tools fail safe today with explicit errors when unwired — preserve that behavior when disabled. + +## Existing graycode capabilities (verified) + +- `ComputerUseTool` reports "no computer backend installed" without `SetComputerBackend` (`internal/tool/computer_use.go:67-94`). +- `GenerateMediaTool` backend nil by default via `SetMediaEngine` (`internal/tool/media_generation.go:69-71`). +- `internal/stt` package exists (`stt.go`); Telegram voice path documented as backend-seamed. +- Custom providers supported via settings (`internal/config/settings.go:50,115`). + +## Decision + +Adopt: opt-in host wiring through the router facade, env-gated, off by default. + +Do not adopt: direct `graycode-router/client` production imports, new secrets paths, always-on media/computer-use. + +## Priority model + +- P0: design note mapping each tool seam → facade method (media/image, STT/audio, computer backend) with guard-safe import path. +- P1: env-gated wiring (e.g. `GRAYCODE_MEDIA=1`) + docs; unwired default error text unchanged. +- P2: `graycode doctor` reports backend status (wired/unwired) without leaking secrets. + +## Steps + +1. Confirm facade methods exist in sibling `../graycode-router/engine`; if missing, file the change there first (router repo owns providers). +2. Implement wiring in graycode behind env gates; keep `Set*` seams for tests. +3. Run `make boundaries` + `make vet` + targeted `go test ./internal/tool/ -run 'TestComputer|TestMedia'`. + +## Verification + +- `make boundaries` green (no client-boundary violation). +- `go test ./internal/tool/ -run 'TestComputerUse|TestMediaGeneration' -count=1`. +- `make vet`.