Repository navigation
feat(acp): accept image and embedded-resource prompt blocks - #611
Merged
Merged
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
Image validation can be bypassed by model changes and currently permits invalid content paths.
4 open findings
What changed in this PR
Adds ACP image and embedded-resource prompt support, backed by runtime attachment validation and model capability metadata.
Changes:
- Maps ACP image/resource blocks into runtime parts and advertises prompt capabilities.
- Adds catalog-driven image-input capability checks.
- Updates contract tests, goldens, architecture docs, and changelog.
Architecture assessment identified validation-boundary gaps around model changes and attachment types.
| File | Description |
|---|---|
models.go |
Exposes image capability lookup. |
internal/turn/turn.go |
Adds image capability metadata. |
internal/modelmeta/modelmeta.go |
Resolves model image support. |
internal/modelmeta/model_info_gen.go |
Records text-only models. |
internal/modelmeta/internal/genctx/main.go |
Generates image capability metadata. |
internal/gates/structure.go |
Registers metadata lookup ownership. |
internal/backend/modelapi/modelapi.go |
Populates backend image capabilities. |
internal/acp/session.go |
Stores capability lookup callback. |
internal/acp/prompt.go |
Maps and validates new prompt blocks. |
internal/acp/newsession.go |
Wires runtime capability lookup. |
internal/acp/mapping_test.go |
Updates prompt mapping tests. |
internal/acp/acp.go |
Advertises ACP prompt capabilities. |
harnesstest/harnesstest.go |
Documents attachment part kinds. |
harnesstest/anthropic.go |
Decodes image and document blocks. |
e2e/testdata/contract/plugin_session_messages_carry_attachments.golden.json |
Records attachment wire output. |
e2e/testdata/acp/unsupported_content_is_invalid_params.json |
Covers invalid attachment content. |
e2e/testdata/acp/unknown_method_is_not_found.json |
Updates initialization capability output. |
e2e/testdata/acp/text_resource_reaches_the_model_with_its_uri.json |
Covers embedded text resources. |
e2e/testdata/acp/second_authenticate_cancels_the_one_in_flight.json |
Updates initialization output. |
e2e/testdata/acp/replay_skips_the_summary_without_the_capability.json |
Updates initialization output. |
e2e/testdata/acp/replay_sends_the_summary_as_a_completed_compaction_with_the_capability.json |
Updates initialization output. |
e2e/testdata/acp/rejected_refresh_during_a_turn_answers_auth_required.json |
Updates initialization output. |
e2e/testdata/acp/prompt_after_the_chatgpt_login_vanished_is_auth_required.json |
Updates initialization output. |
e2e/testdata/acp/project_only_claude_code_provider_raises_no_auth_required_without_a_method.json |
Updates initialization output. |
e2e/testdata/acp/pdf_resource_blob_reaches_the_model.json |
Covers embedded PDF resources. |
e2e/testdata/acp/new_session_without_a_chatgpt_login_is_auth_required.json |
Updates initialization output. |
e2e/testdata/acp/lost_ownership_sends_no_notice_without_the_capability.json |
Updates initialization output. |
e2e/testdata/acp/lost_ownership_sends_a_notice_when_the_client_has_the_capability.json |
Updates initialization output. |
e2e/testdata/acp/logout_removes_the_chatgpt_login.json |
Updates initialization output. |
e2e/testdata/acp/logout_is_not_offered_for_a_claude_code_provider_alone.json |
Updates initialization output. |
e2e/testdata/acp/logout_ends_the_chatgpt_login_and_leaves_the_claude_code_login.json |
Updates initialization output. |
e2e/testdata/acp/initialize_without_version_is_invalid.json |
Updates initialization output. |
e2e/testdata/acp/initialize_newer_version_gets_latest.json |
Updates initialization output. |
e2e/testdata/acp/initialize_lists_the_claude_code_method_for_a_claude_code_provider.json |
Updates initialization output. |
e2e/testdata/acp/initialize_lists_the_chatgpt_method_for_a_chatgpt_provider.json |
Updates initialization output. |
e2e/testdata/acp/initialize_lists_terminal_methods_only_for_a_terminal_client.json |
Updates initialization output. |
e2e/testdata/acp/initialize_answers_agent_info.json |
Updates initialization output. |
e2e/testdata/acp/initialize_advertises_image_and_embedded_context.json |
Covers advertised prompt capabilities. |
e2e/testdata/acp/image_block_reaches_the_model_as_an_image.json |
Covers image delivery. |
e2e/testdata/acp/id_of_other_type_is_invalid.json |
Updates initialization output. |
e2e/testdata/acp/claude_code_logged_out_session_new_is_auth_required.json |
Updates initialization output. |
e2e/testdata/acp/claude_code_failed_turn_with_no_login_is_auth_required.json |
Updates initialization output. |
e2e/testdata/acp/chatgpt_authenticate_cancels_a_claude_login_in_flight.json |
Updates initialization output. |
e2e/testdata/acp/cancel_request_during_claude_authenticate_kills_the_login_process_tree.json |
Updates initialization output. |
e2e/testdata/acp/cancel_request_during_authenticate_stops_the_listener.json |
Updates initialization output. |
e2e/testdata/acp/bad_json_keeps_connection_open.json |
Updates initialization output. |
e2e/testdata/acp/authenticate_with_a_terminal_or_unknown_method_is_invalid.json |
Updates initialization output. |
e2e/testdata/acp/authenticate_then_prompt_succeeds.json |
Updates initialization output. |
e2e/testdata/acp/authenticate_claude_code_runs_the_cli_login_then_prompt_succeeds.json |
Updates initialization output. |
e2e/testdata/acp/authenticate_chatgpt_stores_the_login.json |
Updates initialization output. |
e2e/testdata/acp/attachment_for_a_model_without_image_input_is_invalid_params.json |
Covers text-only model rejection. |
e2e/testdata/acp/a_compaction_sends_the_entity_lifecycle_when_the_client_has_the_capability.json |
Updates initialization output. |
e2e/testdata/acp/a_compaction_sends_no_compaction_frame_without_the_capability.json |
Updates initialization output. |
e2e/testdata/acp/a_claude_code_compaction_in_a_turn_sends_the_entity_lifecycle.json |
Updates initialization output. |
e2e/acp_session_rows_test.go |
Removes obsolete image rejection row. |
e2e/acp_scenarios_test.go |
Registers prompt-content scenarios. |
e2e/acp_prompt_content_rows_test.go |
Adds ACP attachment contract rows. |
docs/design/acp-wasm.md |
Updates ACP host design. |
docs/architecture.md |
Specifies attachment behavior and capabilities. |
CHANGELOG.md |
Documents the user-visible feature. |
🧠 Review effort: Balanced
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
andybons
force-pushed
the
feat/acp-prompt-images
branch
2 times, most recently
from
October 14, 2026 03:07
9a81844 to
223ca57
Compare
Problem: harness acp advertised no promptCapabilities and answered -32602 to every prompt block except text and resource_link, so Zed kept its Image, Selection, and Branch Diff attachments off. The runtime already admits, checks, stores, and sends attachments, and replay already returns user images. Design: promptParts maps an image block (png, jpeg, gif, webp) and a resource block with a blob (also pdf) to a protocol blob part, and a resource block with text to a text part wrapped in a resource element with the uri attribute. The adapter decodes the base64; the runtime keeps the media type, size, and content checks. audio stays -32602 and is not advertised. initialize advertises promptCapabilities image and embeddedContext on every host. Input admission refuses an image for a model that the models.dev facts list without image input, with an invalid_request that names the model; the generator of internal/modelmeta records the flag. An admission error has the reason alone as its message and the input ID in details. rpcError sends the message, data.code, and data.reason. The wasm size budgets follow the rule. Semantic change: image and resource blocks are accepted; initialize gains promptCapabilities; HTTP and Go callers get the image-input check. Verification: ACP rows (image, text resource, pdf resource, invalid content, model without image input, initialize) on binary, in-process, and Node wasm; an HTTP contract test on serve and runtime; go test -race ./...; HARNESS_E2E_RUNTIME=1 HARNESS_E2E_NODE=1 go test -race ./e2e/.
andybons
force-pushed
the
feat/acp-prompt-images
branch
from
October 14, 2026 03:14
223ca57 to
d4ec776
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
harness acpadvertised nopromptCapabilitiesand answered -32602 to every prompt block excepttextandresource_link, so Zed kept its Image, Selection, and Branch Diff attachments off. The runtime already admits, checks, stores, and sends attachments (png, jpeg, gif, webp, pdf, at most 20 MiB), and replay already returns user images.Design
Spec:
docs/architecture.mdACPInitialize,Prompts, andError mapping, andInputs(the attachment checks, now with the model check). Design: docs/design/acp-wasm.md section 14 and section 7.2.imageaccepts only png, jpeg, gif, and webp; a PDF in animageblock is -32602. Aresourcewithblobis an attachment (PDF allowed). The adapter only decodes base64; the runtime keeps the media type, size, and content checks.resourcewithtextbecomes a text part wrapped in<resource uri="…">and</resource>.audiostays -32602 and is not advertised.initializeadvertisespromptCapabilities {image, embeddedContext}on every host.Session.Prompt) owns the model check, after the part checks: an image attachment for a model that themodelmetafacts (models.dev input modalities) list without image input isinvalid_requestthat names the model. A model that the facts lack passes. A later model change is not rechecked. HTTP and Go callers get the same check.rpcError. Forinvalid_requestthe message is the reason alone,data.codeisinvalid_request, anddata.reasonholds the same text. Admission errors carry the input ID indetails.input_id(HTTP) and keep it out of the message.harnesstestAnthropic decoder reports user image and document blocks (base64 as a data URI, URL sources as the URL).Semantic change
Image and resource blocks are accepted;
initializegainspromptCapabilities; an image for a text-only catalog model is -32602 naming the model. The old rowprompt_with_an_image_block_is_invalidis replaced byunsupported_content_is_invalid_params. All initialize goldens gainpromptCapabilities.Runtime.NoImageInputandturn.Capabilities.NoImageInputare new.Verification
Rows (all on binary, in-process, and with
HARNESS_E2E_NODE=1Node wasm):initialize_advertises_image_and_embedded_context,image_block_reaches_the_model_as_an_image,text_resource_reaches_the_model_with_its_uri,pdf_resource_blob_reaches_the_model,unsupported_content_is_invalid_params(svg, audio, zip resource, bad base64, undecodable png),attachment_for_a_model_without_image_input_is_invalid_params. Each was seen red first (unsupported type "image"/"resource", missingpromptCapabilities).go build,go vet,go test -race ./...,gofmt, golangci-lint,HARNESS_E2E_RUNTIME=1 HARNESS_E2E_NODE=1 go test -race ./e2e/pass. No protocol type changed, sogo generate ./protocolhas no diff.CI job times
No job grew by more than 4 s. The wasm size budgets on main already cover the module, so this PR does not change them.