Skip to content

feat(ai): thinking mode switch, vision image input, and API host normalization fixes - #1596

Open
lb1038678031 wants to merge 4 commits into
apache:mainfrom
lb1038678031:feat/ai-enhancement
Open

feat(ai): thinking mode switch, vision image input, and API host normalization fixes#1596
lb1038678031 wants to merge 4 commits into
apache:mainfrom
lb1038678031:feat/ai-enhancement

Conversation

@lb1038678031

Copy link
Copy Markdown

Resubmit of #1594, addressing @LinkinStars's feedback there. Implements #1595 (feature clarification).

What

Three improvements to the AI feature, shipped as independent commits:

  1. fix(ai): normalize API host and harden AI model listing

    • The models endpoint used to call api_host + "/v1/models" while the chat
      client auto-appends /v1. Configuring a base URL that already contains /v1
      (common with OpenAI-compatible gateways such as SenseNova, SiliconFlow,
      OneAPI relays...) produced /v1/v1/models, and the gateway's gRPC-style
      NOT_FOUND error was passed through to the UI.
    • New shared helper schema.NormalizeAPIHost() trims slashes/whitespace,
      appends /v1 when missing, and keeps /v1beta/* endpoints intact.
    • Upstream error bodies are summarized (error.code/message) instead of being
      echoed verbatim; image data is never logged.
    • Stored-key lookup prefers the chosen provider before falling back to host
      matching.
    • Default provider list: Gemini points at its OpenAI-compatible endpoint;
      Anthropic removed because the backend only speaks the OpenAI protocol.
  2. feat(ai): per-provider thinking mode

    • New thinking_mode switch on each provider in Admin → AI settings.
    • When enabled, a custom http.RoundTripper merges "enable_thinking": true
      into chat completion request bodies — the convention used by
      reasoning-capable OpenAI-compatible gateways (DeepSeek V4, Qwen/DashScope,
      SenseNova, vLLM/SGLang, ...). The existing reasoning_content streaming
      path renders the thought process in the bubble UI unchanged.
  3. feat(ai): vision image input behind an admin switch

    • New vision_enabled switch per provider; exposed to clients via site info
      as ai_vision_enabled.
    • Users can attach up to 4 PNG/JPEG/WebP images (≤4 MB decoded each) to the
      first message of a turn; images travel as base64 data URLs or HTTPS links
      and are converted into MultiContent parts.
    • Conversation history stores a textual placeholder instead of raw image data
      (no DB schema change).

Security note (from #1594)

The base64 constant flagged during review was the encoding of a standard 1×1
pixel PNG placeholder used in a unit test, not a credential. To avoid any
misreading, 158516b constructs the placeholder at runtime from the canonical
PNG signature bytes; the source no longer contains any base64 blob.

Testing

  • go test ./internal/schema/ ./internal/controller/ ./internal/service/siteinfo/ ./internal/migrations/
    — new table tests for host normalization, transport injection, and image
    validation (count/type/size).
  • Manual smoke against a SenseNova OpenAI-compatible endpoint: model listing
    succeeds with both root and /v1 base URLs; reasoning_content appears with
    thinking enabled; image questions answered by a vision-capable model.

enhancer and others added 4 commits August 27, 2026 18:20
- Unify host normalization between model listing and chat client so both
  hosts with or without the /v1 suffix resolve identically (fixes upstream
  NOT_FOUND when a full /v1 base URL is configured).
- Trim trailing slashes before building the models URL.
- Summarize upstream error bodies instead of echoing the full response.
- Prefer the chosen provider's stored key over host-based key lookup.
- Return nil client when AI is disabled instead of an invalid client.
- Fix default provider list: use Gemini's OpenAI-compatible endpoint and
  drop Anthropic (not supported by the OpenAI-compatible backend).
Thinking mode (per provider):
- New thinking_mode field on the AI provider config.
- When enabled, a custom http.RoundTripper merges enable_thinking=true
  into chat completion request bodies before they are sent.

Vision input (per provider):
- New vision_enabled field; surfaced to clients via site info
  ai_vision_enabled so the front end can show the attach button.
- Chat requests accept images on the first user message: base64 data
  URLs or HTTPS links, max 4 per message and 4MB decoded each.
- Images are converted into MultiContent parts for vision models.
- Conversation history keeps an '[图片]' placeholder instead of image data.

Also stops logging full request bodies that contain image data.
- Admin AI settings page gains deep-thinking and image-input switches
  per provider, saved with the rest of the provider config.
- Site settings store tracks ai_vision_enabled.
- AiAssistant sender shows an attach-image action when vision is on,
  previews thumbnails, validates count/type/size client-side and sends
  base64 images with the first message. New-conversation first hop now
  carries images through as well.
- zh_CN/en_US translations for all new strings.
The embedded base64 constant in the vision test was mistaken for a leaked
credential. Build the placeholder image from the canonical PNG signature
bytes at runtime instead, so no base64 blob appears in the source.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant