Skip to content

fix(agent): empty assistant turn never completes the node - #74

Merged
wolfy-j merged 2 commits into
masterfrom
fix/agent-empty-result-completion
Sep 2, 2026
Merged

fix(agent): empty assistant turn never completes the node#74
wolfy-j merged 2 commits into
masterfrom
fix/agent-empty-result-completion

Conversation

@wolfy-j

@wolfy-j wolfy-j commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Problem

tool_calling = "auto" completed the agent node on an empty assistant turn: agent_result.result ~= nil is true for "" (the empty string is truthy in Lua), so a model that had just created its answer through CreateArtifact and then returned {"result":"","tool_calls":[]} ended the node "successfully" with a zero-byte result. none rejected "" but accepted whitespace. Downstream consumers (kickside/research deliver) received an empty report and recorded a false success. Report: "Kickside Research false-success with empty report and orphaned artifact".

Two adjacent defects surfaced while proving the fix at loop level:

  • The NO_TOOLS_CALLED feedback observation was queued after the turn's durable yield, so the next prompt (built from persisted history) did not contain it; the model only saw it one turn late.
  • Control artifacts were persisted without node_id, so they could not be attributed to the creating agent node.

Change

  • has_usable_result: text needs a non-whitespace character, a structured table needs at least one field, nil/false never complete. Shared by none and auto.
  • An empty turn in auto records FEEDBACK.EMPTY_RESULT (key <iteration>_empty_result); a nil result keeps NO_TOOLS_CALLED.
  • check_completion reports whether it recorded feedback; finalize_iteration yields immediately in that case so the feedback is applied before the next prompt (yield waits for the orchestrator to apply the commit; a bare submit only queues it).
  • process_artifacts sets node_id on the artifact row.

Tests

  • check_completion_test: 19 cases across none / auto / any (empty, whitespace, nil, false, empty table, structured, numeric, pending tool calls, min_iterations, exit-tool hint).
  • agent_empty_result_test (loop level, deterministic model stub): the model creates an artifact through a control tool, returns an empty turn, then answers once the feedback is in its context. Asserts the artifact is retained with node_id, exactly one empty-result observation exists, and the workflow output is the real answer. A second case runs all-empty turns to the iteration limit and asserts COMPLETED_FAILURE with no output.
  • control_handler_test: artifact rows carry node_id.
  • Full wippy test --profile sqlite suite: 994 passed, 0 failed (two consecutive runs).

https://claude.ai/code/session_01TLD4uyqDLGD9JfgZESaaXH

An empty string is truthy in Lua, so tool_calling=auto treated a final turn with
no text and no tool calls as a successful completion and routed "" downstream as
the node result. NONE rejected "" but not whitespace. Both modes now share one
predicate: text needs a non-whitespace character, a structured table needs at
least one field, nil/false never complete. An empty turn records an
EMPTY_RESULT observation so the model is asked for a real answer.

The observation check_completion records was queued after the turn's durable
yield, so the model only saw it one turn late; finalize now yields right after
recording feedback. Control artifacts are persisted with the creating node's
id so they can be attributed and recovered.

Tests: check_completion matrix across NONE/AUTO/ANY (empty, whitespace, nil,
false, empty table, structured, pending tool calls, min_iterations); a
loop-level run where the model creates an artifact, returns an empty turn, then
answers (artifact retained, exactly one empty-result observation, output is the
real answer); an all-empty run failing at the iteration limit.

Claude-Session: https://claude.ai/code/session_01TLD4uyqDLGD9JfgZESaaXH
wippy 0.3.35a lint loads entries from lock paths only, so the workspace-replaced
module has to be listed for the harness to link; install leaves the entry as is,
so verify-lock stays clean.

Claude-Session: https://claude.ai/code/session_01TLD4uyqDLGD9JfgZESaaXH
@wolfy-j
wolfy-j merged commit 56405d3 into master Sep 2, 2026
11 of 12 checks passed
@wolfy-j
wolfy-j deleted the fix/agent-empty-result-completion branch September 2, 2026 04:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant