Skip to content

[agentic-token-optimizer] Optimize: Agentic Workflow AIC Usage Optimizer — Prompt Verbosity, Sub-Agent Extraction, Batched Reads #481

Description

@github-actions

Target Workflow

Agentic Workflow AIC Usage Optimizer (agentic-token-optimizer)
Selected as the highest-AIC workflow not optimized in the last 14 days (last optimized 2026-08-20, 8 days ago). Second-highest total AIC in the 7-day window at 1,054 AI credits across 4 runs.


Spend Profile

Metric Value
Analysis period 2026-08-24 → 2026-08-27
Completed runs 4
Total AIC 1,054
Avg AIC / run 264
Token data Not captured (null in run log)
Avg duration 7–9 min / run
GitHub API calls / run 10 (consistent)
Error rate 0% (all 4 runs succeeded)
AIC variability High — range 163–328 (1.9× spread)
Per-run AIC breakdown
Run Started AIC Duration Conclusion
§32739762209 2026-08-24 307.4 7.9 min success
§32860631794 2026-08-25 327.9 8.8 min success
§32981366187 2026-08-26 163.0 6.2 min success
§33101714840 2026-08-27 256.1 8.0 min success

The 163-AIC outlier (2026-08-26) suggests the agent selected a simpler target workflow and/or benefited from prompt caching that day.


Recommendations

1. Trim prompt verbosity — ~40–55 AIC/run

Action: Condense the three heavily-documented instructional sections in the prompt body.

The prompt is ~470 lines long. Of those, roughly 150–170 lines are instructional scaffolding that repeats on every run regardless of which workflow is selected:

  • Data Access Guidelines (## Data Access Guidelines): Contains 4 annotated bash code blocks with / annotations and inline comments. These patterns are already well-established — compress to 2–3 key bullet points and a single inline example.
  • Inline Sub-Agent Scoring Rubric (### Inline Sub-Agent Opportunity Analysis): Includes a 4-row scoring table, a 3-tier guidance block, two bullet lists ("Smaller models are a good fit for..." and "Keep with the main agent when..."), and a word limit reminder. This can be halved without loss of fidelity.
  • Phase 3 workflow-source bash examples: Three separate bash snippets (full read, frontmatter-only, prompt-body-only) are embedded as documentation. The agent should use the most appropriate one; it doesn't need all three on every run.

Estimated savings: 150–170 lines × 2–3 tokens/line → ~400–500 tokens of input overhead per run. At the observed AIC/token ratio, that translates to ~15–20% AIC reduction ≈ 40–55 AIC/run.

Evidence: The 163-AIC run (2026-08-26) — the lowest-cost run — also had the shortest duration (6.2 min), suggesting that prompt-read overhead and analysis scope are the primary cost levers, not reliability or retries.


2. Inline sub-agent for workflow source analysis — ~25–40 AIC/run

Action: Extract Phase 3 + Phase 4 structural checks into an inline sub-agent using a smaller model.

Reading the target workflow's .md source and classifying its structure (configured tools, whether it has ## agent: blocks, major section headers, repeated setup prefixes) is entirely extractive and independent of the main agent's strategic reasoning. It is a strong candidate for a smaller model.

Score breakdown:

Dimension Score Rationale
Independence 3/3 Reads a fixed file path; needs no other phase outputs
Small-model adequacy 3/3 File read → field extraction → classification; no synthesis needed
Parallelism 2/2 Can run while the main agent finishes Phase 1/2 data aggregation
Size 2/2 470-line source + structured output; substantial but bounded
Total 10/10 Strong candidate

Proposed change: Add an ## agent: block after Phase 2 with a concise prompt:

## agent: analyze-workflow-source
model: small
tools: [bash]

Read the workflow source at the path provided in `$TARGET_WF_PATH` using:
  gh api "repos/$REPO/contents/$TARGET_WF_PATH" --jq '.content' | base64 -d

Extract and output as JSON:
- tools: list of configured tool names (from frontmatter)
- has_subagents: boolean (whether any "## agent:" block exists in the prompt body)
- major_sections: list of "##"-level section titles in the prompt body
- repeated_setup_lines: any lines appearing verbatim in 2+ section openings

The main agent then consumes this structured output instead of making raw gh api calls and parsing the source itself. This offloads ~80–120 lines of iterative file-reading and analysis work to a cheaper model call.

Estimated savings: ~25–40 AIC/run based on the sub-task's share of total run cost.


3. Batch file reads in Phase 1 — ~5–10 AIC/run

Action: Replace sequential reads of top-workflows.json, optimization-log.json, and optionally a daily snapshot into a single parallel tool call block.

Currently the agent reads these files separately across Phase 1 steps, adding multiple turn-round-trips. Issuing all three reads in one parallel block eliminates the intermediate inference steps between them.

Proposed change: At the start of Phase 1, consolidate:

# Current — 3 separate tool calls
cat /tmp/gh-aw/token-audit/top-workflows.json
cat /tmp/gh-aw/repo-memory/default/optimization-log.json
cat /tmp/gh-aw/repo-memory/default/$(date +%Y-%m-%d).json

into a single bash call:

jq -s '{"top": .[0].top_workflows, "log": .[1]}' \
  /tmp/gh-aw/token-audit/top-workflows.json \
  /tmp/gh-aw/repo-memory/default/optimization-log.json

This reduces Phase 1 turn count by 1–2 and eliminates redundant context re-reads.

Estimated savings: ~5–10 AIC/run.


Tool Usage Audit

Tool Usage Status
bash Used for all data reads, gh api calls, jq processing Keep
github (issues toolset) Used to publish optimization issue via safeoutputs Keep
repo-memory Read optimization-log.json, write updated log Keep

All configured tools are used in every successful run. No removal candidates.


Caveats

  • Token counts are null in run logs for this workflow; AIC-based estimates assume proportional token scaling. Savings ranges are conservative (low end) estimates.
  • The 163-AIC outlier (2026-08-26) may reflect an unusually short target workflow or prompt-cache hit; excluding it raises the average to 297 AIC/run and increases the savings opportunity.
  • Inline sub-agent savings depend on the model cost differential between main and sub-agent models; if a small model is not available in this environment, the savings may be lower.
  • Sample size: 4 completed runs. Recommendations should be validated across 5+ runs post-change.

References:

Generated by Agentic Workflow AIC Usage Optimizer · 275.3 AIC · ⊞ 21.6K ·

  • expires on Sep 4, 2026, 6:40 PM UTC

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions