Add the provider-agnostic session engine with tiered context compaction - #6
Open
Malcolmnixon wants to merge 1 commit into
Open
Malcolmnixon wants to merge 1 commit into
Malcolmnixon wants to merge 1 commit into
Conversation
AgentKit shipped guarded tools but no session lifecycle, so a long-running agent was impossible. This adds AgentKitSessions: an AgentKit-owned session that keeps its own transcript out of session and, when the context window fills, consolidates older history into fixed-budget tiers, disposes the provider session and creates a fresh one seeded with the preserved content. Rotation rather than in-place reduction is the only reduction mechanism both provider shapes support - one sends the whole history again each turn, the other holds it server-side - so the behavior is identical on either. Aging happens only at rotation and only in the tiers that overflowed, so the context is strictly append-only between rotations and a provider prompt cache survives. The context is bounded by construction: system prompt plus tool declarations plus the sum of the tier budgets, asserted when the options are constructed. The summarizer is injected and runs out of session, so the rotation engine is a pure deterministic function and is unit-tested against a fake with no model at all. A saturation signal is raised, not acted on, when a consolidation cannot reduce what it was given. This increment is the engine only; no Copilot or ChatClient wiring. Co-authored-by: Copilot <bot@github.com>
There was a problem hiding this comment.
🟡 Changes recommended
One or more issues must be addressed before approval.
Get a fresh assessment by requesting another Copilot review.
Pull request overview
Adds a provider-agnostic session engine with tiered context compaction, provider rotation, deterministic token accounting, and an in-memory provider implementation.
Changes:
- Adds the Sessions production library and provider/session abstractions.
- Adds comprehensive unit, integration, and XML documentation tests.
- Adds design, verification, requirements, SysML2, solution, and README integration.
File summaries
| File | Description |
|---|---|
| test/DemaConsulting.AgentKit.Sessions.Tests/XmlDocExampleTests.cs | Updated as part of this pull request. |
| test/DemaConsulting.AgentKit.Sessions.Tests/TokenEstimatorTests.cs | Updated as part of this pull request. |
| test/DemaConsulting.AgentKit.Sessions.Tests/SummarizerTests.cs | Updated as part of this pull request. |
| test/DemaConsulting.AgentKit.Sessions.Tests/SessionTestData.cs | Updated as part of this pull request. |
| test/DemaConsulting.AgentKit.Sessions.Tests/ProviderSessionTests.cs | Updated as part of this pull request. |
| test/DemaConsulting.AgentKit.Sessions.Tests/InMemoryProviderSessionTests.cs | Updated as part of this pull request. |
| test/DemaConsulting.AgentKit.Sessions.Tests/FakeSummarizer.cs | Updated as part of this pull request. |
| test/DemaConsulting.AgentKit.Sessions.Tests/DemaConsulting.AgentKit.Sessions.Tests.csproj | Updated as part of this pull request. |
| test/DemaConsulting.AgentKit.Sessions.Tests/ContextUsageTests.cs | Updated as part of this pull request. |
| test/DemaConsulting.AgentKit.Sessions.Tests/ContextLayoutTests.cs | Updated as part of this pull request. |
| test/DemaConsulting.AgentKit.Sessions.Tests/CompactionPolicyTests.cs | Updated as part of this pull request. |
| test/DemaConsulting.AgentKit.Sessions.Tests/AgentSessionTests.cs | Updated as part of this pull request. |
| test/DemaConsulting.AgentKit.Sessions.Tests/AgentSessionOptionsTests.cs | Updated as part of this pull request. |
| src/DemaConsulting.AgentKit.Sessions/DemaConsulting.AgentKit.Sessions.csproj | Updated as part of this pull request. |
| src/DemaConsulting.AgentKit.Sessions/ContextUsage.cs | Updated as part of this pull request. |
| src/DemaConsulting.AgentKit.Sessions/AgentSession.cs | Updated as part of this pull request. |
| requirements.yaml | Updated as part of this pull request. |
| README.md | Updated as part of this pull request. |
| docs/verification/introduction.md | Updated as part of this pull request. |
| docs/verification/definition.yaml | Updated as part of this pull request. |
| docs/verification/agent-kit-sessions/token-estimator.md | Updated as part of this pull request. |
| docs/verification/agent-kit-sessions/summarizer.md | Updated as part of this pull request. |
| docs/verification/agent-kit-sessions/session-transcript.md | Updated as part of this pull request. |
| docs/verification/agent-kit-sessions/provider-session.md | Updated as part of this pull request. |
| docs/verification/agent-kit-sessions/in-memory-provider-session.md | Updated as part of this pull request. |
| docs/verification/agent-kit-sessions/context-usage.md | Updated as part of this pull request. |
| docs/verification/agent-kit-sessions/context-layout.md | Updated as part of this pull request. |
| docs/verification/agent-kit-sessions/compaction-policy.md | Updated as part of this pull request. |
| docs/verification/agent-kit-sessions/compacting-agent-session.md | Updated as part of this pull request. |
| docs/verification/agent-kit-sessions/agent-session.md | Updated as part of this pull request. |
| docs/verification/agent-kit-sessions/agent-session-options.md | Updated as part of this pull request. |
| docs/verification/agent-kit-sessions.md | Updated as part of this pull request. |
| docs/sysml2/views/design-views.sysml | Updated as part of this pull request. |
| docs/sysml2/model/agent-kit-sessions/token-estimator.sysml | Updated as part of this pull request. |
| docs/sysml2/model/agent-kit-sessions/summarizer.sysml | Updated as part of this pull request. |
| docs/sysml2/model/agent-kit-sessions/session-transcript.sysml | Updated as part of this pull request. |
| docs/sysml2/model/agent-kit-sessions/rotation-engine.sysml | Updated as part of this pull request. |
| docs/sysml2/model/agent-kit-sessions/provider-session.sysml | Updated as part of this pull request. |
| docs/sysml2/model/agent-kit-sessions/in-memory-provider-session.sysml | Updated as part of this pull request. |
| docs/sysml2/model/agent-kit-sessions/context-usage.sysml | Updated as part of this pull request. |
| docs/sysml2/model/agent-kit-sessions/context-layout.sysml | Updated as part of this pull request. |
| docs/sysml2/model/agent-kit-sessions/compaction-policy.sysml | Updated as part of this pull request. |
| docs/sysml2/model/agent-kit-sessions/compacting-agent-session.sysml | Updated as part of this pull request. |
| docs/sysml2/model/agent-kit-sessions/agent-session.sysml | Updated as part of this pull request. |
| docs/sysml2/model/agent-kit-sessions/agent-session-options.sysml | Updated as part of this pull request. |
| docs/sysml2/model/agent-kit-sessions.sysml | Updated as part of this pull request. |
| docs/reqstream/agent-kit-sessions/token-estimator.yaml | Updated as part of this pull request. |
| docs/reqstream/agent-kit-sessions/summarizer.yaml | Updated as part of this pull request. |
| docs/reqstream/agent-kit-sessions/session-transcript.yaml | Updated as part of this pull request. |
| docs/reqstream/agent-kit-sessions/rotation-engine.yaml | Updated as part of this pull request. |
| docs/reqstream/agent-kit-sessions/provider-session.yaml | Updated as part of this pull request. |
| docs/reqstream/agent-kit-sessions/platform-requirements.yaml | Updated as part of this pull request. |
| docs/reqstream/agent-kit-sessions/in-memory-provider-session.yaml | Updated as part of this pull request. |
| docs/reqstream/agent-kit-sessions/context-usage.yaml | Updated as part of this pull request. |
| docs/reqstream/agent-kit-sessions/context-layout.yaml | Updated as part of this pull request. |
| docs/reqstream/agent-kit-sessions/compaction-policy.yaml | Updated as part of this pull request. |
| docs/reqstream/agent-kit-sessions/compacting-agent-session.yaml | Updated as part of this pull request. |
| docs/reqstream/agent-kit-sessions/agent-session.yaml | Updated as part of this pull request. |
| docs/reqstream/agent-kit-sessions/agent-session-options.yaml | Updated as part of this pull request. |
| docs/design/introduction.md | Updated as part of this pull request. |
| docs/design/definition.yaml | Updated as part of this pull request. |
| docs/design/agent-kit-sessions/token-estimator.md | Updated as part of this pull request. |
| docs/design/agent-kit-sessions/summarizer.md | Updated as part of this pull request. |
| docs/design/agent-kit-sessions/session-transcript.md | Updated as part of this pull request. |
| docs/design/agent-kit-sessions/rotation-engine.md | Updated as part of this pull request. |
| docs/design/agent-kit-sessions/provider-session.md | Updated as part of this pull request. |
| docs/design/agent-kit-sessions/in-memory-provider-session.md | Updated as part of this pull request. |
| docs/design/agent-kit-sessions/context-usage.md | Updated as part of this pull request. |
| docs/design/agent-kit-sessions/context-layout.md | Updated as part of this pull request. |
| docs/design/agent-kit-sessions/compaction-policy.md | Updated as part of this pull request. |
| docs/design/agent-kit-sessions/compacting-agent-session.md | Updated as part of this pull request. |
| docs/design/agent-kit-sessions/agent-session.md | Updated as part of this pull request. |
| docs/design/agent-kit-sessions/agent-session-options.md | Updated as part of this pull request. |
| AGENTS.md | Updated as part of this pull request. |
| AgentKit.slnx | Updated as part of this pull request. |
Review details
Suppressed comments (4)
src/DemaConsulting.AgentKit.Sessions/AgentSession.cs:138
- The summary says compaction happens before the answer, but the remarks and implementation deliberately compact after the provider returns. This contradicts the public API contract and can mislead callers about whether a send waits for summarization; describe the post-answer ordering in the summary.
src/DemaConsulting.AgentKit.Sessions/CompactingAgentSession.cs:252 - The replacement is installed before the old provider is disposed, but
LayoutandUsageare published only after that await. If an adapter'sDisposeAsyncthrows, the next call uses the replacement seeded fromoutcome.Layoutwhile the engine still exposes the pre-rotation layout, so provider history and engine accounting diverge. Publish the new layout/state before disposal, or otherwise handle disposal failures while preserving a consistent session state.
src/DemaConsulting.AgentKit.Sessions/CompactionPolicy.cs:179 - This public default list is an exposed mutable array. A caller can change it, after which
new CompactionPolicy()reads the corrupted defaults even though the sharedCompactionPolicy.Defaultwas initialized earlier. Publish a read-only wrapper rather than the array itself.
src/DemaConsulting.AgentKit.Sessions/ProviderSession.cs:139 - The
ProviderTurndocumentation promises immutability, but a non-empty caller-supplied list is stored directly. The producer can mutate that list after returning the turn, changing what the engine records and making the returned turn's history unstable. Copy the entries into a read-only collection.
- Files reviewed: 93/93 changed files
- Comments generated: 12
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+264
to
+268
| ArgumentNullException.ThrowIfNull(seed); | ||
| cancellationToken.ThrowIfCancellationRequested(); | ||
|
|
||
| var session = new InMemoryProviderSession(seed, _responder, WindowTokens, ReportsUsage); | ||
| _sessions.Add(session); |
| Text = text; | ||
| Usage = usage; | ||
| RotationOccurred = rotationOccurred; | ||
| Saturations = saturations ?? []; |
|
|
||
| Summarizer = summarizer; | ||
| Instructions = instructions; | ||
| Tools = toolList; |
| SystemTokens = systemTokens; | ||
| ToolDeclarationTokens = toolTokens; | ||
| EffectiveWindowTokens = effective; | ||
| RotationThresholdTokens = (int)(effective * policy.RotationThreshold); |
Comment on lines
+179
to
+186
| // Record the outgoing message in our own transcript before sending it, so the transcript | ||
| // matches what the provider holds even if the turn fails partway through. | ||
| Layout = Layout.WithTranscript(Layout.Transcript.Append(TranscriptEntry.User(message))); | ||
|
|
||
| // Take the turn against the live session. The answer belongs to this session; any rotation | ||
| // below prepares the replacement for the turn after. | ||
| var turn = await _provider.SendAsync(message, cancellationToken).ConfigureAwait(false); | ||
| Layout = Layout.WithTranscript(Layout.Transcript.Append(turn.Entries)); |
| /// Holds one fewer element than <see cref="CompactionPolicy.TierCount"/>, because tier zero | ||
| /// is the transcript rather than a consolidated record. Element zero is tier one. | ||
| /// </remarks> | ||
| public IReadOnlyList<ContextTier> CoarseTiers => _coarseTiers; |
| /// configuration alone — it does not depend on what the session has done — which is what | ||
| /// makes it something an application can reason about before starting. | ||
| /// </remarks> | ||
| public int MaximumBoundTokens => SystemTokens + ToolDeclarationTokens + Policy.TotalTierBudgetTokens; |
Comment on lines
+65
to
+67
| Instructions = instructions; | ||
| Tools = tools; | ||
| History = history; |
| ArgumentOutOfRangeException.ThrowIfNegative(consolidationCount); | ||
|
|
||
| Layout = layout; | ||
| Saturations = saturations; |
| /// <summary> | ||
| /// Gets the entries, oldest first. | ||
| /// </summary> | ||
| public IReadOnlyList<TranscriptEntry> Entries => _entries; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
First increment of the session layer: the provider-agnostic engine. No Copilot or ChatClient wiring yet — that follows.
Why
AgentKit ships guarded tools that behave identically on two backends, but has no session lifecycle, so a long-running agent is impossible. This adds the missing half.
Design, validated by measurement
Rotation, not in-place reduction. One backend resends history each turn, the other keeps it server-side; re-seeding a fresh session is the only reduction both support.
Tiered retention with fixed per-tier budgets, aged only in batches at rotation. Between rotations the context is append-only, which preserves prompt caching; at rotation the cache is invalidated anyway, so consolidation is free there.
Measured over 50 rotations (compressed window, one run per arrangement). Recall by rotations-ago:
Overall 5/23 flat versus 13/17 tiered, and tiered cost 17% less in summarizer tokens (381,440 vs 461,173). A flat rolling summary collapses because repeated re-summarization only ratchets downward — its final context had shrunk to 363 tokens against tiered's 3,158.
Honest cost: tiering yielded about 27% fewer turns of work per rotation, because tier content occupies window space.
Two constraints that came out of measurement
The summarizer runs out of session. Asking the live session to summarize itself consumes context and triggers the provider's own compactor — self-defeating.
Never ask a model to hit a token budget. Requested 9,870-19,741 tokens; got 1,665 and 4,259. Prompt for specificity and content; measure size ourselves.
Scope
Provider-agnostic engine only, deterministic and unit-testable with a fake summarizer. Ships an in-memory provider session so the engine is exercised end to end.
Sessions takes no reference on Core — only \Microsoft.Extensions.AI.Abstractions\ for \AIFunction\ — so an adapter can depend on both with no cycle.
Gates
eviewmark --plan --enforce: full coverage
Full companion artifact set created: SysML2 model, design, verification, requirements, review sets.