Skip to content

Docs/abot world multi session - #45

Open
youngmagician114514 wants to merge 2 commits into
Tele-AI:mainfrom
youngmagician114514:docs/abot-world-multi-session
Open

Docs/abot world multi session#45
youngmagician114514 wants to merge 2 commits into
Tele-AI:mainfrom
youngmagician114514:docs/abot-world-multi-session

Conversation

@youngmagician114514

Copy link
Copy Markdown
Contributor

Description

Adds a bilingual technical blog post describing ABot-World multi-session serving, including session state isolation,
compatible batching, multi-GPU worker ownership, migration, backpressure, and fallback behavior.

The article also presents the validated four-H100 trace results for the multi-session serving path.

Motivation

ABot-World multi-session serving was introduced in #36, but the implementation design and measured serving benefits
were not documented in the TeleFuser Technical Blog.

This PR explains how per-session state remains isolated while compatible sessions share model execution, and records
the observed batching, admission, latency, and delivery results.

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Performance improvement
  • Code refactoring
  • Documentation update
  • Other (please describe):

Changes Made

  • Add English and Chinese ABot multi-session serving blog posts.
  • Document session isolation, compatible batching, process-NCCL workers, migration, backpressure, and fallback behavior.
  • Record the four-H100, 30-minute trace results, including a 15.1% reduction in model calls relative to all-B1 execution.
  • Add the article to both blog indexes and the MkDocs navigation.

Testing

  • Unit tests pass (pytest tests/)
  • Manual testing performed
  • Benchmarks added/updated (if applicable)

Test commands:

git diff --check

git diff --check passes. The local environment does not currently provide the mkdocs command, so
mkdocs build --strict was not run locally.

## Checklist

- [ ] Code follows the project's coding standards (ruff)
- [ ] Pre-commit hooks pass (pre-commit run --all-files)
- [ ] All tests pass (pytest tests/)
- [ ] New tests added for new functionality
- [x] Documentation updated (README, CLAUDE.md, docstrings)
- [x] Commit messages are clear and descriptive
- [x] PR title follows the convention: [TYPE] Brief description

## Related Issues

Follow-up documentation for #36.

## Additional Notes

This is a documentation-only follow-up to the merged ABot multi-session serving PR. It does not change runtime code,
configuration defaults, or serving behavior.

The blog includes both English and Chinese versions and follows the existing TeleFuser Technical Blog structure.

## GPU Architecture Support

Not applicable; this PR does not add or modify GPU kernels.

- [ ] SM80 (Ampere, Ada Lovelace)
- [ ] SM90 (Hopper H100)
- [ ] SM100+ (Blackwell)

## Performance Impact

This documentation-only PR has no runtime performance impact.

The documented four-H100 trace completed 5,475 successful model calls for 6,452 chunk items. Compatible B2/B3
batching reduced the number of model calls by 977, or 15.1%, compared with all-B1 execution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant