Skip to content

fix(slack): deduplicate event retries across replicas - #822

Draft
izadoesdev wants to merge 3 commits into
stagingfrom
codex/slack-event-dedupe
Draft

izadoesdev wants to merge 3 commits into
stagingfrom
codex/slack-event-dedupe

Conversation

@izadoesdev

@izadoesdev izadoesdev commented Sep 17, 2026

Copy link
Copy Markdown
Member

Slack event retries could run the agent again on another process. Claim each event atomically in the existing Redis cache before Bolt dispatches it, and retain the claim for 25 hours. Claims remain after handler failures because an action may already have happened.

Stops and message deletions bypass the claim. Other events fail closed when Redis is unavailable; explicit human mentions and plain DMs receive a retry notice. Bot messages and unrelated channel conversations stay silent, including messages identified only by a bot profile.

Validation: 74 Slack tests, root lint, all 33 typecheck tasks, and all 29 pre-push test tasks pass. Two native Bolt instances plus an isolated, nonpersistent Redis server verified duplicate suppression, stop bypass, queue cancellation, and later requests in combination with #819. Independent code review covered the rebased change and the bot-profile regression fix.

No migrations, new packages, scopes, reinstall, or account linking. Depends on #819: its production cancellation handlers must land before the stop bypass is enabled. This PR will be rebased onto that merged staging commit before final review. Deduplication limits event attempts during the cache window; it does not provide durable delivery or transaction-level exactly-once actions. No live Slack workspace was used.

Research: Slack Events API retries and event IDs, Socket Mode delivery across connections. AI-assisted implementation and review.

@vercel

vercel Bot commented Sep 17, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
databuddy-status Ready Ready Preview Sep 17, 2026 6:42pm UTC
2 Skipped Deployments
Project Deployment Actions Updated
dashboard Skipped Skipped Sep 17, 2026 6:42pm UTC
documentation Skipped Skipped Sep 17, 2026 6:42pm UTC

@coderabbitai

coderabbitai Bot commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 9d7a8485-6d77-4176-ad44-6b36dfb1d944

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@unkey-deploy

unkey-deploy Bot commented Sep 17, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Unkey Deploy

Name Status Preview Inspect Updated (UTC)
links (preview) Ready Visit Preview Inspect Sep 17, 2026 6:42pm

@greptile-apps

greptile-apps Bot commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 4/5

The PR is not yet safe to merge because stop mentions can bypass distributed deduplication while still launching ordinary agent runs, and the explicit repository typing requirement must also be satisfied.

Findings

  1. P1 Stop Mentions Bypass Deduplication
  2. P2 Unknown Type Violates Requirement

Summary

This PR adds a Redis-backed Slack Events API claim middleware with a 25-hour TTL and regression coverage for delivery retries, outages, handler failures, and listener bypasses.

  • Atomically scopes claims by Slack app, workspace, and event identifiers.
  • Suppresses duplicate deliveries across app replicas and restarts.
  • Fails closed when Redis coordination is unavailable and selectively notifies directly addressed users.
  • Bypasses coordination for message deletions and stop-text events; the latter does not match production cancellation routing and can still execute twice across replicas.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Slack event envelope] --> B{Has event_id?}
  B -- No --> L[Bolt listeners]
  B -- Yes --> C{Deletion or text equals stop?}
  C -- Yes --> L
  C -- No --> D[Redis SET NX with 25-hour TTL]
  D -- Claim acquired --> L
  D -- Existing claim --> E[Suppress duplicate]
  D -- Redis error --> F{Human mention or plain DM?}
  F -- Yes --> G[Attempt ephemeral failure notice]
  F -- No --> H[Drop event quietly]
  L --> I[Production routing]
  I --> J{Process-local message claim}
  J -- Acquired --> K[Execute agent or cancellation handler]
  J -- Duplicate in same replica --> E
Loading

Reviews (1) · Last reviewed commit: "fix(slack): suppress outage notices for ..."

Comment on lines +31 to +34
if (
event.subtype === "message_deleted" ||
stripLeadingMention(text).trim().toLowerCase() === "stop"
) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Stop Mentions Bypass Deduplication

A normal app mention whose text is stop bypasses the cross-replica Redis claim. The production app-mention listener does not treat stop as cancellation; it routes the text into handleAgentRun after only a process-local dedupe. If Slack redelivers the event to another replica, the agent can therefore run twice. The test's synthetic stop branch does not exist in production routing.

Suggested change
if (
event.subtype === "message_deleted" ||
stripLeadingMention(text).trim().toLowerCase() === "stop"
) {
if (event.subtype === "message_deleted") {

Comment thread apps/slack/src/slack/event-dedupe.test.ts
@izadoesdev
izadoesdev marked this pull request as draft September 17, 2026 18:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant