Skip to content

JIT runner registration leaks when a container stops before its job completes #9

Description

@hanneslipusch

On v1.0.10, when a container stops before GitHub reports the job complete (idle sleepAfter stop, OOM, lost communication), the JIT registration stays offline until GitHub's 24 h reaper. runnerStopped moves the job to stopped-awaiting-completion but never calls deleteGitHubRunner; only the provisioning-failure and alarm-recovery paths do.

On our fleet this leaves ~30 offline cf-* registrations in steady state (88 registered, 31 offline, all Cloudflare), which makes a starting container indistinguishable from a dead one in the runner list.

Fix that works for us, in runnerStopped before the stopped-awaiting-completion update:

// A stopped container can never run its job, so drop its JIT registration now.
await this.retryRunnerCleanup(job);

A stopped container can never pick the job up again, so the registration is safe to remove immediately; retryRunnerCleanup already handles retries and the not-found case.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions