Skip to content

Release test gate never fails: steps.test.conclusion is always success after continue-on-error #176

Description

@rahlk

Problem

.github/workflows/release.yml gates the release on the test step with continue-on-error: true (line 78) and then tests steps.test.conclusion == 'failure' (line 93 and 102) to delete the tag and fail the job. In GitHub Actions conclusion is the result after continue-on-error is applied and is always success; only outcome carries the real result. Both conditions are therefore never true: a red test suite still builds, publishes the GitHub Release and PyPI wheels, and the tag is never deleted.

Found during the codeanalyzer-iac release-pipeline review (codellm-devkit/codeanalyzer-iac#3), whose workflow was mirrored from this one; the iac copy was fixed in codellm-devkit/codeanalyzer-iac#6 and the fail-closed path was then exercised for real by two aborted v0.1.0 tags (codeanalyzer-iac runs 34003621288 and 34004180477).

Scope boundary

In: make the release gate fail closed. Either drop continue-on-error so the test step fails the job and add one if: failure() && startsWith(github.ref, 'refs/tags/') delete-tag step placed after the last pre-publish step (the iac shape), or keep the current shape and switch both conditions to steps.test.outcome == 'failure'. actionlint clean.
Out: any other change to the release workflow; the tap or PyPI configuration.

Goals

  • A failing test step prevents wheel build, Release creation, and PyPI publish.
  • The tag is deleted on any pre-publish failure and never after a Release exists.
  • workflow_dispatch runs still publish nothing and delete nothing.

Caveats and known risks

  • The delete-tag step must stay before the publish steps; placed last it would delete a tag whose Release already exists.

Definition of done

  • Merged; the next tag's run shows the gate step without continue-on-error (or with outcome) and actionlint passes in CI.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions