Overview
Testing a change to the automations, or to the sync workflow, currently means setting up a separate
GitHub organization with its own app, secrets, and Google service account.
learningequality/test-actions already has all of that except two secrets. Make it the standard
place to test, and a developer needs no setup at all.
Complexity: Low
Target branch: main
Context
The automations run in a consumer repo and need the bot app and the secrets. test-actions has
both.
Several of them also gate on is-contributor, so they only run for an author outside the
organization. Triggering those needs a second GitHub account that is not a member, which needs no
permissions, secrets or app of its own.
The sync workflow walks the whole organization and writes to what it finds. Containment today comes
from a token scoped to repos the developer owns, which is the reason a separate organization is
needed at all.
The Change
Narrow a sync run to one repo. Add --only <repo> to scripts/sync-automation-template.js,
exposed as an only input on the workflow. A developer dispatches with only: test-actions and it
runs as the real app against one repo, so containment is an input rather than a credential they
have to build.
Make test-actions a real consumer. Remove the three callers pointing at
MisRob/.github@community-automations-updates, which belong to the process #97 replaced, and add an
unedited copy of automation-template.yml at .github/workflows/automation.yml.
Point the spreadsheet at a test sheet. Create a Google Sheet, share it with the production
service account's client_email as an editor, and set CONTRIBUTIONS_SPREADSHEET_ID and
CONTRIBUTIONS_SHEET_NAME as repository secrets on test-actions. Repository secrets take
precedence over organization ones, so no new service account is needed.
They stay unset by default rather than inheriting production, because an unset id fails loudly while
a production id would quietly write test rows into the real sheet.
Document it. Add docs/testing-automations.md and link it from docs/automation.md. Put the
setup steps for a separate organization in docs/testing-in-a-separate-org.md, so the page a
developer reads first stays short.
Out of Scope
- Changing what any automation does.
- Changing the separate sandbox organization. It stays the only way to test against a
non-production app, and its setup is documented rather than altered.
Acceptance Criteria
Testing
- Dispatch with
only: test-actions and dry_run. Expect one row and no other repo.
- Drift the copy, dispatch with
only: test-actions. Expect one pull request, and none elsewhere.
- Open a pull request in
test-actions from an account outside the organization. Expect the
contributor reply, and the row in the test sheet. Opening it as a member instead skips both,
because is-contributor is false.
- Dispatch without
only. Expect every consumer, including test-actions.
References
AI usage
I used Claude Code to check what test-actions already provides and to draft this issue. I decided
the approach, and verified the secret visibility and existing callers against the repo.
Overview
Testing a change to the automations, or to the sync workflow, currently means setting up a separate
GitHub organization with its own app, secrets, and Google service account.
learningequality/test-actionsalready has all of that except two secrets. Make it the standardplace to test, and a developer needs no setup at all.
Complexity: Low
Target branch: main
Context
The automations run in a consumer repo and need the bot app and the secrets.
test-actionshasboth.
Several of them also gate on
is-contributor, so they only run for an author outside theorganization. Triggering those needs a second GitHub account that is not a member, which needs no
permissions, secrets or app of its own.
The sync workflow walks the whole organization and writes to what it finds. Containment today comes
from a token scoped to repos the developer owns, which is the reason a separate organization is
needed at all.
The Change
Narrow a sync run to one repo. Add
--only <repo>toscripts/sync-automation-template.js,exposed as an
onlyinput on the workflow. A developer dispatches withonly: test-actionsand itruns as the real app against one repo, so containment is an input rather than a credential they
have to build.
Make
test-actionsa real consumer. Remove the three callers pointing atMisRob/.github@community-automations-updates, which belong to the process #97 replaced, and add anunedited copy of
automation-template.ymlat.github/workflows/automation.yml.Point the spreadsheet at a test sheet. Create a Google Sheet, share it with the production
service account's
client_emailas an editor, and setCONTRIBUTIONS_SPREADSHEET_IDandCONTRIBUTIONS_SHEET_NAMEas repository secrets ontest-actions. Repository secrets takeprecedence over organization ones, so no new service account is needed.
They stay unset by default rather than inheriting production, because an unset id fails loudly while
a production id would quietly write test rows into the real sheet.
Document it. Add
docs/testing-automations.mdand link it fromdocs/automation.md. Put thesetup steps for a separate organization in
docs/testing-in-a-separate-org.md, so the page adeveloper reads first stays short.
Out of Scope
non-production app, and its setup is documented rather than altered.
Acceptance Criteria
--only <repo>narrows discovery to that repo, with a test, and the workflow exposes it.MisRobcallers are gone andtest-actionsholds an unedited copy of the template.repository secrets on
test-actions.test-actionsruns the automations with no failed step. The onesgated on
is-contributorskip for an org member, so a second account outside theorganization is needed to exercise those.
docs/testing-automations.mdcovers both flows and notes that a branch-pinned caller at thediscovered path is proposed against on every run, so experimental ones belong elsewhere.
docs/testing-in-a-separate-org.mdholds the setup steps for that route, and says when it isworth the effort.
Testing
only: test-actionsanddry_run. Expect one row and no other repo.only: test-actions. Expect one pull request, and none elsewhere.test-actionsfrom an account outside the organization. Expect thecontributor reply, and the row in the test sheet. Opening it as a member instead skips both,
because
is-contributoris false.only. Expect every consumer, includingtest-actions.References
AI usage
I used Claude Code to check what
test-actionsalready provides and to draft this issue. I decidedthe approach, and verified the secret visibility and existing callers against the repo.