Skip to content

Externalise cache keys to prevent unneeded invalidation - #3922

Open
pljones wants to merge 1 commit into
jamulussoftware:mainfrom
pljones:cache-keys
Open

Externalise cache keys to prevent unneeded invalidation#3922
pljones wants to merge 1 commit into
jamulussoftware:mainfrom
pljones:cache-keys

Conversation

@pljones

@pljones pljones commented Aug 22, 2026

Copy link
Copy Markdown
Collaborator

Short description of changes

I noticed whilst I've been working on autobuild.sh that every change invalidates all the caches. Because autobuild.sh itself is included in the cache key calculation. This change replaces that mechanism on macOS and Windows with am externalised list of dependencies. It adds the same mechanism to iOS, which lacked caching.

Github cache clean up will remove unused cache entries with no need to have the cache consumer included in the cache key.

(My work in progress on the Android build uses the same approach, which is where this comes from.)

Scope, guidance, testing and review by myself, coding my VSCode Github CoPilot.

CHANGELOG: Build: improve cache persistence

Context: Fixes an issue?

Caching should depend on what is being cached, rather than the cache consumer.

Does this change need documentation? What needs to be documented and how?

No.

Status of this Pull Request

Tested against the current branch in my own repository before raising the PR here.

What is missing until this pull request can be merged?

Need to do some more testing.

Checklist

  • I've verified that this Pull Request follows the general code principles
  • I tested my code and it does what I want
  • My code follows the style guide
  • I waited some time after this Pull Request was opened and all GitHub checks completed without errors.
  • I've filled all the content above

AUTOBUILD: Please build all targets

@pljones pljones self-assigned this Aug 22, 2026
@pljones pljones added tooling Changes to the automated build system AI AI generated or potentially AI generated labels Aug 22, 2026
@pljones pljones added this to Tracking Aug 22, 2026
@github-project-automation github-project-automation Bot moved this to Triage in Tracking Aug 22, 2026
@pljones pljones moved this from Triage to Waiting on Team in Tracking Aug 22, 2026
Comment thread .github/workflows/autobuild.yml Outdated
@pljones
pljones force-pushed the cache-keys branch 4 times, most recently from fdae3db to 4b51804 Compare August 23, 2026 17:39

@softins softins left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks ok in general, just a couple of comments.

Comment thread .github/workflows/autobuild.yml Outdated
Comment thread .github/workflows/bump-dependencies.yml Outdated
Comment thread .github/workflows/bump-dependencies.yml Outdated
@pljones
pljones force-pushed the cache-keys branch 11 times, most recently from 73de281 to 3f7be01 Compare August 25, 2026 20:08
@pljones
pljones marked this pull request as draft August 25, 2026 20:11
@pljones

pljones commented Aug 25, 2026

Copy link
Copy Markdown
Collaborator Author

Well, it's not quite there, is it....

@ann0see
ann0see self-requested a review August 25, 2026 21:25
@mcfnord

mcfnord commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

🤖 AI: The four former param() defaults at deploy_windows.ps1 kept their trailing commas and deploy_mac.sh kept the ../.. that suits a script one level deeper, and two more sit behind that Windows parse error: the launch line still passes the two Qt paths positionally to a param() that no longer declares them, and $QtCompile64 is now msvc2022 while the aqt arch is still win64_${QtCompile64}_64.

@pljones
pljones force-pushed the cache-keys branch 3 times, most recently from 48e4892 to 081b45d Compare August 26, 2026 17:05
@pljones
pljones force-pushed the cache-keys branch 12 times, most recently from 54278a8 to d364a54 Compare August 28, 2026 17:29
@pljones

pljones commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator Author

Getting there -- the caches are being made and restored. Windows and macOS don't realise this - they go off and install again...

iOS gets it right.

Cache hit for: ios-0f7a19096b2d214a9d0ef45f70b89972b1f10eec3e4f6ae85a49f4dbcf9925d3-ios
Using Qt installation from previous run (actions/cache)

Bump dependencies is working, too.

@pljones
pljones force-pushed the cache-keys branch 3 times, most recently from d7b1d7b to c411a51 Compare August 29, 2026 15:31
@pljones

pljones commented Aug 29, 2026

Copy link
Copy Markdown
Collaborator Author

Right - all tests now passing:

  1. Cache keys created for iOS, macOS qt5, macOS qt6, Windows ASIO, Windows Jack
  2. Caches populated on successful completion for each key
  3. Caches restored on subsequent run on same branch (that's just how the cache works)
  4. Caches used by the matrix setup runner
  5. Bump Dependencies correctly spots version matches and mismatches
    • One caveat: as we currently have divergent macOS and Windows Qt6 versions, it raises one PR to bring them together
    • I've only checked the diff for the PR its created - pljones@812e9d7

@pljones
pljones marked this pull request as ready for review August 29, 2026 15:47
@@ -0,0 +1,6 @@
#!/bin/bash

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This then dropped the comment about pinning?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Eh?

Comment thread .github/autobuild/mac.sh
PROJECT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
readonly PROJECT_DIR
# shellcheck disable=SC1090
source "${PROJECT_DIR}/.github/autobuild/mac-dependencies_qt${QT}.sh"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have a guarantee that ${QT} is safe and correct? I'd add an error check if the file exists before.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's set from the autobuild workflow file and, if not, defaults to 6 at the top of the file. If someone sets the wrong value, this command will safely crash the workflow.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd still like a guard to reject invalid values.

# Important:
# - Do not update ASIO SDK without checking for license-related changes.
# - Do not copy (parts of) the ASIO SDK into the Jamulus source tree without
# further consideration as it would make the license situation more complicated.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably outdated as they moved to the GPL.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As it's version dependent (if the old version can still be found), I suppose it makes sense to keep it. If and when the old version vanishes, this can go, too.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The old version will probably not vanish. But we can of course keep this.

@ann0see

ann0see commented Aug 31, 2026

Copy link
Copy Markdown
Member

I think we might have an issue about some OS not being cached. Worth checking the issues for that and linking it here. If the issue is iOS related, we can close it.

@ann0see

ann0see commented Sep 1, 2026

Copy link
Copy Markdown
Member

The iOS caching issue was already documented. Thus:

Fixes: #2462

@pljones pljones added this to the Release 4.0.0 milestone Sep 1, 2026
Comment thread .github/autobuild/ios.sh
Comment on lines -52 to -53
# The following version pinnings are semi-automatically checked for
# updates. Verify .github/workflows/bump-dependencies.yaml when changing those manually:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pljones Maybe I am blind, but I think this comment got dropped?

@pljones pljones Sep 1, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, right. I should have put that on all the *-dependencies*.

@pljones pljones linked an issue Sep 1, 2026 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AI AI generated or potentially AI generated tooling Changes to the automated build system

Projects

Status: Waiting on Team

Development

Successfully merging this pull request may close these issues.

Add caching for iOS build

4 participants