About building multiple independent packages.

[akritkbehera]

Hi @pbuncic
If I use this command

bits/bitsBuild build --force-unknown-architecture -c cms.bits -a el9_amd64_gcc14 -w sw -j 16 --debug system-externals python_tools --write-store b3://bitsrep --remote-store b3://bits

which builds two packages system-externals and python_tools and stem-externals is a standalone package not needed by any other package.

What will be installed first?

[pbuncic]

You can add --dry-run option, it will tell you in which order packages will be built.

[akritkbehera]

So you can see I mentioned system-externals first but it's building gcc-prerequisites first. Is this the expected behavior? Both are independent packages.

Singularity> bits/bitsBuild build --force-unknown-architecture -c cms.bits -a el9_amd64_gcc14 -w sw -j 16 --dry-run system-externals gcc-prerequisites --write-store b3://bitsrep --remote-store b3://bits
/home/cmsbld/.local/lib/python3.9/site-packages/boto3/compat.py:89: PythonDeprecationWarning: Boto3 will no longer support Python 3.9 starting April 29, 2026. To continue receiving service updates, bug fixes, and security updates please upgrade to Python 3.10 or later. More information can be found here: https://aws.amazon.com/blogs/developer/python-support-policy-updates-for-aws-sdks-and-tools/
  warnings.warn(warning, PythonDeprecationWarning)

==> Always-on providers loaded:
      bits-providers        /data/akd/sw/REPOS/bits-providers/76ef2b3a10  (commit 76ef2b3a10)

==> Recipe origins: 3 package(s) from 1 source(s)

==> Configured directory:
    /data/akd/cms.bits

==> Package Recipe will be searched in the following order
    /data/akd/sw/REPOS/bits-providers/76ef2b3a10

==> Packages will be built in the following order:
     - gcc-prerequisites/1.0  [cms.bits@08235ae]
     - system-externals/1     [cms.bits@08235ae]

[pbuncic]

If two packages are truly independent when the order does not matter - why do you care that system-externals is build before gcc-prerequisites? If order matter, make system-externals build requirement of gcc-prerequisites.

[akritkbehera]

Yeah they don't matter. We have a requirement where need to have system-externals installed before building any other package is there a way to do it?

[pbuncic]

I think you should simply make all other packages depend on your system-externals, that is the cleanest way. I guess you are trying to avoid re-building the stack?

[akritkbehera]

Yeah system-externals is just a package that reads something from the system it doesn't provide anything. If there is no way we can always run bits/bitsBuild twice ig

[pbuncic]

Right, running bits twice is probably the best solution if there is no information that needs to be passed from one package to another so, there are no implicit or explicit dependencies.

[akritkbehera]

Yeah do you have CI setup already for building IBs that I can take a look at?

[pbuncic]

Have a look at https://gitlab.cern.ch/buncic/bits-console (bits-console.web.cern.ch). In the repo you will find a pipeline that drives bits build and publish to CVMFS. If you are interested, we can work together to wire CMS in bits-console.

[akritkbehera]

That would be nice to set it up I have the recipes on my repo.

[akritkbehera]

git@github.com:akritkbehera/cms.bits.git recipes are here on branch IB/CMSSW_20_1_X/gcc14