Flexibility in building bottles & offering bottles from homebrew-core formulae

[rgoldberg]

Output of brew config

n/a

Output of brew doctor

n/a

Description of issue

The mas project should be able to build its executable for all the macOS versions & platforms that it supports in a single GitHub workflow on a single macOS GitHub runner.

e.g., Xcode 16.1 on a GitHub macos-15 runner should be able to build an arm64 executable that will run on macOS 11+ on Apple Silicon, and an x86_64 executable that will run on macOS 10.13+ on Intel.

Doing so would simplify distribution of mas to users (rendering unnecessary both the mas custom tap & building from source for all macOS versions 12.5+ for which Homebrew doesn't currently provide bottles), and would conserve Homebrew bottle building resources.

Additionally, older versions of mas support older versions of macOS that are not supported by the current or future versions of mas. For such legacy macOS versions, it would be useful for the current mas formula to be able to provide bottles of a legacy version of mas that supports them.

I assume that at least some other formulae would be in similar situations for either or both of the above.

The seemingly optimal setup for mas would be to allow its formula to have Homebrew:

1. Build only 2 bottles total for each newly released mas version, both on a the same runner (either macos-15 or macos-15-arm64):
   1. An arm64 bottle for big_sur+ by passing --arch & arm64 arguments to the mas build script
   2. An x86_64 bottle for high_sierra+ by passing --arch & x86_64 arguments to the mas build script
2. Allow the mas formula to provide legacy mas 1.8.6 x86_64 bottles that were previously bottled by Homebrew for sierra & el_capitan, as 1.8.6 is the last mas version to support those 2 macOS versions.

Is there any way to achieve the above or something similar within the homebrew-core formula? (preferably cleanly, but I’ll accept any solution that Homebrew maintainers will, even if it's kludgy)

If not, are there any open issues / draft PRs for any of this?

Would Homebrew accept a PR for at least part of this? (being able to build bottles on a newer macOS for older macOS versions is the most important part for me, followed by providing legacy bottles for legacy macOS versions, followed by building bottles for different platforms on one runner)

Thanks.

[SMillerDev]

Is there any way to achieve the above or something similar within the homebrew-core formula?

There is not because:

1. This will block Homebrew from making a build optimal to the platform it's building on and therefore is not desirable for homebrew. Also since we only support the macOS versions we have in CI.
2. Because it would make the package a giant mess when the version number isn't what's defined but instead whatever was last build for your OS.

[rgoldberg]

@SMillerDev Thanks for the info. A few questions:

1. This will block Homebrew from making a build optimal to the platform it's building on and therefore is not desirable for homebrew.

Sorry if I’m being obtuse, but I think that, for mas’s circumstances, the process that I’ve outlined will build a more optimal executable for all platforms than the current setup.

mas is a Swift Package Manager project using Swift 5.7.1. It can be built on newer macOS versions, then deployed to older macOS versions (in fact, it cannot be built at all on many of the macOS versions on which it can be deployed).

Please let me know if anything below seems wrong, or if you have info that I obviously don’t know:

There is nothing of which I’m aware that optimizes mas for the macOS version it’s built on. The only optimizations of which I’m aware are building with a newer version of Xcode to use a newer compiler to generate a more efficient executable. An executable built by a newer version of Xcode on macOS 15 should therefore run better on macOS 13 than an executable built by an older version of Xcode on macOS 13.

Xcode can produce binaries for either platform (or a universal). I haven’t heard anything about binaries compiled for a non-native platform being any less optimal. If that is the case, however, then my proposed workflow for mas could be revised to just building one bottle per platform, each on its own platform-specific runner, but only requiring one bottle per platform instead of building 3 for arm64 & 2 for x86_64, while also providing bottle support for older macOS versions with little (no?) change outside of allowing bottles to be renamed.

I assume that some other Swift projects are in similar circumstances. Some Java code is probably also similar, in that the newer runners probably have newer JDKs which can produce better bytecode than the older JDKs on older runner OSes (though Java builds are probably just run on the latest Ubuntu).

Also since we only support the macOS versions we have in CI.

IIUC, Homebrew core still provides existing bottles for legacy macOS versions where the formula hasn’t been updated after support was dropped for that macOS version. From a potentially naive perspective, it seems like if Homebrew allowed certain bottles to just have their name changed (e.g., from *sequoia* to *big_sur*), everything would work.

2. Because it would make the package a giant mess when the version number isn't what's defined but instead whatever was last build for your OS.

I know that Homebrew cask allows installing legacy versions of a cask on older macOS versions on which the current version of the cask cannot run. I haven’t looked for formulae that follow the same paradigm, but it seems reasonable to me.

Even if all the above is theroretically a good idea, obviously it might not be worth the implementation time, but it does seem like building bottles in newer macOS for older macOS wouldn’t be that hard (maybe I’m completely wrong about that).

Thanks again.

[SMillerDev]

Sorry if I’m being obtuse, but I think that, for mas’s circumstances, the process that I’ve outlined will build a more optimal executable for all platforms than the current setup.

That's fine, but Homebrew isn't created to build MAS, it's created to be a package manager for macOS.

There is nothing of which I’m aware that optimizes mas for the macOS version it’s built on. The only optimizations of which I’m aware are building with a newer version of Xcode to use a newer compiler to generate a more efficient executable. An executable built by a newer version of Xcode on macOS 15 should therefore run better on macOS 13 than an executable built by an older version of Xcode on macOS 13.

I'm not hugely knowledgable on compilers. But I can imagine a compiler can strip out a bunch of codepaths that are removed in macOS 15 if you tell it to build a macOS 15 executable, while those might have to stay in a macOS 13+ executable. Additionally, please refer to point 1.

Xcode can produce binaries for either platform (or a universal). I haven’t heard anything about binaries compiled for a non-native platform being any less optimal. If that is the case, however, then my proposed workflow for mas could be revised to just building one bottle per platform, each on its own platform-specific runner, but only requiring one bottle per platform instead of building 3 forarm64 & 2 for x86_64, while also providing bottle support for older macOS versions with little (no?) change outside of allowing bottles to be renamed.

The bottle name is just a symptom of a different build. If you build a bottle which uses the macOS system components in macOS 15, it'll still try to do that if the filename is different.

I assume that some other Swift projects are in similar circumstances. Some Java code is probably also similar, in that the newer runners probably have newer JDKs which can produce better bytecode than the older JDKs on older runner OSes (though Java builds are probably just run on the latest Ubuntu).

No, no other swift projects are trying to do this.

IIUC, Hokebrew core still provides existing bottles for legacy macOS versions where the formula hasn’t been updated after support was dropped for that macOS version. From a potentially naive perspective, it seems like if Homebrew allowed certain bottles to just have their name changed (e.g., from sequoia to big_sur), everything would work.

Just because something exists in the repo does not mean it'll be supported.

I know that Homebrew cask allows installing legacy versions of a cask on older macOS versions on which the current version of the cask cannot run. I haven’t looked for formulae that follow the same paradigm, but it seems reasonable to me.

Still not a supported configuration, but if upstream supports older macOS versions casks might allow the same thing (at the users own risk). For formulae this is impossible since they need to be build for every version, which requires CI we do not have.

Even if all the above is theroretically a good idea, obviously it might not be worth the implementation time, but it does seem like building bottles in newer macOS for older macOS wouldn’t be that hard (maybe I’m completely wrong about that).

It might not have been that hard, if it was ever a goal of Homebrew. Nothing in Homebrew is set up to support this, so at this point I expect it'll be pretty complicated.

[rgoldberg]

@SMillerDev Thanks for all the info.