on_system conditionals not respected for `:all` manifest tab cause issues in keg relocation #18695

calvinit · 2024-11-01T07:26:39Z

`brew gist-logs <formula>` link OR `brew config` AND `brew doctor` output

# brew config

HOMEBREW_VERSION: 4.4.3-60-g745596b
ORIGIN: https://github.com/Homebrew/brew
HEAD: 745596b55579f3b9acb198f6aaa4c8aba0ef9844
Last commit: 5 hours ago
Core tap HEAD: bff87d7ce5eb665fdd9b28e1ff8ee510d8cf352b
Core tap last commit: 2 hours ago
Core tap branch: openjdk@8_1.8.0-432
Core tap JSON: 01 Nov 06:23 UTC
Core cask tap origin: https://github.com/Homebrew/homebrew-cask.git
Core cask tap HEAD: 649f802bb9d4ba0a703685b982d661b570173bee
Core cask tap last commit: 6 hours ago
Core cask tap JSON: 01 Nov 06:23 UTC
HOMEBREW_PREFIX: /usr/local
HOMEBREW_BOTTLE_DOMAIN: https://mirrors.tuna.tsinghua.edu.cn/homebrew-bottles
HOMEBREW_CASK_OPTS: []
HOMEBREW_EDITOR: code
HOMEBREW_MAKE_JOBS: 12
HOMEBREW_NO_ENV_HINTS: set
HOMEBREW_SORBET_RUNTIME: set
Homebrew Ruby: 3.3.5 => /usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/3.3.5/bin/ruby
CPU: dodeca-core 64-bit kabylake
Clang: 16.0.0 build 1600
Git: 2.47.0 => /usr/local/bin/git
Curl: 8.7.1 => /usr/bin/curl
macOS: 15.1-x86_64
CLT: 16.1.0.0.1.1729049160
Xcode: 16.1

brew doctor

Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry or file an issue; just ignore this. Thanks!

Warning: Some installed kegs have no formulae!
This means they were either deleted or installed manually.
You should find replacements for the following formulae:
  [email protected]

Warning: Some installed formulae are deprecated or disabled.
You should find replacements for the following formulae:
  [email protected]
  [email protected]
  [email protected]
  [email protected]
  [email protected]
  [email protected]

Verification

My brew doctor output says Your system is ready to brew. and am still able to reproduce my issue.
I ran brew update and am still able to reproduce my issue.
I have resolved all warnings from brew doctor and that did not fix my problem.
I searched for recent similar issues at https://github.com/Homebrew/homebrew-core/issues?q=is%3Aissue and found no duplicates.

What were you trying to do (and why)?

Submit a PR to upgrade openjdk@8 to version 1.8.0-432.

What happened (include all command output)?

brew test --retry --verbose embulk failed because it selected the incorrect version of openjdk@21 (it should actually select openjdk@8 on the Intel architecture):

on_arm do
  depends_on "openjdk@21"
end
on_intel do
  depends_on "openjdk@8"
end

/usr/local/Homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/bin/bundle clean
  ==> Testing embulk
  ==> Downloading https://search.maven.org/remotecontent?filepath=org/jruby/jruby-complete/9.4.8.0/jruby-complete-9.4.8.0.jar
  ==> Downloading from https://repo1.maven.org/maven2/org/jruby/jruby-complete/9.4.8.0/jruby-complete-9.4.8.0.jar
  /usr/bin/env /usr/local/Homebrew/Library/Homebrew/shims/shared/curl --disable --cookie /dev/null --globoff --show-error --user-agent Homebrew/4.4.3-60-g745596b\ \(Macintosh\;\ Intel\ Mac\ OS\ X\ 14.7\)\ curl/8.7.1 --header Accept-Language:\ en --fail --retry 3 --remote-time --output /Users/brew/Library/Caches/Homebrew/downloads/db8894972d158eba6af122d132a624d09cfd39c16a8db6b30173a70899e18307--jruby-complete-9.4.8.0.jar.incomplete --location https://repo1.maven.org/maven2/org/jruby/jruby-complete/9.4.8.0/jruby-complete-9.4.8.0.jar
    % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                   Dload  Upload   Total   Spent    Left  Speed
  
    0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  100 31.9M  100 31.9M    0     0   100M      0 --:--:-- --:--:-- --:--:--  100M
  ==> Verifying checksum for 'db8894972d158eba6af122d132a624d09cfd39c16a8db6b30173a70899e18307--jruby-complete-9.4.8.0.jar'
  cp -p /Users/brew/Library/Caches/Homebrew/downloads/db8894972d158eba6af122d132a624d09cfd39c16a8db6b30173a70899e18307--jruby-complete-9.4.8.0.jar /private/tmp/jruby-complete-20241101-7906-5s7uk4/jruby-complete-9.4.8.0.jar
  ==> /usr/local/Cellar/embulk/0.11.5/bin/embulk -X jruby=file:///private/tmp/embulk-test-20241101-7906-yw06o/jruby-complete-9.4.8.0.jar gem install embulk --version 0.11.5
  /usr/local/Cellar/embulk/0.11.5/bin/embulk: line 3: /usr/local/opt/openjdk@21/libexec/openjdk.jdk/Contents/Home/bin/java: No such file or directory
  /usr/local/Cellar/embulk/0.11.5/bin/embulk: line 3: exec: /usr/local/opt/openjdk@21/libexec/openjdk.jdk/Contents/Home/bin/java: cannot execute: No such file or directory
  ==> Testing embulk (again)
  cp -p /Users/brew/Library/Caches/Homebrew/downloads/db8894972d158eba6af122d132a624d09cfd39c16a8db6b30173a70899e18307--jruby-complete-9.4.8.0.jar /private/tmp/jruby-complete-20241101-8200-84vjdr/jruby-complete-9.4.8.0.jar
  ==> /usr/local/Cellar/embulk/0.11.5/bin/embulk -X jruby=file:///private/tmp/embulk-test-20241101-8200-lwwcgy/jruby-complete-9.4.8.0.jar gem install embulk --version 0.11.5
  /usr/local/Cellar/embulk/0.11.5/bin/embulk: line 3: /usr/local/opt/openjdk@21/libexec/openjdk.jdk/Contents/Home/bin/java: No such file or directory
  /usr/local/Cellar/embulk/0.11.5/bin/embulk: line 3: exec: /usr/local/opt/openjdk@21/libexec/openjdk.jdk/Contents/Home/bin/java: cannot execute: No such file or directory
  Error: embulk: failed
  An exception occurred within a child process:
    BuildError: Failed executing: /usr/local/Cellar/embulk/0.11.5/bin/embulk -X jruby=file:///private/tmp/embulk-test-20241101-8200-lwwcgy/jruby-complete-9.4.8.0.jar gem install embulk --version 0.11.5
  /usr/local/Homebrew/Library/Homebrew/formula.rb:3082:in `block in system'
  /usr/local/Homebrew/Library/Homebrew/formula.rb:3018:in `open'
  /usr/local/Homebrew/Library/Homebrew/formula.rb:3018:in `system'
  /usr/local/Homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/gems/sorbet-runtime-0.5.11630/lib/types/private/methods/call_validation.rb:270:in `bind_call'
  /usr/local/Homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/gems/sorbet-runtime-0.5.11630/lib/types/private/methods/call_validation.rb:270:in `validate_call'
  /usr/local/Homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/gems/sorbet-runtime-0.5.11630/lib/types/private/methods/_methods.rb:277:in `block in _on_method_added'
  /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/e/embulk.rb:65:in `block in <class:Embulk>'
  /usr/local/Homebrew/Library/Homebrew/formula.rb:2817:in `block (3 levels) in run_test'
  /usr/local/Homebrew/Library/Homebrew/extend/kernel.rb:539:in `with_env'
  /usr/local/Homebrew/Library/Homebrew/formula.rb:2816:in `block (2 levels) in run_test'
  /usr/local/Homebrew/Library/Homebrew/formula.rb:1199:in `with_logging'
  /usr/local/Homebrew/Library/Homebrew/formula.rb:2815:in `block in run_test'
  /usr/local/Homebrew/Library/Homebrew/mktemp.rb:90:in `block in run'
  /usr/local/Homebrew/Library/Homebrew/mktemp.rb:90:in `chdir'
  /usr/local/Homebrew/Library/Homebrew/mktemp.rb:90:in `run'
  /usr/local/Homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/gems/sorbet-runtime-0.5.11630/lib/types/private/methods/call_validation.rb:270:in `bind_call'
  /usr/local/Homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/gems/sorbet-runtime-0.5.11630/lib/types/private/methods/call_validation.rb:270:in `validate_call'
  /usr/local/Homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/gems/sorbet-runtime-0.5.11630/lib/types/private/methods/_methods.rb:277:in `block in _on_method_added'
  /usr/local/Homebrew/Library/Homebrew/formula.rb:3141:in `mktemp'
  /usr/local/Homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/gems/sorbet-runtime-0.5.11630/lib/types/private/methods/call_validation.rb:270:in `bind_call'
  /usr/local/Homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/gems/sorbet-runtime-0.5.11630/lib/types/private/methods/call_validation.rb:270:in `validate_call'
  /usr/local/Homebrew/Library/Homebrew/vendor/bundle/ruby/3.3.0/gems/sorbet-runtime-0.5.11630/lib/types/private/methods/_methods.rb:277:in `block in _on_method_added'
  /usr/local/Homebrew/Library/Homebrew/formula.rb:2809:in `run_test'
  /usr/local/Homebrew/Library/Homebrew/test.rb:48:in `block in <main>'
  /usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/3.3.5/lib/ruby/3.3.0/timeout.rb:1[86](https://github.com/Homebrew/homebrew-core/actions/runs/11624571402/job/32374214285?pr=194719#step:3:87):in `block in timeout'
  /usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/3.3.5/lib/ruby/3.3.0/timeout.rb:41:in `handle_timeout'
  /usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/3.3.5/lib/ruby/3.3.0/timeout.rb:195:in `timeout'
  /usr/local/Homebrew/Library/Homebrew/test.rb:54:in `<main>'

What did you expect to happen?

Select openjdk@8 correctly (@@HOMEBREW_JAVA@@).

Or

bottle do
  sha256 cellar: :..., arm: "xxxxxxxxxx"
  sha256 cellar: :..., x86_64: "yyyyyyyyyy"
end

Step-by-step reproduction instructions (by running `brew` commands)

==> brew fetch --retry embulk
==> brew install --only-dependencies embulk
==> brew install embulk
==> brew install --only-dependencies embulk
==> brew linkage --test embulk
==> brew linkage --cached --test --strict embulk
==> brew install --only-dependencies --include-test embulk
==> brew test --retry --verbose embulk

For more detailed logs, see here.

The text was updated successfully, but these errors were encountered:

calvinit · 2024-11-01T07:37:31Z

https://formulae.brew.sh/formula/embulk

It’s necessary to distinguish between different architectures here.

cho-m · 2024-11-01T17:53:38Z

In theory, it should not need multiple bottles as the Java path replacement is done at install time. The bottle should be using a placeholder, e.g. @@HOMEBREW_JAVA@@.

The issue appears to be combining dependencies into its variations isn't handled correctly for API.

When using a local tap, it seems okay, e.g.

❯ brew info embulk --json=v2 --variations | jq '.formulae[].dependencies'
[
  "openjdk@21"
]

❯ arch -x86_64 brew info embulk --json=v2 --variations | jq '.formulae[].dependencies'
[
  "openjdk@8"
]

❯ brew info embulk --json=v2 --variations | jq '.formulae[].variations.arm64_sequoia'
null

❯ brew info embulk --json=v2 --variations | jq '.formulae[].variations.sequoia'
{
  "dependencies": [
    "openjdk@8"
  ]
}

❯ arch -x86_64 brew info embulk --json=v2 --variations | jq '.formulae[].variations.arm64_sequoia'
{
  "dependencies": [
    "openjdk@21"
  ]
}

❯ arch -x86_64 brew info embulk --json=v2 --variations | jq '.formulae[].variations.sequoia'
null

However, the API data is generated based on single platform:

❯ curl -sL "https://formulae.brew.sh/api/formula/embulk.json" | jq '.dependencies'
[
  "openjdk@21"
]

carlocab · 2024-11-01T17:57:06Z

However, the API data is generated based on single platform:

That's a bug, but I wonder why it affects is in CI when we set HOMEBREW_NO_INSTALL_FROM_API.

cho-m · 2024-11-01T18:23:30Z

As workaround, can just use same OpenJDK on embulk. I split dependency based on upstream status, but probably doesn't matter if we just use newer one.

For issue, could also be a general keg relocation problem. Would need to check on that code. The bottle itself has:

export JAVA_HOME="${JAVA_HOME:-@@HOMEBREW_JAVA@@}"
exec "${JAVA_HOME}/bin/java"  -jar "@@HOMEBREW_CELLAR@@/embulk/0.11.5/libexec/embulk-0.11.5.jar" "$@"

So as long as bottle pour substitutes right path then it should have worked.

EDIT: Another possibility is if it is reading tab from manifest.

cho-m · 2024-11-01T18:44:19Z

It is the manifest rather than JSON API. Sadly no idea how to handle this for accurate tab in all bottle. May need something similar to Perl's preferred option.

Another idea is skipping manifest's tab if OS/arch doesn't match, though this may not work if we remove info based on #17395

Tab from https://github.com/Homebrew/homebrew-core/pkgs/container/core%2Fembulk/290843166

"sh.brew.tab": "{ ... {\"full_name\":\"openjdk@21\",\"version\":\"21.0.5\",\"revision\":0,\"pkg_version\":\"21.0.5\",\"declared_directly\":true}],

calvinit added the bug Reproducible Homebrew/brew bug label Nov 1, 2024

calvinit mentioned this issue Nov 1, 2024

openjdk@8 1.8.0-432 Homebrew/homebrew-core#194719

Merged

6 tasks

carlocab changed the title ~~Should a Formula that depends on different versions of openjdk for different architectures have multiple bottles?~~ on_system conditionals not respected during API generation Nov 1, 2024

carlocab transferred this issue from Homebrew/homebrew-core Nov 1, 2024

carlocab added the install from api Relates to API installs label Nov 1, 2024

cho-m removed the install from api Relates to API installs label Nov 1, 2024

cho-m changed the title ~~on_system conditionals not respected during API generation~~ on_system conditionals not respected for :all manifest tab cause issues in keg relocation Nov 1, 2024

carlocab added the help wanted We want help addressing this label Nov 2, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

on_system conditionals not respected for `:all` manifest tab cause issues in keg relocation #18695

on_system conditionals not respected for `:all` manifest tab cause issues in keg relocation #18695

calvinit commented Nov 1, 2024 •

edited

Loading

calvinit commented Nov 1, 2024

cho-m commented Nov 1, 2024

carlocab commented Nov 1, 2024

cho-m commented Nov 1, 2024 •

edited

Loading

cho-m commented Nov 1, 2024 •

edited

Loading

on_system conditionals not respected for :all manifest tab cause issues in keg relocation #18695

on_system conditionals not respected for :all manifest tab cause issues in keg relocation #18695

Comments

calvinit commented Nov 1, 2024 • edited Loading

brew gist-logs <formula> link OR brew config AND brew doctor output

brew doctor

Verification

What were you trying to do (and why)?

What happened (include all command output)?

What did you expect to happen?

Step-by-step reproduction instructions (by running brew commands)

calvinit commented Nov 1, 2024

cho-m commented Nov 1, 2024

carlocab commented Nov 1, 2024

cho-m commented Nov 1, 2024 • edited Loading

cho-m commented Nov 1, 2024 • edited Loading

on_system conditionals not respected for `:all` manifest tab cause issues in keg relocation #18695

on_system conditionals not respected for `:all` manifest tab cause issues in keg relocation #18695

calvinit commented Nov 1, 2024 •

edited

Loading

`brew gist-logs <formula>` link OR `brew config` AND `brew doctor` output

Step-by-step reproduction instructions (by running `brew` commands)

cho-m commented Nov 1, 2024 •

edited

Loading

cho-m commented Nov 1, 2024 •

edited

Loading