build.mill files compiled by Scala 3

[bishabosha]

This is work in progress to fix #3152

Numerous fixes were also needed to com-lihaoyi/mainargs, com-lihaoyi/sourcecode, and com-lihaoyi/mill-moduledefs

With the current state, only 1 example/integration tests is still failing:

• integration.feature[plugin-classpath].local

known TODOs:

• Discover macro
• Applicative macro
• Caller macro
• Cross.Factory macro
• EnclosingClass macro
• Task macros
• Cacher macro
• Moduledefs compiler plugin (override inferrence)
• All core Mill modules compile with Scala 3.5.0
• Fix Zinc reporter patch linenumbers of build scripts
• Check that bytecode analyzers work with Scala 3
• cleanup library dependency conflicts
• Support new Scala 3 syntax in build.sc files
• Fix BSP reporter linenumbers for build scripts (Zinc reporter forwards to bsp)
• Cleanup compiler warnings for outdated syntax

known incompatibilities:

• can't use ExplicitResultTypes scalafix rule - need to upgrade mill-scalafix
• Cross.scala uses the new quoted type syntax which scalafmt crashes on, (and version is frozen) so skip the file
• skipping Mima currently due to 1000s of errors (perhaps we should generate filters?)
• filtered one flaky test from example.thirdparty[mockito]
• filtered out integration.feature[plugin-classpath] due to third party plugin dep

[lihaoyi]

Thanks @bishabosha ! Lets get your mainargs changes landed, that way we can cut a release and you can test your WIP in CI

[lefou]

+1 for switching over to using directives!

• Support an alternative to $-imports #3062

[bishabosha]

@lihaoyi see com-lihaoyi/mainargs#148

[bishabosha]

[Edit: after switching to depending on mainargs 0.7.2] in https://github.com/com-lihaoyi/mill/actions/runs/10418081276/job/28853542122?pr=3369 you can see integration tests like example/tasks/6-workers/local passing

[bishabosha]

Rebased and included the new mill-moduledefs 0.11.0-M1

[bishabosha]

@lihaoyi just recording here that the mockito third party test can be flaky (specifically StubbingLookupListenerCallbackTest.add_listeners_concurrently_sanity_check):

https://github.com/mockito/mockito/blob/1e337ed703ce2b78ce9ea40ff8b8c645f8219cf2/src/test/java/org/mockitousage/debugging/StubbingLookupListenerCallbackTest.java#L199-L202

as seen in the CI run https://github.com/com-lihaoyi/mill/actions/runs/10909315276/job/30277476457?pr=3369#step:9:4941

[lihaoyi]

@bishabosha got it. Could you open a discussion thread on https://github.com/scala/scala3 with your findings so far? If it turns out to be impossible to implement line-number/position fixing 100% correctly with Scala 3, then we don't have much of a choice, but we should make sure we have a discussion about it so that the upstream folks have a chance to add support for this use case in future

[lihaoyi]

Seems like Ammonite does the bytecode postprocessing approach and scalacli does as well https://github.com/com-lihaoyi/Ammonite/blob/main/amm/compiler/src/main/scala-3/ammonite/compiler/AsmPositionUpdater.scala

[lefou]

If we can't access the full path from ASM, we could make the generated filename more special, e.g. by always appending a MillGenerated, then we can detect and handle it also from ASM.

[bishabosha]

So an ASM rewrite could be done the same-time/after methodCodeHashSignatures?

this would still mean semanticdb, tasty will be wrong.

plugin could still handle rewriting sourcecode.{Line, FileName}.

Edit: Right, I guess you can propose a compiler plugin that does the correct position rewrite, so that all those things are correct, and then an Additional ASM post processor that has to correct the corrected-but-still-wrong class line numbers.
At which point perhaps line-numbers plugin with two phases would work

[bishabosha]

My other solution: patch ASAP in compiler in backend the ability to trust positions in the code - I looked in the Scala 2 implementation and it accounts for "underlying" sourcefiles

Edit: I quickly prototyped this fix to dotty in scala/scala3#21763, and used it in my linenumbers mill branch and it works - meaning that the linenumber plugin can be very simple - and no double remapping in the zinc reporter (still need one remapping for typing errors)

Edit 2: added a test with a similar plugin design

[bishabosha]

so the compiler backend fix was merged