First pass at an Mill vs SBT case study (#3421)

Using the gatling codebase. All compiles and passes tests, and ran some rough benchmarks. There's definitely a lot of improvements we can make tightening up the case study, but its a start --------- Co-authored-by: Tobias Roeser <[email protected]>
com-lihaoyi · Aug 26, 2024 · 90ec87d · 90ec87d
1 parent a34932e
commit 90ec87d
Show file tree

Hide file tree

Showing 24 changed files with 3,015 additions and 8 deletions.
diff --git a/.github/workflows/actions.yml b/.github/workflows/actions.yml
@@ -80,7 +80,7 @@ jobs:
           - java-version: 11
             millargs: "'example.thirdparty[{mockito,acyclic,commons-io}].local.test'"
           - java-version: 17
-            millargs: "'example.thirdparty[{fansi,jimfs,netty}].local.test'"
+            millargs: "'example.thirdparty[{fansi,jimfs,netty,gatling}].local.test'"
           - java-version: 11
             millargs: "'example.depth.__.local.test'"
           - java-version: 17

diff --git a/build.sc b/build.sc
@@ -1359,7 +1359,8 @@ object example extends Module {
     "jimfs" -> ("google/jimfs", "5b60a42eb9d3cd7a2073d549bd0cb833f5a7e7e9"),
     "commons-io" -> ("apache/commons-io", "b91a48074231ef813bc9b91a815d77f6343ff8f0"),
     "netty" -> ("netty/netty", "20a790ed362a3c11e0e990b58598e4ac6aa88bef"),
-    "mockito" -> ("mockito/mockito", "97f3574cc07fdf36f1f76ba7332ac57675e140b1")
+    "mockito" -> ("mockito/mockito", "97f3574cc07fdf36f1f76ba7332ac57675e140b1"),
+    "gatling" -> ("gatling/gatling", "3870fda86e6bca005fbd53108c60a65db36279b6")
   )
   object thirdparty extends Cross[ThirdPartyModule](listIn(millSourcePath / "thirdparty"))
   trait ThirdPartyModule extends ExampleCrossModule {

diff --git a/docs/modules/ROOT/images/GatlingCompileGraph.svg b/docs/modules/ROOT/images/GatlingCompileGraph.svg
diff --git a/docs/modules/ROOT/images/GatlingCompileProfile.png b/docs/modules/ROOT/images/GatlingCompileProfile.png
diff --git a/docs/modules/ROOT/images/IntellijGatlingMillPlugin1.png b/docs/modules/ROOT/images/IntellijGatlingMillPlugin1.png
diff --git a/docs/modules/ROOT/images/IntellijGatlingMillPlugin2.png b/docs/modules/ROOT/images/IntellijGatlingMillPlugin2.png
diff --git a/docs/modules/ROOT/images/IntellijGatlingMillTask1.png b/docs/modules/ROOT/images/IntellijGatlingMillTask1.png
diff --git a/docs/modules/ROOT/images/IntellijGatlingMillTask2.png b/docs/modules/ROOT/images/IntellijGatlingMillTask2.png
diff --git a/docs/modules/ROOT/images/IntellijGatlingMillTask3.png b/docs/modules/ROOT/images/IntellijGatlingMillTask3.png
diff --git a/docs/modules/ROOT/images/IntellijGatlingSbtPlugin1.png b/docs/modules/ROOT/images/IntellijGatlingSbtPlugin1.png
diff --git a/docs/modules/ROOT/images/IntellijGatlingSbtPlugin2.png b/docs/modules/ROOT/images/IntellijGatlingSbtPlugin2.png
diff --git a/docs/modules/ROOT/images/IntellijGatlingSbtTask1.png b/docs/modules/ROOT/images/IntellijGatlingSbtTask1.png
diff --git a/docs/modules/ROOT/images/IntellijGatlingSbtTask2.png b/docs/modules/ROOT/images/IntellijGatlingSbtTask2.png
diff --git a/docs/modules/ROOT/images/MockitoCompileGraph.svg b/docs/modules/ROOT/images/MockitoCompileGraph.svg
diff --git a/docs/modules/ROOT/images/MockitoCompileProfile.png b/docs/modules/ROOT/images/MockitoCompileProfile.png
diff --git a/docs/modules/ROOT/images/NettyCompileGraph.svg b/docs/modules/ROOT/images/NettyCompileGraph.svg
diff --git a/docs/modules/ROOT/images/NettyCompileProfile.png b/docs/modules/ROOT/images/NettyCompileProfile.png
diff --git a/docs/modules/ROOT/nav.adoc b/docs/modules/ROOT/nav.adoc
@@ -22,6 +22,7 @@
 * xref:Testing_Scala_Projects.adoc[]
 // * xref:Publishing_Scala_Projects.adoc[]
 * xref:Scala_Web_Examples.adoc[]
+* xref:Case_Study_Mill_vs_SBT.adoc[]
 
 // This section is all about developing a deeper understanding of specific
 // topics in Mill. This is the opposite of `Quick Start` above: while we touch

diff --git a/docs/modules/ROOT/pages/Case_Study_Mill_vs_Gradle.adoc b/docs/modules/ROOT/pages/Case_Study_Mill_vs_Gradle.adoc
@@ -181,6 +181,43 @@ For both Mill and Gradle, we see small speedups relative to the <<Incremental Co
 benchmark above, which likely comes from not having to compile any Java source files at all. Mill
 remains faster than Gradle by about 2.0x.
 
+== Debugging Tooling
+
+Another area that Mill does better than Gradle is providing builtin tools for you to understand
+what your build is doing. For example, the Mockito project build discussed has 21 submodules
+and associated test suites, but how do these different modules depend on each other? With
+Mill, you can run `./mill show visualize __.compile`, and it will show you how the
+`compile` task of each module depends on the others:
+
+image::MockitoCompileGraph.svg[]
+
+Apart from the static dependency graph, another thing of interest may be the performance
+profile and timeline: where the time is spent when you actually compile everything. With
+Mill, when you run a compilation using `./mill -j 10 __.compile`, you automatically get a
+`out/mill-chrome-profile.json` file that you can load into your `chrome://tracing` page and
+visualize where your build is spending time and where the performance bottlenecks are:
+
+image::MockitoCompileProfile.png[]
+
+If you want to inspect the tree of third-party dependencies used by any module, the
+built in `ivyDepsTree` command lets you do that easily:
+
+```bash
+$ ./mill subprojects.junit-jupiter.ivyDepsTree
+├─ org.junit.jupiter:junit-jupiter-api:5.10.3
+│  ├─ org.apiguardian:apiguardian-api:1.1.2
+│  ├─ org.junit.platform:junit-platform-commons:1.10.3
+│  │  └─ org.apiguardian:apiguardian-api:1.1.2
+│  └─ org.opentest4j:opentest4j:1.3.0
+└─ org.objenesis:objenesis:3.3
+```
+
+None of these tools are rocket science, but Mill provides all of them out of the
+box in a convenient package for you to use. Whether you want a visual graph layout,
+a parallel performance profile, or a third-party dependency tree of your project,
+Mill makes it easy and convenient without needing to fiddle with custom configuration
+or third party plugins. This helps make it easy for you to explore, understand, and
+take ownership of the build tool.
 
 == Conclusion
 
@@ -197,4 +234,6 @@ Again, the Mill build used in this comparison is for demonstration purposes, and
 work would be necessary to make the Mill build production ready: publishing configuration,
 code coverage integration, and so on. However, hopefully it demonstrates the potential value:
 significantly improved performance, so that you spend less time waiting for your code to
-compile and more time doing the work that actually matters.
+compile and more time doing the work that actually matters, with builtin debugging tools
+to help turn normally opaque "build config" into something that's transparent and
+easily understandable.
diff --git a/docs/modules/ROOT/pages/Case_Study_Mill_vs_Maven.adoc b/docs/modules/ROOT/pages/Case_Study_Mill_vs_Maven.adoc
@@ -762,6 +762,46 @@ much more concise or verbose than the other. Rather, what is interesting is that
 it is much easier to work with this kind of _build logic_ via _concise type-checked code_,
 rather than configuring a bunch of third-party plugins to try and achieve what you want.
 
+
+== Debugging Tooling
+
+Another area that Mill does better than Maven is providing builtin tools for you to understand
+what your build is doing. For example, the Mockito project build discussed has 21 submodules
+and associated test suites, but how do these different modules depend on each other? With
+Mill, you can run `./mill show visualize __.compile`, and it will show you how the
+`compile` task of each module depends on the others:
+
+image::NettyCompileGraph.svg[]
+
+Apart from the static dependency graph, another thing of interest may be the performance
+profile and timeline: where the time is spent when you actually compile everything. With
+Mill, when you run a compilation using `./mill -j 10 __.compile`, you automatically get a
+`out/mill-chrome-profile.json` file that you can load into your `chrome://tracing` page and
+visualize where your build is spending time and where the performance bottlenecks are:
+
+image::NettyCompileProfile.png[]
+
+If you want to inspect the tree of third-party dependencies used by any module, the
+built in `ivyDepsTree` command lets you do that easily:
+
+```bash
+$ ./mill handler.ivyDepsTree
+├─ org.jctools:jctools-core:4.0.5
+├─ org.junit.jupiter:junit-jupiter-api:5.9.0
+│  ├─ org.apiguardian:apiguardian-api:1.1.2
+│  ├─ org.junit.platform:junit-platform-commons:1.9.0
+│  │  └─ org.apiguardian:apiguardian-api:1.1.2
+│  └─ org.opentest4j:opentest4j:1.2.0
+└─ com.google.protobuf:protobuf-java:2.6.1
+```
+
+None of these tools are rocket science, but Mill provides all of them out of the
+box in a convenient package for you to use. Whether you want a visual graph layout,
+a parallel performance profile, or a third-party dependency tree of your project,
+Mill makes it easy and convenient without needing to fiddle with custom configuration
+or third party plugins. This helps make it easy for you to explore, understand, and
+take ownership of the build tool.
+
 == Conclusion
 
 Both the Mill and Maven builds we discussed in this case study do the same thing: they
@@ -777,4 +817,7 @@ Again, the Mill build used in this comparison is for demonstration purposes, and
 work would be necessary to make the Mill build production ready: compatibility with
 different operating system architectures, Java versions, and so on. However, hopefully
 it demonstrates the potential value: improved performance, conciseness of the build logic,
-and easy extensibility so you can fine-tune your build logic to your requirements.
+and easy extensibility so you can fine-tune your build logic to your requirements.
+Mill provides builtin tools to help you navigate,
+visualize, and understand your build, turning a normally opaque "build config" into
+something that's transparent and easily understandable.