Using directives syntax

[keynmol]

Inevitable bikeshedding is about to happen :) I think the directives syntax not being valid Scala code is the most controversial aspect surrounding this amazing tool, so I was hoping we can discuss it.

This discussion is foreshadowed and linked to from the https://scala-cli.virtuslab.org/docs/guides/using-directives#most-important-using-directives-supported-by-scala-cli page - but the discussion link just doesn't exist.

Currently, there are 3 documented ways of using directives:

• using ...
• //using ...
• @using ...

The only one that immediately works with any tooling (scalameta, Metals, regular Scala compiler CLI, what have you) is the // using one.

The documentation states that

We believe that syntax similar to using directives should become a part of Scala in the future

But I foresee the following issues with this:

1. using is a soft keyword in Scala 3, so if it's not part of the comment, it will confuse both the users and the highlighters
2. if directives are indeed supported by compiler, they will have to be backported into Scala 2 parser, and it would also mean that one cannot use scala 2 versions before the parser was changed
3. Scalameta will have to be updated for tooling (scalafmt, for example) to work properly
4. what will be the semantics of using directives when used with naked compiler (not scala-cli)? Is the expectation to just not expose the naked scala REPL anymore?
5. What should the tools like mdoc do with directives, given that they interact directly with the compiler? Will the compiler interface handle artifact resolution and classpath?

This is a long lead up to the main question:

Why shouldn't using directives stay comments or valid Scala syntax like (import $ivy....) if it enables existing tooling to work seamlessly?

Update: to set the tone a little bit, I understand the experimental nature of directives and as such want to offer feedback and discussion, not treat it as a done deal I disagree with :)

[bjornregnell]

Can/could I point to another of my scala programs, like //using script /path/to/my/code.scala

or is ther some other mechanism that I havent found in the doc pages on how to include script code in script code?

[romanowski]

https://scala-cli.virtuslab.org/docs/guides/scripts#difference-with-ammonite-scripts

Basically, you run your sctipts together and they see each other without imports.

[bjornregnell]

Aha. So just use code from one file in another file? And dependent files will be compiled together? That's rellay cool!!

[bjornregnell]

I think it would be nice to add an example after "[...] passing the directory to scala-cli" of how scripts can use each other.

[bjornregnell]

BTW: I like very much the idea of having all config in the code and no separate config files!
//using is fine with me -- not that fond of using or @using if it breaks code in other tools, so if those should stay I'd like them to become valid Scala code as seen by the standard Scala 2 & 3 compilers.

[przemek-pokrywka]

@keynmol makes a great point. If the developer experience is to be flawless, we need to have something that will work with all IDE, editors, and different Scala versions.

Why shouldn't using directives stay comments or valid Scala syntax like (import $ivy....) if it enables existing tooling to work seamlessly?

in my opinion, they very much should stay valid Scala syntax, excellent tooling support is a requirement if scala-cli aims to lower the barrier to entry for the newcomers, reduce development tooling friction and bring us people from different ecosystems.

For me, the design choices for directives form the following hierarchy:

1. valid Scala syntax other than comment - it's the best because the compiler can check if the syntax is valid
2. comment - worse because of lack of compiler support, but at least compiler and tooling don't break
3. everything else - for all the reasons listed in a brilliant way by @keynmol

Now, a gotcha! The Ammonite magic imports are (unfortunately!) NOT a valid Scala

---

Proposal

Let's use the old, boring annotations, which would do quite a good job in this context, actually. One would be them on either:

• the main method of the script
• the class which extends the App
• nothing, just straight in the code for scripts that miss the main method (this kind of script is invalid Scala anyway)

Example (for Scala 3):

@using(@lib("com.lihaoyi::upickle:1.4.0"))
@main
def hello(name: String) = println(s"Hello ${name}!")

Example (for Scala 2, App):

@using(@lib("com.lihaoyi::upickle:1.4.0"))
object Hello extends App {
  println("Hello!")
}

Example (for a main-less "invalid Scala" script):

@using(@lib("com.lihaoyi::upickle:1.4.0"))
println("Hello!")

Q1 - How could the compiler and IDEs not break on these annotations?!

The library with the annotations could place them in the scala.lang (or any other automatically imported) package. The scala-cli would be automatically adding the annotations library to the compile-time dependencies. It would also communicate them to the IDEs/editors, which use BSP. Hey, you would be even able to jump to the annotation definitions!

Q2 - How do you ensure, that the annotations cross-compile to all versions of Scala?

We'd write them in Java.

Q3 - if we are going to use a language construct, why not macros?

Although it would be compelling to be able to write:

runWith(lib("com.lihaoyi::upickle:1.4.0")) {
  println("Hello!")
}

the system would become overly complicated (cross-version support issues, etc.) in my opinion.

---

Conclusion

Developer experience would be best when:

• all of the tooling (IDEs, editors, Scala compilers for different language versions) works out of the box
• nothing needs to be configured outside of the script:
  • fantastic shareability, paste a file on a different computer and you are good to go
  • possibly keep many unrelated scripts in a single directory, each using distinct dependencies!
  • no need to switch between files for short tasks
  • truly self-contained examples that can be used in documentation
• the compiler highlights syntax errors for incorrect dependency declarations

All of the above would be provided by scala-cli switching to annotations for directives.

[romanowski]

The @using syntax is de facto variation of an annotation one. The main problem with annotations is that the last thing we want to do is to mix the configuration and the code. We've played with many options and there we were not able to find any Scala construct that can be placed on top of any other Scala code that is not cryptic.

[bjornregnell]

Yeah. I think I prefer // using for the simple use case. The @-stuff tend to look cryptic. I guess it's better be transparent with all tools and runtimes (JVM, JS, Native, Android).

[kubukoz]

Personally I found the original configuration syntax (HOCON) to work best.

Example:

scala = {
  version = "2.13.6"
  options = [
    "-encoding",
    "utf8",
    "-explaintypes",
    "-feature",
    "-language:existentials",
    "-language:experimental.macros",
    "-language:higherKinds",
    "-language:implicitConversions",
    "-unchecked",
    "-Xcheckinit",
    "-Xlint:adapted-args",
    "-Xlint:constant",
    "-Xlint:delayedinit-select",
    "-Xlint:deprecation",
    "-Xlint:doc-detached",
    "-Xlint:inaccessible",
    "-Xlint:infer-any",
    "-Xlint:missing-interpolator",
    "-Xlint:nullary-unit",
    "-Xlint:option-implicit",
    "-Xlint:package-object-classes",
    "-Xlint:poly-implicit-overload",
    "-Xlint:private-shadow",
    "-Xlint:stars-align",
    "-Xlint:strict-unsealed-patmat",
    "-Xlint:type-parameter-shadow",
    "-Xlint:-byname-implicit",
    "-Wunused:nowarn",
    "-Wdead-code",
    "-Wextra-implicit",
    "-Wnumeric-widen",
    "-Wunused:implicits",
    "-Wunused:explicits",
    "-Wunused:imports",
    "-Wunused:locals",
    "-Wunused:params",
    "-Wunused:patvars",
    "-Wunused:privates",
    "-Wvalue-discard",
    "-Vimplicits",
    "-Vtype-diffs",
    "-Xsource:3.0"
  ]
}

extra-jars = [
  ./src/main/resources
]

dependencies = [
  "io.circe::circe-generic:0.14.1",
  "io.circe::circe-parser:0.14.1",
  "io.circe::circe-literal:0.14.1",
  "org.typelevel::cats-effect:3.2.2",
  "org.http4s::http4s-ember-client:0.23.0-RC1",
  "org.http4s::http4s-circe:0.23.0-RC1",
  "ch.qos.logback:logback-classic:1.2.3",
  "org.systemfw::dynosaur-core:0.3.0",
  "com.github.cb372::cats-retry:3.0.0",
  "io.chrisdavenport::cormorant-fs2:0.5.0-M1",
  "io.chrisdavenport::cormorant-generic:0.5.0-M1",
  "io.github.timwspence::cats-stm:0.11.0"
]

HOCON:

• doesn't pretend to be Scala and wouldn't require changes to the language for good tooling support. Doesn't use macros, annotations or any other feature that is bound to confuse new Scala users if they have to deal with it on day 1.
• uses syntax that's easy to learn and is well known in the Scala ecosystem already, with a variety of libraries that can read & decode it (lightbend config, pureconfig, ficus...) which would make it easy to build tooling around (I know that directives also have a library for reading them, but there would be no need for anything new like that if we stuck to the well known)
• allows grouping things like dependencies into collapsible/expandable code blocks which work by default in Metals and IntelliJ. AFAIK any kind of grouping of directives in the current shape can only be done by splitting a config file into many, which is less than ideal.
• had comments that actually behaved like comments (you could disable a line/block with a shortcut or whatever) - unless editors are aware of the directive syntax and it works without comments, I don't see that happening with directives.

--

Some general thoughts, removed of HOCON...

• Directives leak. They give the impression of being local to the file they're used in, but are actually build-wide - if you have two files and set a flag in one of them, it'll be used by the second file as well. If they're conflicting you won't even know which one is picked.

I understand that one of the goals of scala-cli is to support prototyping with a single Scala file, but is it really worth the cost of increased complexity for anything larger than that? We could support the simple case while maintaining a nice experience beyond that:

1. If you only need dependencies (which is a fair share of single-file snippets I've seen) the existing Ammonite syntax (import $ivy.) would do the trick.
2. If you need compiler options you could pass them in the command line.
3. For anything larger than one file you would use a config file.

We already have the duality of command line options and directives/$ivy syntax, so this doesn't introduce any more confusion.

I know scala-cli https://gist.github.com/foo/bar.scala makes for a cool demo and concise snippets, but it's just the "7 character hello world" effect - it's impressive the first time, but making it the primary usecase seems limiting.

As much as I see a point of having one-file scripts, I think it would be a waste of a great tool if you optimize for that one specific purpose. scala-cli could very well be the standard build tool for microlibraries and command-line applications in Scala, and I feel like that space is being ignored with the current approach to configuration.

Finally, supporting directives natively in Scala - I would love to know how you'd like to do that and keep support for existing versions which can't parse that. Say, 2.13.6 or 3.1.0.

[przemek-pokrywka]

To me, the value of being able to have a self-contained script is hard to overstate. Reducing it to "cool demo" and "7 characters hello world" effects would be a gross underappreciation of its wide applicability.

Probably the biggest benefit of being able to keep a script self-contained is in teaching. StackOverflow examples, library documentation, language construct explanations, all frequently suffer the "paper will take anything" syndrome. I can't count the number of times I've seen people complaining that an example is missing important import clauses, that it only works with a particular (undocumented) version of a given library or language. With the ability to keep the critical information within the example, this all can be suddenly fixed.

Also, when a novice reaches for something new, the amount of friction frequently decides whether one picks one or another technology. There is a lot of JavaScript developers simply because to code in it one doesn't need any tools other than what they already have - the browser. A lot of people use Bash not because they love it, but because it is so widespread on every kind of *-nix system.
One can argue that having to add one more file is no deal-breaker, but in many contexts, it actually will be one! If a piece of code that lets one serve files over HTTP is two files in Scala and a single line in Python, people will inevitably pick the latter.

[przemek-pokrywka]

Fortunately, directives embedded in the script's comments are not mutually exclusive with anything in separate files. The files can simply override whatever is declared within the script. Everybody's happy 😃

[bjornregnell]

I agree, esp. regarding the benefits of teaching with a single-file runable thing! This is really important and I would love to have this for my 200+ students that I teach every year using Scala first.

[kubukoz]

To me, the value of being able to have a self-contained script is hard to overstate. Reducing it to "cool demo" and "7 characters hello world" effects would be a gross underappreciation of its wide applicability.

Sure, it's useful. But scala-cli isn't (only) a tool for sharing runnable snippets, and I think that shouldn't be the only usecase that gets special treatment of this sort. Single-module apps (another main usecase) would benefit from having configuration in a file specifically meant for that.

I understand your point about giving a good first impression. Perhaps that's really as valuable and I didn't consider it that much :)

Fortunately, directives embedded in the script's comments are not mutually exclusive with anything in separate files. The files can simply override whatever is declared within the script. Everybody's happy 😃

That would probably be best.

[przemek-pokrywka]

It is natural, that people caring about certain use cases won't always be concerned with other people's use cases. Furthermore, people coming with different use cases in mind will sometimes have conflicting preferences. Therefore it's good that in this case there exists a design choice which allows for the different styles to coexist peacefully, bringing value to respective users.

Regarding the first impressions, I do find them important, but they are just a tip of an iceberg... The low amount of friction is one of the most impactful factors of technology popularity.

[bjornregnell]

Thanks for food for thought @keynmol . I'll think about this some more. Spontaneous reaction:

• We could have // using ... or import $ivy... or @using(...)for the really simple use case (But I like the first option // using as it will not collide with any code/tool and is simpler / less cryptic to a beginner programmer. We need to get actual beginners onboard like pre-university kids and university students for a long term growth and to really compete with python... So simple is indeed a virtue IMHO
• And also we could offer HCON config for the more elaborate use case
  That makes sense to me, but needs more thinking.

[bjornregnell]

I guess it is some work implementing HCON config for the elaborate use case and it needs to work across runtimes.

[przemek-pokrywka]

It seems to me, that we are converging onto a view, that it is best to have:

• directives within comments - for self-contained, simple use
• separate HOCON file - for more elaborate uses

And the language-level construct (both using soft keyword and the annotations) are either ecosystem unfriendly (Scala compiler versions, tooling) or cryptic and difficult to get right.

I'd like to point out that HOCON and comment-embedded directives are not mutually exclusive, because one can easily have an overriding scheme, which is very frequently found in many systems:

• command-line switches take the highest precedence
• environment variables are the second-most important
• configuration files are third
• values embedded within a program get used as a last resort

[kubukoz]

yes, yes, it's all coming together 😂

[romanowski]

Generally, full HOCON is a big no for us since it contains variable substitution, includes etc. All too complicated, we want just a configuration without any magic :)

Why do we need a HOCON when we can have e.g. config.scala file that contains only using directives?

It is actually a bug in current implementation that single using directive cannot be multi-line. To @kubukoz 's config will be placed in conf.scala (or even scala.conf), and it will look like below. For me, this is much more reliable than HOCON.

using scala 2.13.6

using options
    "-encoding"
    "utf8"
    "-explaintypes"
    "-feature"
    "-language:existentials"
    "-language:experimental.macros"
    "-language:higherKinds"
    "-language:implicitConversions"
    "-unchecked"
    "-Xcheckinit"
    "-Xlint:adapted-args"
    "-Xlint:constant"
    "-Xlint:delayedinit-select"
    "-Xlint:deprecation"
    "-Xlint:doc-detached"
    "-Xlint:inaccessible"
    "-Xlint:infer-any"
    "-Xlint:missing-interpolator"
    "-Xlint:nullary-unit"
    "-Xlint:option-implicit"
    "-Xlint:package-object-classes"
    "-Xlint:poly-implicit-overload"
    "-Xlint:private-shadow"
    "-Xlint:stars-align"
    "-Xlint:strict-unsealed-patmat"
    "-Xlint:type-parameter-shadow"
    "-Xlint:-byname-implicit"
    "-Wunused:nowarn"
    "-Wdead-code"
    "-Wextra-implicit"
    "-Wnumeric-widen"
    "-Wunused:implicits"
    "-Wunused:explicits"
    "-Wunused:imports"
    "-Wunused:locals"
    "-Wunused:params"
    "-Wunused:patvars"
    "-Wunused:privates"
    "-Wvalue-discard"
    "-Vimplicits"
    "-Vtype-diffs"
    "-Xsource:3.0"

using resources "./src/main/resources"

using libs
  "io.circe::circe-generic:0.14.1"
  "io.circe::circe-parser:0.14.1"
  "io.circe::circe-literal:0.14.1"
  "org.typelevel::cats-effect:3.2.2"
  "org.http4s::http4s-ember-client:0.23.0-RC1"
  "org.http4s::http4s-circe:0.23.0-RC1"
  "ch.qos.logback:logback-classic:1.2.3"
  "org.systemfw::dynosaur-core:0.3.0"
  "com.github.cb372::cats-retry:3.0.0"
  "io.chrisdavenport::cormorant-fs2:0.5.0-M1"
  "io.chrisdavenport::cormorant-generic:0.5.0-M1"
  "io.github.timwspence::cats-stm:0.11.0"

[przemek-pokrywka]

I can't decide if I like the simple-but-limited configuration or HOCON. The first option's pros are:

• uniformity between comment-embedded directive syntax and the external file
• arguable benefit of punishing people trying to go overboard with the amount of configuration - maybe the inconvenience should be gently pushing them into the proper build tools? If people had all variable substitutions and includes, they might want to stay with scala-cli even for bigger projects
• much clearer syntax
• somewhat less to learn

HOCON's strengths are:

• power, because with variable substitution one can extract some duplication (in group names, versions for example) - but it can be a drawback as well, see above
• appeal to experienced developers, who already know it

I certainly would have the config in a file not called *.scala, because the older Scala compilers would definitely break on it, along with most of the tooling.

[kubukoz]

Why do we need a HOCON when we can have e.g. config.scala file that contains only using directives?

If we can have config.scala that is proper valid Scala and uses comments for actual dead code, I'm fine with that :)

Directives feel like a different language. The syntax is only useful for this one tool and things that will integrate with it. Why should that be part of the language?

Directives outside of comments don't even parse (so far). And comments... the whole purpose of comments is to have plain text that doesn't impact the build (things that are descriptions, or just disabled code), isn't it? That seems like the opposite of what directives are supposed to do.

The question I'd ask is:

Why do we need the configuration to be Scala?

[przemek-pokrywka]

I think that the main argument against HOCON was not "it is not Scala" but its complexity and power. Directives make for a different (simple) language, regardless if only scala-cli or other tools (Scala compiler) can understand it, and regardless whether they would be within a comment or in a separate file.

Directives in comments continue a long tradition of embedding structured information into program's comments. Compiler pragmas, hints on the used character set, javadoc come to mind. Not to say that it is ideal, but it works quite well in a world that's not ideal, either.

Embedding a different language within comments lets one keep the whole tooling ecosystem to work non-distrupted, which is a great productivity booster.

Whether the directives should become part of future Scala, outside of comments, I don't know, that'd make for an entirely new and possibly long discussion.

[bjornregnell]

I like the idea of keeping it simple -- then people does not need to learn HCON just to put it in a file. It is indeed much more uniform to have the same syntax and semantics in the simple comment based `// using ...``and in the config file. The potential strengths of HCON does not compensate for the added complexity IMHO.

[SethTisue]

I guess //> is now settled, but then what about using?

I believe the word using was originally chosen because it's already a keyword in Scala 3. But now that we've settled on the comment syntax, there's no need for keyword parsimony, and in fact, clashing with an existing keyword is also a significant negative and just creates unnecessary ambiguity, in my opinion.

I don't mean parsing ambiguity; I mean ambiguity in speech, in casual references to using in chat rooms and forum posts, and when Googling things.

Two possible alternatives:

1. Find a different word
2. Use no word at all, and allow //> to be sufficient to introduce a directive such as lib or scala

[SethTisue]

(I wasn't sure whether to raise this here, or in the SIP context, or what. This did come up at the SIP meeting last week, briefly, but there wasn't time to delve into it.)

[izzyreal]

Great point. I'm somewhat surprised so far nothing seems to have been done with it, at least not visibly through https://scala-cli.virtuslab.org/docs/reference/directives/ where the second header, "using directives", is a great example of said ambiguity, and could easily be read as "how to use directives".

[bjornregnell]

When now the scala-cli == the new scala is accepted by the SIP committee as an official thing in stage experimental, I think we should set up an official thread at https://contributors.scala-lang.org/ to gather feedback and link to that from the official download page from scala-lang.org when an installer is ready that makes the new scala available on path. Your comment, @SethTisue is exactly such things I would expect to come in the feedback round as more and more tries the new scala out, to hraden it before the SIP will vote for promoting it to stable. Could you do that or should someone from Virtuslab setup a feedback thread once the installer is live?

[SethTisue]

um, someone from Virtus. I'm just a bystander, cheerleader, and feedback-provider :-)

happy to repeat my remarks on the feedback thread once there is one

[bjornregnell]

ok great (sorry for trying to land that potato in your knee :) ) I can start a thread myself at contributors if not someone from Virtuslabs rather prefer to do it, e.g. @romanowski

[romanowski]

@bjornregnell we will set up a feedback thread within the next few days. I wonder maybe we should now have a general thread for Scala CLI = Scala and one specialized for using directives?

[bjornregnell]

@romanowski I think I prefer one thread where you 1) explain that it was accepted by the SIP committee as experimental an 2) outline a couple of areas where we want feedback in particular including the using syntax. I prefer one thread so that contributors that want to give feedback knows where to go (although a downside could be that the thread might get too long, but we could always spawn sub-threads later).

I also think that there should be ready, when the feedback thread goes live, a link to an installer on the scala-lang.org Install page where visitors can choose between the old and new scala command and a link to the feedback thread on contributors with an encouraging text to give feedback.

[SethTisue]

I would like to re-raise the issue of the choice of the word using. See my remarks four months ago, above.

It was initially chosen exactly because it is a Scala 3 keyword, and the idea was to allow it at the top level: using scala ...

Since then, we have settled on the //> syntax.

It seems to me that the original reason for choosing using is now exactly wrong, exactly the opposite of what is desirable. There is no need to choose an existing Scala 3 keyword, and there are excellent reasons not to. scala-cli's using and Scala 3's using simply aren't the same thing at all and shouldn't have the same name.

I thought this might be resolvable here on this thread, but since there hasn't been much discussion, I'm inclined to push to bring it up at the next SIP meeting (cc @lrytz).

I don't have a strong feeling whether it's better to choose a different word, or omit the word and simply have //> dep ... and //> scala ... and so on. Either seems plausible to me. But I believe that continuing with using is an error we shouldn't make, and the sooner we change it, the better. (We just missed the 0.2.0 boat.)

[bjornregnell]

Thanks @SethTisue for reminding us of this relaxed constraint opportunity on directives. Yes if it should be changed, the sooner the better.

I'm happy to bring this up on the next SIP meeting @julienrf if we can prepare the agenda item before the meeting by discussing this here, in particular focus on peoples' views on:

1. Should we change using, given the arguments put forward by @SethTisue above? Esp. we need to surface pros and cons of changing. @romanowski It would be good with your input on how this would affect the scala-cli roadmap.
2. If we want to change the directives syntax then how should we change it? No keyword or another keyword?
3. How do we go about migrating to a changed directive syntax, including deprecation warnings and a grace period where the old syntax still works.

In regards to number 2 I propose use as a keyword:

//> use scala "3.3.0-RC2"
//> use toolkit "latest"

I have actually more than once by accident written use instead of using in directives to scala-cli and then scratched my head hard on what's going wrong: #1636 and I think there is something inherently intuitive by telling my build tool to use stuff in imperative form ("use" rather than "using", "build" rather than "building", "run" rather than "running" ...)

[bjornregnell]

I agree that a specific keyword is better than no keyword as it 1) says something to a reader rather than just a cryptic //> and 2) that it is less of a change to change the keyword than to remove it compared to what people have gotten used to.

Yes, the "best" actual keyword is subjective, but the fact that the current keyword using overloads the meaning of a language keyword is objectively true. We can debate how serious that problem is, but I think it is true to say that that changing it later is worse than changing it earlier as more and more people will get used to it and what happens after "experimental" is "stable", thus with the expectation of less change.

It does not have to take a long time if our goal is to keep have a keyword but have it not clash with Scala's using. I have already suggested use and it fulfills that and if no-one discovers problems with that keyword, there is nothing stopping us to decide it decided fast.

I'll try to summarize this at contributors now, and we'll see what people there thinks.

[julienrf]

In my opinion, reusing “using” is not so bad because that keyword is about contextual abstractions. Here, the directives also define the context (Scala version, dependencies, etc.) in which the program can be compiled and run.

[bjornregnell]

Yes, there are some similarity in meaning, but they are still different and e.g. searching the net for scala using might be confusing etc. Anyway -- perhaps you want to chip into the discussion @julienrf @tgodzik with these and other arguments on the Contributors forum so that we get discussions going there among people not following this thread but are hanging on Contributors...

[sjrd]

I'm going to pull a @som-snytt and remark that writing your conf with using might be confusing.

[kubukoz]

confusing would be a great library name :D

reusing “using” is not so bad because that keyword is about contextual abstractions

I seem to remember another keyword that had several interconnected meanings...

[prolativ]

If there's any chance we're going to modify the syntax of using directives I think it would be good to take into account the fact that currently there are actually two kinds of directives, with very different semantics, and clear separation of them would be beneficial:

1. target directives (like //> using target.foo, e.g. //> using target.platform "scala-js" or //> using target.scala.>= "2.13") - they apply only to the file in which they occur and specify to which build scope the file belongs
2. "setting" directives (all the remaining like //> using foo, e.g. //> using dep "com.lihaoyi::upickle:3.0.0-M1") - they apply to all other files in the same build scope

So while in most cases it doesn't really matter in which file a directive is added, it does matter if this file also contains any target directives because then the "setting" directive will be applied only in this build scope.

Even though using might seem redundant by itself if it is always written after //> we could have a clear syntactic distinction between the two types of directives by writing e.g.
//> target platform "scala-js"
//> using dep "com.lihaoyi::upickle:3.0.0-M1"

[bjornregnell]

Aha! That supports revising the syntax and keeping a keyword for all settings directives.

With a new keyword, here use, it looks like this:

//> target platform "scala-js"
//> use dep "com.lihaoyi::upickle:3.0.0-M1"
//> use scala "3.3.0-RC2"
//> use toolkit "latest"

[prolativ]

There are also two aspects of the using directives syntax that seem kind of forgotten (or at least I can't remember seeing anyone actually using them):

1. reusing using keyword for multiple/all directives

//> using
//>   target.platform "scala-js"
//>   scala "3.3.0-RC2"
//>   dep "com.lihaoyi::upickle:3.0.0-M1"
//>   toolkit "latest"

2)using multiline comments

/*> using target.platform "scala-js"
    using scala "3.3.0-RC2"
    using dep "com.lihaoyi::upickle:3.0.0-M1"
    using toolkit "latest"
*/

One can even combine these two options

/*> using
      target.platform "jvm"
      scala "3.3.0-RC2"
      dep "com.lihaoyi::upickle:3.0.0-M1"
      toolkit "latest"
*/

Should we keep supporting these or should we have just a single way to express the configuration of a project?

[SethTisue]

didn't even know those syntaxes existed. I would suggest removing them.

[som-snytt]

//> for target and //< for dependency and //| for config

[bjornregnell]

I have now pinged the community for specific input here:

https://contributors.scala-lang.org/t/pre-sip-scala-cli-as-new-scala-command/5628/28?u=bjornregnell

@SethTisue @tgodzik @julienrf @prolativ @som-snytt @keynmol Let me know if I can improve something in the above post on contributors to better solicit good feedback.

[bjornregnell]

I also just noted that there is a big red warning on that using might be subjected to change here:
https://scala-cli.virtuslab.org/docs/guides/using-directives/
So changing it could be OK from that pint of view.

[bjornregnell]

I have just written a summary of the discussions by contributors and a sketch of my recommendation here: https://contributors.scala-lang.org/t/pre-sip-scala-cli-as-new-scala-command/5628/79?u=bjornregnell

[bjornregnell]

My current thinking, informed by the engaging debate at contributors, including arguments and counter-arguments for using using as keyword or not, is to give my (revised) recommendation as a SIP-46 reviewer as follows:

• keeping the one-line using-syntax as is, release scala-cli as scala, after multi-line directives syntax is removed + tidy up docs + update SIP-46 accordingly while promoting it from experimental to stable as soon as possible,

and then encourage a new SIP with a proposal base on more analysis of different use cases and requirements on a possible configuration syntax including the potential of alignment across runner and other tools, in dialog with maintainers of such tools. If it should be based on some subset of JSON, TOML, YAML, ..., is part of that investigation, in conjunction with a new SIP.

If such a future SIP proposal is successful, then the current //> using- syntax can work alongside a potential new config syntax, perhaps for as long as until a couple of LTS releases or similar.

[SethTisue]

SGTM. I said so on Discourse as well.

[som-snytt]

I just realized I don't know how //> using options x, y, z is parsed.

Currently, partest's // scalac: x y z tokenizes using scala.sys.process.Parser.tokenize, so it looks like a shell command line with quoting conventions one might be comfortable with.

I haven't experimented yet, but might guess that //> using options "-Wconf:cat=deprecation:e,cat=feature:s", -Xlint works because double-quotes and commas just work. Maybe that is the only rule. I could only guess about embedded quotes and backslashes, which is why I liked using tokenize.

[SethTisue]

If you want to use the official parser, https://github.com/VirtusLab/using_directives is pure Java and allowable as a test dependency.