Expressions that are "as constant as possible"

[eernstg]

The language team has had discussions about the need to change many parts of an expression in response to a small modification that makes a previously constant subexpression non-constant, or vice versa. For example:

class A {
  const A([_, _, _, _]);
}

int v = 1;

void main() {
  // Assume that we have this expression somewhere.
  const A(
    A(), 
    1, 
    'Hello', 
    A(<int>[], A(), A()),
  );

  // Now we need to change `[]` to `[v]`, where `v` isn't constant.
  A(
    const A(), 
    1, 
    'Hello', 
    A(<int>[v], const A(), const A()),
  );
}

This causes a lot of inconveniences in daily development because it's a non-trivial editing operation to make "as much as possible" constant in a given expression when we can't just say "all of it".

Arguably, the process is even more tricky in the other direction, because we can't see locally whether or not the list is expected to be modified in the future.

A more subtle semantic difference is the behavior of identical, where constant expressions yield canonicalized instances, which means that const <int>[] is the same object as another occurrence of the syntax const <int>[], whereas <int>[] and <int>[] are two distinct objects (assuming that they aren't constant based on a const modifier on some enclosing expression), even in the case where we are considering two distinct evaluations of the same expression in the source code.

Proposal: const? expressions

We introduce the expression modifier const?, which amounts to a request that every subexpression of the expression that carries this modifier is constant, if possible. The grammar changes as follows:

<constQuestion> ::= 'const?'
<constOrQuestion> ::= ('const' | <constQuestion>)
<setOrMapLiteral> ::= <constOrQuestion>? <typeArguments>? '{' <elements>? '}'
<listLiteral> ::= <constOrQuestion>? <typeArguments>? '[' <elements>? ']'
<recordLiteral> ::= <constOrQuestion>? <recordLiteralNoConst>
<constObjectExpression> ::= <constOrQuestion>? <constructorDesignation> <arguments>

<primary> ::= ... // Existing cases unchanged
    <constQuestion> '(' <expression> ')'

The point is that it is much safer for an automatic mechanism to introduce const modifiers on a set of expressions if this operation is justified by an explicit request in the source code. We could say that the developer promises that it won't be a bug to implicitly add const to any of the subexpressions of the expression that has the modifier const?, nor will it be a bug if no such modifier is added.

The example above then becomes simpler:

void main() {
  // Assume that we have this expression somewhere.
  const? A(
    A(), 
    1, 
    'Hello', 
    A(<int>[], A(), A()),
  );

  // Now we need to change `[]` to `[v]`, where `v` isn't constant.
  // Just go ahead and add `v`, no other change is needed.
  const? A(
    A(), 
    1, 
    'Hello', 
    A(<int>[v], A(), A()),
  );
}

This would largely eliminate the churn which is caused by changes to the constness of subexpressions of a bigger expression.

We define the semantics of the const? modifier in terms of constant capable expressions. On the target expression, the mechanism would add const modifiers implicitly, in a bottom-up traversal, such that every expression gets the const modifier if and only if it is constant capable.

• An expression is constant capable if it has the modifier const. For example, const A() or const <int>[].
• An identifier is constant capable if and only if it is a constant expression (this is a reference to the rules that we have today). For example, a name that denotes a constant variable or a top-level function.
• An instance creation expression is constant capable if (1) every actual type argument is a constant type expression, or actual type arguments have been inferred and they do not contain any type variables, (2) every actual argument is constant capable, and (3) it invokes a constant constructor. For example, C<int>(1).
• A list literal is constant capable if (1) it has an actual type argument which is a constant type expression, or it has no actual type argument and the inferred type argument (fully alias-expanded) does not contain any type variables, and (2) every element is constant capable. Similarly for a set literal and a map literal.
• An operator expression (like e1 + e2, !e3, e4 ? e5 : e6) is constant capable if the operands are constant capable, and if it yields a constant expression when const is added (where it can be added syntactically) in a bottom-up traversal of the expression, to every subexpression which is constant capable. Similarly for other composite expressions (that is, expressions whose constness depends on the constness of its subexpressions, and possibly other criteria like the actual value or its type).

With this mechanism in place, it would be possible, for example, for a Flutter developer to put a const? at the outermost level of a large expression creating a widget tree. This expression would then be a constant expression "as far as possible". This would amount to the same thing as adding const at the top, or adding const to any number of children, depending on the ability of each subtree of the widget tree to be constant or not. Crucially, there's no need to change anything about constness in the entire expression when some tiny subexpression is changed from constant to non-constant; we just proceed to make that change, and the constness of the entire expression will then implicitly be modified as needed.

Proposal: const? constructors

@abitofevrything introduced the following idea.

A constructor can have the modifier const?. The grammar changes as follows:

<factoryConstructorSignature> ::=
    <constOrQuestion>? 'factory' <constructorName> <formalParameterList>
<redirectingFactoryConstructorSignature> ::=
    <constOrQuestion>? 'factory' <constructorName> <formalParameterList> '=' <constructorDesignation>
<constantConstructorSignature> ::=
    <constOrQuestion>? <constructorName> <formalParameterList>

Exactly the same compile-time errors are raised for such a constructor declaration as would be the case if it had had the modifier const.

A constructor that has the modifier const? is a constant constructor. This implies that it can be invoked with the const modifier, yielding a constant expression, subject to all the usual checks to confirm that said expression is indeed a correct constant expression. Also, const can be omitted when the constructor is invoked in a constant context, just like other constant constructors.

Assume that e is an instance creation expression that invokes a const? constructor k. If e does not start with const or new, and e does not occur in a constant context then e is treated as const? e.

This implies that invocations of this constructor will always be "as constant as possible", except that an e of the form const e1 requires the invocation to be a constant expression, no ifs or buts, and new e1 requires the invocation to be non-constant. Note that new has the same effect as removing const? from the constructor, in the sense that it also cancels the automatic conversion of subexpressions to constant expressions, unless they have their own reason te become constant, e.g., by having their own const modifier, or being invocations of a const? constructor, etc.

Proposal: const? parameters

A formal parameter declaration in a function declaration would be able to have the modifier const?. The effect would be that every actual argument a which is passed to this parameter in a situation where the statically known signature of the function has this modifier on the given parameter will be treated as const? (a). In other words, every actual argument passed to this parameter will be "as constant as possible".

The grammar changes as follows:

<normalFormalParameter> ::=
    <metadata> <constQuestion>? <normalFormalParameterNoMetadata>

<defaultNamedParameter> ::=
    <metadata> <constQuestion>? 'required'? 
    <normalFormalParameterNoMetadata> ('=' <expression>)?

Discussion

A fundamental property of this proposal is that it makes the request for constant expressions more flexible ("just write const? at the outermost level, and the compiler/analyzer will sort it out"), but it preserves the property that it takes a human expression of intent to make an expression constant.

With the first proposal (supporting const? e where e is an expression), the expression of intent is local: It is always possible to see the keyword const? somewhere in the physically enclosing source code.

The second proposal is intended to be an addition to the first proposal, not a replacement. It is more aggressive than the first one, in the sense that it will make some expressions constant without any locally visible indication.

The second proposal relies on the assumption that there will be some classes (or at least some constructors of some classes) whose semantics is such that it is always OK to choose to make the constructor arguments constant as far as possible, because there are no bugs which can be caused by making the choice to turn some of those constructor arguments and/or their subexpressions into constant expressions, and there are no bugs which can be caused by not turning them into constant expressions. This is a heavy burden to lift for the authors and maintainers of that constructor, and the enclosing class, and its collaborators, but it might work quite well in practice. The community will have to develop a culture around how to use this feature well.

Conversely, the notion of a const? constructor will allow code that creates instances of these classes to obtain a maximal amount of const-ness in client code, with no effort (writing or maintaining) from the client developer.

Finally, the third proposal allows us to specify that every actual argument a to a specific formal parameter must be treated as const? (a), which means that we make "constness" the default for those actual arguments.

The second and third proposals are somewhat overlapping: If we choose to make a constructor const? then the constructor must also itself be a constant constructor, and every actual argument passed in an invocation of that constructor is automatically made "as constant as possible". We can achieve very much the same effect by marking every formal parameter of the same constructor as a const? parameter. This means that every subexpression will be "as constant as possible", but the invocation of this constructor itself will not be constant (unless there is some other reason why it is requested to be constant).

const? formal parameters can be parameters of any function, including static and instance methods, local functions, function literals, etc. This means that they are a much broader mechanism than const? constructors.

Finally, note that function types do not support marking a parameter as const?, it is a mechanism which is only available for invocations of function declarations, and it has no effect on invocations of function objects, nor on dynamic invocations.

Versions

• Sep 25, 2024: Add proposal about const? parameters.
• Sep 10, 2024: Add proposal about const? constructors. Change the phrase 'evidently constant' to 'constant capable'.
• Sep 6, 2024: Initial version.

[abitofevrything]

Could this be expanded to allow const? constructors, e.g const? Widget(), such that any invocation of that constructor is made "as constant as possible"?

This would allow data classes that should never preoccupy themselves with canonicalization issues to advertise they should be const when possible, without the developer having to add const or const? to every instance creation expression.

Of course this then creates the issue that a constructor might be const?, but then a developer using that class might run into a use case where they need the instance not to be const, so we'd need to introduce a way to explicitely disable the implicit const-ness.

[abitofevrything]

A user on discord (@wdestroier) has pointed out that new already provides a way to force the un-constedness of an expression.

[eernstg]

Could this be expanded to allow const? constructors, e.g const? Widget(), such that any invocation of that constructor is made "as constant as possible"?

That's an interesting idea! It might be possible. It might also be a bit too implicit (in the sense that we'll end up having some lists that are constant "by accident", even though they must actually be mutated at some point). But it's certainly worth keeping in mind.

... need to introduce a way to explicitely disable ...

Indeed, that's another reason why it might be a bit too aggressive to have a const? constructor.

new already provides a way to force the un-constedness of an expression.

Right. We can't say new [1, 2, 3], but perhaps that would be the way to do it.

[lrhn]

Can you add const? only where you can write const today, or can you write const? (1 + 2) too?
Can you use it on the LHS of a declaration, const? c = ...;?

I'll assume it's only a replacement for const, so before an object creation or collection literal expression.

That would put the reast of that expression in a const? context, like const puts it in a const context.

Then we use the same rules as for constant expressions today, with small twists:

• If it's not one of the expression contexts which can be constant, it's not constant.
• Otherwise if it's an object creation or collection literal prefixed by const then that introduces a const context on subexpressions.
• Otherwise the subexpressions are in the same const or const? context as the parent expression, and we recurse as normal.
• When we come back, then for every syntactic form we use the same rules as today for deciding whether the expression is constant, but if it's in a const? context instead of a const context, it's not an error to not be constant.

So effectively we pass down a flag on the const context that says whether it's going to be an error to not be const.
If any required expression ends up not being constant, and therefore not having a value at compile time, then so does the expression containing it.

There are expressions where a sub-expression doesn't have to be constant, if it's in a branch not taken.

const c = true ? "banana" : (1 ~/ 0);

Here (1 ~/ 0) is not a constant expression because it doesn't evaluate to a value. It throws. But it's also not used.

If we do:

var v = const? [bool.hasEnvironment("override") ? int.fromEnvironment("override") : computeValue()];

then the expression is not "syntactically a constant expression", which is a bad concept anyway.
It should be equivalent to:

var v = [const bool.hasEnvironment("override") ? const int.fromEnvironment("override") : computeValue()];

I think this can be built into the existing rules with fairly little work. The biggest issue would be existing assumptions and bad definitions (and the lack of actually defining "constant evaluation" and when it happens, making var x = "a" + "b";'s initializer expression be not in a constant context, but "is a constant expression". Is it evaluated as a constant? That's the question we need to answer.).

[lrhn]

Can you add const? only where you can write const today, or can you write const? (1 + 2) too?
Can you use it on the LHS of a declaration, const? c = ...;?

I'll assume it's only a replacement for const, so before an object creation or collection literal expression.

That would put the reast of that expression in a const? context, like const puts it in a const context.

Then we use the same rules as for constant expressions today, with small twists:

• If it's not one of the expression contexts which can be constant, it's not constant.
• Otherwise if it's an object creation or collection literal prefixed by const then that introduces a const context on subexpressions.
• Otherwise the subexpressions are in the same const or const? context as the parent expression, and we recurse as normal.
• When we come back, then for every syntactic form we use the same rules as today for deciding whether the expression is constant, but if it's in a const? context instead of a const context, it's not an error to not be constant.

So effectively we pass down a flag on the const context that says whether it's going to be an error to not be const.
If any required expression ends up not being constant, and therefore not having a value at compile time, then so does the expression containing it.

There are expressions where a sub-expression doesn't have to be constant, if it's in a branch not taken.

const c = true ? "banana" : (1 ~/ 0);

Here (1 ~/ 0) is not a constant expression because it doesn't evaluate to a value. It throws. But it's also not used.

If we do:

var v = const? [bool.hasEnvironment("override") ? int.fromEnvironment("override") : computeValue()];

then the expression is not "syntactically a constant expression", which is a bad concept anyway.
It should be equivalent to:

var v = [const bool.hasEnvironment("override") ? const int.fromEnvironment("override") : computeValue()];

I think this can be built into the existing rules with fairly little work. The biggest issue would be existing assumptions and bad definitions (and the lack of actually defining "constant evaluation" and when it happens, making var x = "a" + "b";'s initializer expression be not in a constant context, but "is a constant expression". Is it evaluated as a constant? That's the question we need to answer.).

[eernstg]

Can you add const? only where you can write const today,

That's my starting point, but if it turns out to be useful to allow it in additional positions then we can scrutinize the grammar and see if it can be done. Allowing const? (<expression>) is probably easy to handle in the grammar.

Can you use it on the LHS of a declaration, const? c = ...;?

That seems less intuitive to me. I'm thinking of const? as an operator that works on a structure, not a property of the variable. For example, we might want to make the initializing expression as constant as possible, and still allow the variable to be mutable or not, and late or not. In that situation const? seems to be noise more than signal, if it is intended to have an effect on the initializing expression and not on the variable.

A 'const? context' is a nice way to describe it. I'm not sure about the distinction (it might be completely equivalent to the notion of 'evidently constant expressions' that I mentioned), or about pros/cons in this choice.

[abitofevrything]

(in the sense that we'll end up having some lists that are constant "by accident", even though they must actually be mutated at some point

Simple enough: don't make the list constructors const?.

I would see const? constructors as more of a feature for custom dataclasses, e.g classes generated by freezed, or maybe potentially widget classes.

[cedvdb]

Extending this to declarations, it could look like const? a = ... similar to how a const declaration doesn't need an additional redundant const. Assuming this would work for non const constructor, it would remove almost all need for final. const? foo = MyNonConstConstructor( ConstConstructor() ) - except when you do not want to use consts at all, for equality reasons.

I'm also wondering if pushing const so much (via the analyzer, prefer const) was the way to go. I've yet to see a realistic benchmark that a flutter project using const has a non negligile performance boost.

[mateusfccp]

I'm also wondering if pushing const so much (via the analyzer, prefer const) was the way to go. I've yet to see a realistic benchmark that a flutter project using const has a non negligile performance boost.

It seems there are many people who don't like consting the code. You are not the first one I've seen complaining about this linter, and many even disable it and simply don't put const in front of their constable objects.

Particularly, I think it's a costless optimization. Even if it may be negligible for many cases, it's basically costless, except for a little bit of verbosity, and this is exactly what this issue proposes to solve.

There's an alternative approach. We could make something similar to what was proposed in #1410, but instead of making everything const, make everything const?, according to the semantics defined in this issue.

Then, we would have to revert const-ness to be opt-out instead of opt-in. That is, whenever we don't want a constable object creation to be const, we would have to use new.

Is this better than what we currently have? I'm not sure. We would have to extract some data to have a better idea which case is more common. In my personal experience with Flutter, we would have much less new than we have const today (considering that we const everything that should be const). I have no extensive experience with non-Flutter Dart to have any impression on it.

However, even if the data says that this is also true for non-Flutter Dart, it is an extremely breaking change to introduce in the language now.

[cedvdb]

@mateusfccp I wondered about that too

const? opens the door to const? by default because it does not suffer from the same drawbacks as const by default.

The first argument against const by default was that people would not see if something is not "consted". This is solved by const? by default, as if you want errors if an expression is not a const, you can still use const.

The second issue with const by default, and admittedly the bigger one was the identity issue. Const? by default could be implemented in such a way that every declaration that is not const a = ... or const? a = .. won't be const?ed by default, ie: like the "noconst" that was proposed somewhere (if that's even possible for the compiler). So every var a = ... and final a = .. out there would still act as they do today, if there is no const? present in the expression, regarding potential identity issues that would have been otherwise introduced if it was applied everywhere. So as long as there is an assignment the identity issue would not applicable. Other expressions would be const?ed by default. Potential identity issues would therefor only appear where there a is const constructor AND there was no assignment, which greatly reduces the surface. So code like this would still be different pre / post const? by default:

void main () {
  print(buildFoo() == buildFoo());
}

buildFoo() {
  return MyConstConstructor();
}

But this would not:

void main () {
  print(buildFoo() == buildFoo());
}

buildFoo() {
 final foo = MyConstConstructor(); 
 return foo;
}

Still a dangerous change but maybe one worth investigating ? I don't know, just an idea.

I think it's a costless optimization

Well it's not costless at the moment, that's why this proposal exists

[lucavenir]

I like the fact that there's const? and const. The former helps with an optimization, the latter is an explicit take the developer chooses to follow.

Still, I'm not sure I've understood how this proposal helps with declarations. That's half of the times I encounter the "churn" problem. Example:

class Point {
  const Point(this.x, this.y);
  final double x;
  final double y;
}

const x = 2.0;
const y = 3.0;

// can be const, so the auto fix will eventually make this `const` on save
final p = Point(x, y);

At this point I save and the lint successfully turns final p into const p.

// ...
const p = Point(x, y); // auto fixed

Then, I need to change y for some reason.

import 'dart:math';
// ...

const x = 2.0;
const y = Random().nextDouble();

const p = Point(2, 3);  // error

So const? would help even on the LHS.
The difference with const, at that point, would be explicitness.

In other words, if I understood this proposal correctly, when performance is a must, the developer must avoid const?.

[eernstg]

@lucavenir, did you mean Point(x, y) rather than Point(2, 3)? Also, final y rather than const y in the second version? Otherwise, I can't see why there would be an error at the declaration of p.

You could use the following if you want the best performance, and you don't want to touch the declaration of p in response to a change to the constness of x and/or y:

final p = const? Point(x, y);

This would make p final, but not constant, which means that it cannot be used in constant expressions. However, p will be initialized with the value of a constant expression if and only if both x and y are constant doubles. This means that p will be just as good as a constant variable when it is used for any other purpose than being a subexpression of a constant expression (where it's an error).

Of course, you could also declare p as const p = const? Point(x, y); but that's silly because that occurrence of const? can never make a difference: if it doesn't simply mean const Point(x, y), it's an error because we have const p, and const Point(x, y) is indeed the meaning of Point(x, y) when p is constant.

when performance is a must, the developer must avoid const?.

Well, if you insist that a given expression must be a constant expression then you must use const rather than const?.

But otherwise it's the other way around: You would use const? in a large number of locations in order to make the target expression a constant expression as far as possible, which would presumably ensure that your program has a larger number of constant expressions than it would otherwise have had (because we don't like the churn, so we don't add const meticulously in every single location that allows it).

[lucavenir]

@lucavenir, did you mean Point(x, y) rather than Point(2, 3)? Also, final y rather than const y in the second version? Otherwise, I can't see why there would be an error at the declaration of p.

Yes, sorry! I'll edit my comment to make it clearer.

[rakudrama]

I would hope that a const? expression modifier as proposed would be extended to declaring const? constructors.

  const? Point(this.x, this.y);

For the use-case of Flutter widget trees, framework users would almost never have to write const or const?.

See #3399

[Levi-Lesches]

After all this discussion, I'm still not sure I see the benefits of const?.

• It makes more declarations about const-ness implicit, which isn't necessarily good
• It hides whether the objects are actually const, forcing you to either notice or use the standard form, const
• It adds another option, "plain", const, or const? to the already-confusing concept of const-ness
• It doesn't solve the case of having a non-const object deep inside a const? object
• I can see it becoming encouraged as "use const? when you don't know what to do", and then objects suddenly become const, which will trip up beginners because they can't see what is and what isn't const
• Judging by the conversation here, it's not necessarily clear to everyone what the exact semantics of const? are or would be.

Given that the original "problem" was the chore of needing to add const everywhere, I would agree with this comment and say that running dart fix is quick enough to make this a moot point. As an advantage, developers are made aware of when objects can now or no longer be const, and get to make those choices for themselves, while preserving readability in the future.

[cedvdb]

As an advantage, developers are made aware of when objects can now or no longer be const, and get to make those choices for themselves, while preserving readability in the future.

I somewhat disagree with this.

• Often times I spin up a test project just to try things out and I have to deal with const / unconsting stuff. When I'm just prototyping this gets in the way. This is especially true in flutter, where your top widgets may be const then 2 minutes later when you iterate over a design, it's only the middle parts.
• The fact that prefer const is used by default is weird, most projects don't reach the production phase, let alone a squeeze as much perf as you can phase.
• Having the option to have the potential performance gains and not having to care about the above would be nice. Ultimately as an end user a mechanism that wouldn't necessitate an additional marker would be better though.

[eernstg]

I adjusted the proposal to include const? constructors. I'm a little worried about their ability to introduce subtle bugs at a grandiose scale, but it might be OK when we get used to them. 😀

[cedvdb]

@eernstg

about const? constructors

This implies that it can be invoked with the const modifier,

It would be more useful if a const? constructor would be allowed on constructors that cannot be const.

Otherwise in flutter, in the build method, return const? will still have to be used unless you know for a fact that the constructor of the top level widget is const?, which would be painful to check (and maintain) on a case by case basis. This would render their use a bit moot.

The implications are the same anyway, concerning potential bugs this could introduce.

I'm a little worried about their ability to introduce subtle bugs at a grandiose scale, but it might be OK when we get used to them.

This looks more dangerous than if Dart was designed from the get go with a noconst modifier and const? by default. That would just be a feature of the language, easy to reason about, here it's feature of the constructor. Personally, I'm not comfortable with it yet.

[eernstg]

It would be more useful if a const? constructor would be allowed on constructors that cannot be const.

Oh, that's a great point! So we'd want to express the rule that all subexpressions of a given constructor invocation should be treated as if they were in scope of a const? expression modifier, even though we already have a guarantee that we can't make the outermost expression const.

One gnarly problem is the syntax: I to think that a constructor declaration like const? const A(); is intolerably keyword heavy.

We could also consider having support for a const? modifier on a formal parameter declaration. This would ensure that every actual argument a passed to that parameter would be treated as const? a.

We would then be able to express the behavior that you mention by adding const? on all formal parameters of the constructor which should otherwise have the const? modifier even though it can't be const itself (so we avoid the contradiction, and we avoid the bulky use of const? const).

[eernstg]

I added a proposal for const? parameters as well.