Proposal: changes to type: $dimension and type: $duration tokens

[drwpow]

Summary

This proposes the following changes to $type: dimension and $type: duration tokens, which have the same designs:

• Changes $value from a string → object with value and unit keys for both dimension and duration

Reasoning

This PR proposes that the following discussions be resolved like so:

• Split units and values in type definitions #121: ✅ Accepted (as written)
• Remove REM/EM from specification? #218: ❌ Rejected (that moves in the opposite direction, turning dimension/durations into number-only values)

Pros

• Simplifies tooling because these token types would no longer need string parsing (which could potentially introduce bugs)
  • Other types like shadows and borders already split their values into an object with keys
• Reduces errors in output.
  • For example, if someone was exporting dimension tokens to .js, they could normalize everything to px and just output numbers (which is the default behavior of JS styling). Tooling wouldn’t have to try and parse the string from the number, potentially introducing bugs.
• Simplifies validation because string pattern matching is no longer required
  • Validation is now as simple as checking for the object wrapper, both keys, and both keys match their expected primitive types.

Cons

• Objects are more verbose to type

Alternatives

• We could be more lax about $value.unit for dimensions, however, that seems to introduce more problems and regress that part of the schema (see comment below)
• We could accept Remove REM/EM from specification? #218, and downconvert all dimensions into px values and durations into ms. But this would break many setups in tokens (chiefly, web—rems and px are distinct values with different uses, and based on implementation, aren’t always mathematically-equivalent).
  • We could also split the difference, and keep the object for dimensions, and just number for duration (standardizing on milliseconds only). That’s just “syntactic sugar” anyway.

Notes

• The original proposal had $value.number instead of $value.value, possibly to reduce duplication but I didn’t find a rationale. This changes to value because that’s a more common nomenclature for this term, e.g. csstree:

  

(PR description edited from original version to reflect changes in code)

[netlify]

✅ Deploy Preview for dtcg-tr ready!

Name	Link
🔨 Latest commit	896be91
🔍 Latest deploy log	https://app.netlify.com/sites/dtcg-tr/deploys/66c4a509595753000823ef57
😎 Deploy Preview	https://deploy-preview-244--dtcg-tr.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[drwpow]

Note: this new subheading is inconsistent with the other sections. This was done thinking back to #200 that observes the spec could be more explicit about validation.

This would be a change from the current structure of validation rules existing in prose, usually in the initial description. However, in many places, those aren’t exhaustive/complete (for good reason—it’d be hard to read).

All that said, we don’t have to introduce these subheadings in this PR; we could fold everything into prose form to match the existing format.

[dev-nicolaos]

I like this proposal, it seems likely to make parsing easier for tools without introducing much burden on those who'd prefer to write tokens by hand.

Making dimension units unrestricted seems like a win for platform agnosticism too. The web has so many dimension units and this lets token maintainers take advantage of that without introducing a bunch of web specific stuff to the spec.

I wonder if the value property names couldn't use a little bikeshedding but, I can't think of anything better right now.

[romainmenke]

Thank you for this proposal @drwpow!
Overall this looks good to me.

---

The original proposal had $value.number instead of $value.value, possibly to reduce duplication but I didn’t find a rationale. This changes to value because that’s a more common nomenclature for this term,

Yes, $value.value is better!

---

Dimensions: no restrictions on what $value.unit can be (author-defined)

We can't have unrestricted units.

We will end up having these issues:

• tools will have conflicting unit definitions, fracturing the ecosystem
• tools will squat units that should be in the spec itself, but with a different definition than desired for the specification

Instead we should have a fixed set of specified units and allow any other unit as long as it is vendor prefixed.

Then there can't be conflicts between tools as those would be different vendors.
There also can't be conflicts between tools and future spec expansion.

It still leaves plenty of room for custom definitions.

Keep in mind that design token files are a carrier format.
How a unit is encoded in a file doesn't dictate how it is exposed in a design tool or the final output for developers.

Figma can create a custom unit y that is exposed to designers as y and output to developers as y while encoding it as figma-y in token files.

If a custom, vendor prefixed, unit is popular and gains adoption among other tools it is also a clear signal that it is useful. Such a custom unit should then go through the spec process to standardize it.

Instead of figma-y as the encoded unit it could also be a separate prop.

{
  "value": 10,
  "unit": "y",
  "vendor": "figma"
}

[drwpow]

@romainmenke Those are all great points, thanks for raising. I was originally trying to solve for multiple things here, but agree that may be too big a change here:

• Tools may interpret units differently anyway (e.g. 10px == 1rem). But to your point, the specification should still define these values because then it’s clear the tool doesn’t match the spec
• I was scared the review process of adding more units may take too long, and many other units hadn’t even been been proposed. But that’s also a “feature, not a bug,” and no point in throwing away the consensus we already have.

But you address those nicely with the vendor addition proposal. I really like this version specifically for the following reasons:

{
  "value": 10,
  "unit": "y",
  "vendor": "figma"
}

• I agree not using $extensions here because that is not part of the value, and shouldn’t affect interpretation of the value in every tool ($extensions are for individual tools, and should be discardable by default)
• Having some sort of prefix in unit is dicey because it brings back string parsing/pattern matching that this PR tries to move away from that puts it back on tooling. Defining what a valid or invalid prefix is seems like a can of worms (best case scenario it’s tough to tell where the prefix ends and unit begins; worst-case scenario we’re back in “allow all units” territory but more complicated).
• The inclusion of vendor as an optional property is great because it unlocks more units, without disabling validations.
• We also have other tokens (including @resource11’s upcoming proposal to the color format) where the addition of one object key changes the interpretation of the other values (it’s all one “value”)

Minor point: I would assume that if the following were provided:

{
  "value": 10,
  "unit": "px",
  "vendor": "foobar"
}

This would NOT be interpreted as px as-defined by the spec. Because if, say, the intent was that it’s parsed as a px dimension by tooling by default, but a particular tool wants to convert it differently, then that’s the case where $extensions should be used instead (all tooling operates in a standard way, but in one particular tool, additional metadata overrides behavior). So am I correct in understanding unit + vendor = a unique unit that has no relation to any other?

[drwpow]

But all that said, I’ll make this PR easier to review by NOT adding vendor here, and we could review that in a followup. I’ll restore the current behavior of only allowing px and rem and keeping those as currently-defined. Will also update the PR + description.

🤔 I think this still rejects #218 because that proposes numbers-only values.

[romainmenke]

So am I correct in understanding unit + vendor = a unique unit that has no relation to any other?

Yes :)

I’ll make this PR easier to review by NOT adding vendor here

I agree.

This will also give people the time to think about something like that and give feedback.

[c1rrus]

Thanks for opening this PR @drwpow. It looks good. I spotted a few minor inconsistencies though - see the other review comments.

However, I'm on the fence about whether or not dimension and duration values need to be changed to an object format. Especially since it's a breaking change (i.e. files and tools written the current draft spec will need to be updated if this goes in). Not a deal breaker since we're still in "editor's draft" territory, but also not something to take lightly given that there are a number of DTCG tools and files out there in the wild.

Making parsing easier is nice, but I'm not convinced the current string values are a significant blocker or source of bugs for implementors. The more significant improvement if this were to go in is more consistency within the DTCG format's syntax. Especially since color values are also due to change to an object structure in order to support more color spaces and wider gamuts. I think aligning all type's value syntaxes around some JSON-esque rather than having some encode multiple pieces of info into a string and others not.

As you've noted this does make things more verbose for humans wanting to read or hand-edit DTCG files. However, I think the proposed syntax is still understandable. Also, I guess this would make creating a JSON schema for the DTCG format easier, and thus also make it easier auto-complete suggestions in editors like VSCode. So on balance, I think it could be a worthwhile trade off.

Nonetheless, I'd like some other format editors to take a look and chime in with their thoughts before I approve this PR.

[c1rrus]

Btw, i agree with the earlier point from @romainmenke about keeping a defined set of allowed units. Without that I fear we'd quickly end up in a world of incompatible tools and files due different folks supporting different units, or interpreting the same unit in different ways.

I am intrigued and potentially in favour of adding support for custom units though. But, agree that should be done separate from this PR.

[drwpow]

Making parsing easier is nice, but I'm not convinced the current string values are a significant blocker or source of bugs for implementors.

I was of this opinion until rereading #121. There wasn’t a specific comment that touched on this, but comments like @jonathantneal’s reminded me that parsing can be trickier than it seems.

In most languages, validating it isn’t too bad (can be done with RegEx). But when you need to separate it into number and unit, it is trickier than it seems at first glance. In JS, for one, you have parseFloat() and parseInt(), and one may parse as an integer when it’s not. But then you also have BigInt! And don’t forget negative numbers!

For other languages (that aren’t JS) it gets harder. For example, in Rust, Serde is a popular library for interpreting JSON into Rust data structures and can automate a lot of the mapping and memory management. It gets trickier when you’re storing values that are delivered as strings, but they ultimately want to be numbers paired with enums. The difference goes from anyone using an off-the-shelf library to traverse tokens JSON, to having to deal with parsing issues. It’s within the realm of tooling authors, sure, but I think it raises the bar for most devs who haven’t dealt with this extensively.

And of course that’s a shared problem among all languages: storing those numbers and units somewhere. After parsing, now your tooling has to have additional scaffolding to store those saved numbers and units separately from the original token JSON, and keep the mapping between the two preserved.

I think in hindsight, offloading all that work onto tooling doesn’t make sense, when even CSS ASTs like csstree separate values & units so they can be validated independently. It works when we’re plugging tokens straight into CSS, but for any other platform and language, it requires a lot more work than I think we originally realized.

[kevinmpowell]

Any ideas on how this would support unitless line-height? Would the unit just be an empty string in that case?

[romainmenke]

Any ideas on how this would support unitless line-height? Would the unit just be an empty string in that case?

That is just a number, right?
Not a dimension.

I don't think it is needed to add dimensions with a "null unit" as a special form of number when regular numbers exist :)

[ChucKN0risK]

Thanks for creating this PR @drwpow 👍

I think splitting the value to an object format is the best solution. I still easy to read while it makes it easier to parse for other languages and tools as you explained.

I also think we should create a new PR to introduce the vendor property.

And finally I agree with @romainmenke about the fact we don't need to add dimensions with a "null unit". Unitless line height can be created as number tokens.

[SpikyC]

Suggested change

	- `$value.unit` is still requird even if `$value.value` is `0`.
	- `$value.unit` is still required even if `$value.value` is `0`.