feat: optimize custom selection set type

[iartemiev]

Draft pending team discussion

Issue #, if available:
aws-amplify/amplify-category-api#2592

Description of changes:

• Optimizes custom selection set path type, specifically for bi-directional relationships. Dramatically reduces the number of literal strings in the generated union by dropping redundant paths.
  In highly-interconnected schemas, such as the one in the linked issue, this resolves error TS2590: Expression produces a union type that is too complex to represent.
• Improves e2e benchmark performance. P50 and P99 CRUDL and selection-set benches are improved by ~30-60%. All other benches are either completely unaffected or minor impact of +/- ~1%. See output section at the bottom of the PR description for exhaustive diff.

Basic example

const schema = a.schema({
  Post: a.model({
    title: a.string().required(),
    description: a.string(),
    comments: a.hasMany('Comment', 'postId'),
  }),
  Comment: a.model({
    content: a.string().required(),
    postId: a.id(),
    post: a.belongsTo('Post', 'postId'),
  }),
});

Would result in a selectionSet path type allowing bi-directional traversal 6 levels deep, e.g.,
client.models.Post.list({ selectionSet: ['id', 'comments.post.comments.post.comments.post.id']})
This is redundant, as going back between parent and child in this fashion will return the same data (both id values in the example will be the same) and leads to a combinatorial explosion of the generated literal string union type.

With the changes in this PR, we short-circuit this path for belongsTo relational fields on the child that reference the parent model. E.g. for the schema above, the generated paths are (identifier and system fields omitted for brevity, but they're selectable as well):

title | description | comments.* | comments.content | comments.postId

Note: deeply-nested relationships retain the 6-level depth limit. See example in this integ test.

`npm run bench:all` output

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Benchmarking: ./comparisons/scratch.bench.ts ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[email protected] tsx
tsx ./comparisons/scratch.bench.ts

🏌️ large string union as Record key
⛳ Result: 4 instantiations
🎯 Baseline: 4 instantiations
📊 Delta: 0.00%

🏌️ large string union discrimination
⛳ Result: 20031 instantiations
🎯 Baseline: 20031 instantiations
📊 Delta: 0.00%

🏌️ super basic sanity check - simple object prop @ 1
⛳ Result: 0 instantiations
🎯 Baseline: 0 instantiations
📊 Delta: NaN%

🏌️ super basic sanity check - simple object prop @ 2
⛳ Result: 0 instantiations
🎯 Baseline: 0 instantiations
📊 Delta: NaN%

🏌️ super basic sanity check - simple object prop @ 4
⛳ Result: 0 instantiations
🎯 Baseline: 0 instantiations
📊 Delta: NaN%

🏌️ basic case - tuple -> union extract
⛳ Result: 52 instantiations
🎯 Baseline: 52 instantiations
📊 Delta: 0.00%

🏌️ basic case - tuple -> UnionToIntersection
⛳ Result: 161 instantiations
🎯 Baseline: 161 instantiations
📊 Delta: 0.00%

🏌️ basic case - Tuple remapped to intersection
⛳ Result: 1990 instantiations
🎯 Baseline: 1990 instantiations
📊 Delta: 0.00%

🏌️ as augmentation - union extract @ 1
⛳ Result: 283 instantiations
🎯 Baseline: 283 instantiations
📊 Delta: 0.00%

🏌️ as augmentation - union extract @ 2
⛳ Result: 417 instantiations
🎯 Baseline: 417 instantiations
📊 Delta: 0.00%

🏌️ as augmentation - union extract @ 4
⛳ Result: 685 instantiations
🎯 Baseline: 685 instantiations
📊 Delta: 0.00%

🏌️ as augmentation - union extract - util @ 1
⛳ Result: 254 instantiations
🎯 Baseline: 254 instantiations
📊 Delta: 0.00%

🏌️ as augmentation - union extract - util @ 2
⛳ Result: 301 instantiations
🎯 Baseline: 301 instantiations
📊 Delta: 0.00%

🏌️ as augmentation - union extract - util @ 4
⛳ Result: 395 instantiations
🎯 Baseline: 395 instantiations
📊 Delta: 0.00%

🏌️ as augmentation - UnionToIntersection @ 1
⛳ Result: 180 instantiations
🎯 Baseline: 180 instantiations
📊 Delta: 0.00%

🏌️ as augmentation - UnionToIntersection @ 2
⛳ Result: 199 instantiations
🎯 Baseline: 199 instantiations
📊 Delta: 0.00%

🏌️ as augmentation - UnionToIntersection @ 4
⛳ Result: 237 instantiations
🎯 Baseline: 237 instantiations
📊 Delta: 0.00%

🏌️ as augmentation - UnionToIntersection - util @ 1
⛳ Result: 250 instantiations
🎯 Baseline: 250 instantiations
📊 Delta: 0.00%

🏌️ as augmentation - UnionToIntersection - util @ 2
⛳ Result: 271 instantiations
🎯 Baseline: 271 instantiations
📊 Delta: 0.00%

🏌️ as augmentation - UnionToIntersection - util @ 4
⛳ Result: 313 instantiations
🎯 Baseline: 313 instantiations
📊 Delta: 0.00%

🏌️ as augmentation - UnionToIntersection - util @ 1
⛳ Result: 267 instantiations
🎯 Baseline: 267 instantiations
📊 Delta: 0.00%

🏌️ as augmentation - UnionToIntersection - util @ 2
⛳ Result: 294 instantiations
🎯 Baseline: 294 instantiations
📊 Delta: 0.00%

🏌️ as augmentation - UnionToIntersection - util @ 4
⛳ Result: 348 instantiations
🎯 Baseline: 348 instantiations
📊 Delta: 0.00%

Benchmarking: ./combine/combine-CRUDL.bench.ts

[email protected] tsx
tsx ./combine/combine-CRUDL.bench.ts

🏌️ combined SQL and DDB schema w client types
⛳ Result: 2133852 instantiations
🎯 Baseline: 2128533 instantiations
📊 Delta: +0.25%

Benchmarking: ./p50/p50-prod.bench.ts

[email protected] tsx
tsx ./p50/p50-prod.bench.ts

🏌️ prod p50
⛳ Result: 20835 instantiations
🎯 Baseline: 20835 instantiations
📊 Delta: 0.00%

🏌️ prod p50 w/ client types
⛳ Result: 22777 instantiations
🎯 Baseline: 22777 instantiations
📊 Delta: 0.00%

🏌️ prod p50 combined w/ client types
⛳ Result: 26955 instantiations
🎯 Baseline: 26955 instantiations
📊 Delta: 0.00%

Benchmarking: ./p50/operations/p50-selection-set.bench.ts

[email protected] tsx
tsx ./p50/operations/p50-selection-set.bench.ts

🏌️ p50 CRUDL
⛳ Result: 274049 instantiations
🎯 Baseline: 453520 instantiations
📉 p50 CRUDL was under baseline by 39.57%! Consider setting a new baseline.

Benchmarking: ./p50/operations/p50-prod-selection-set.bench.ts

[email protected] tsx
tsx ./p50/operations/p50-prod-selection-set.bench.ts

🏌️ prod p50 CRUDL
⛳ Result: 403680 instantiations
🎯 Baseline: 773656 instantiations
📉 prod p50 CRUDL was under baseline by 47.82%! Consider setting a new baseline.

Benchmarking: ./p50/operations/p50-prod-CRUDL.bench.ts

[email protected] tsx
tsx ./p50/operations/p50-prod-CRUDL.bench.ts

🏌️ prod p50 CRUDL
⛳ Result: 389477 instantiations
🎯 Baseline: 751338 instantiations
📉 prod p50 CRUDL was under baseline by 48.16%! Consider setting a new baseline.

Benchmarking: ./p50/operations/p50-CRUDL.bench.ts

[email protected] tsx
tsx ./p50/operations/p50-CRUDL.bench.ts

🏌️ p50 CRUDL
⛳ Result: 260129 instantiations
🎯 Baseline: 431534 instantiations
📉 p50 CRUDL was under baseline by 39.72%! Consider setting a new baseline.

Benchmarking: ./p50/p50.bench.ts

[email protected] tsx
tsx ./p50/p50.bench.ts

🏌️ p50
⛳ Result: 8512 instantiations
🎯 Baseline: 8512 instantiations
📊 Delta: 0.00%

🏌️ p50 w/ client types
⛳ Result: 10439 instantiations
🎯 Baseline: 10439 instantiations
📊 Delta: 0.00%

🏌️ p50 combined schema w/ client types
⛳ Result: 13849 instantiations
🎯 Baseline: 13849 instantiations
📊 Delta: 0.00%

Benchmarking: ./basic/custom-operations.bench.ts

[email protected] tsx
tsx ./basic/custom-operations.bench.ts

🏌️ custom op returning primitive types; schema only
⛳ Result: 10454 instantiations
🎯 Baseline: 10454 instantiations
📊 Delta: 0.00%

🏌️ custom op returning primitive types; schema + ClientSchema
⛳ Result: 12328 instantiations
🎯 Baseline: 12328 instantiations
📊 Delta: 0.00%

🏌️ custom op returning primitive types; schema + ClientSchema + client types
⛳ Result: 12605 instantiations
🎯 Baseline: 12605 instantiations
📊 Delta: 0.00%

🏌️ custom op returning an enum; schema only
⛳ Result: 10653 instantiations
🎯 Baseline: 10653 instantiations
📊 Delta: 0.00%

🏌️ custom op returning an enum; schema + ClientSchema
⛳ Result: 12526 instantiations
🎯 Baseline: 12526 instantiations
📊 Delta: 0.00%

🏌️ custom op returning an enum; schema + ClientSchema + client types
⛳ Result: 12821 instantiations
🎯 Baseline: 12821 instantiations
📊 Delta: 0.00%

🏌️ custom op returning custom type; schema only
⛳ Result: 10692 instantiations
🎯 Baseline: 10692 instantiations
📊 Delta: 0.00%

🏌️ custom op returning custom type; schema + ClientSchema
⛳ Result: 12565 instantiations
🎯 Baseline: 12565 instantiations
📊 Delta: 0.00%

🏌️ custom op returning custom type; schema + ClientSchema + client types
⛳ Result: 12860 instantiations
🎯 Baseline: 12860 instantiations
📊 Delta: 0.00%

🏌️ custom op returning model; schema only
⛳ Result: 10770 instantiations
🎯 Baseline: 10770 instantiations
📊 Delta: 0.00%

🏌️ custom op returning model; schema + ClientSchema
⛳ Result: 12643 instantiations
🎯 Baseline: 12643 instantiations
📊 Delta: 0.00%

🏌️ custom op returning model; schema + ClientSchema + client types
⛳ Result: 12938 instantiations
🎯 Baseline: 12938 instantiations
📊 Delta: 0.00%

Benchmarking: ./basic/basic-schema.bench.ts

[email protected] tsx
tsx ./basic/basic-schema.bench.ts

🏌️ basic schema
⛳ Result: 2405 instantiations
🎯 Baseline: 2405 instantiations
📊 Delta: 0.00%

🏌️ basic schema w client types
⛳ Result: 4622 instantiations
🎯 Baseline: 4622 instantiations
📊 Delta: 0.00%

Benchmarking: ./basic/secondaryIndexes.bench.ts

[email protected] tsx
tsx ./basic/secondaryIndexes.bench.ts

🏌️ secondary index of model has only ModelFields
⛳ Result: 16703 instantiations
🎯 Baseline: 16703 instantiations
📊 Delta: 0.00%

🏌️ secondary index without using enum field of a model that has enum field
⛳ Result: 15203 instantiations
🎯 Baseline: 15203 instantiations
📊 Delta: 0.00%

🏌️ secondary index with using enum field of a model that has enum field
⛳ Result: 15203 instantiations
🎯 Baseline: 15203 instantiations
📊 Delta: 0.00%

🏌️ secondary index without using enum field of a model that has enum ref field
⛳ Result: 17447 instantiations
🎯 Baseline: 17447 instantiations
📊 Delta: 0.00%

🏌️ secondary index with using enum field of a model that has enum ref field
⛳ Result: 17447 instantiations
🎯 Baseline: 17447 instantiations
📊 Delta: 0.00%

Benchmarking: ./p99/over-limit/operations/p99-complex-relationships.bench.ts

[email protected] tsx
tsx ./p99/over-limit/operations/p99-complex-relationships.bench.ts

🏌️ complex relationships real world CRUDL
⛳ Result: 40994 instantiations
🎯 Baseline: 40994 instantiations
📊 Delta: 0.00%

Benchmarking: ./p99/within-limit/p99-complex-sql.bench.ts

[email protected] tsx
tsx ./p99/within-limit/p99-complex-sql.bench.ts

🏌️ complex SQL
⛳ Result: 41814 instantiations
🎯 Baseline: 41814 instantiations
📊 Delta: 0.00%

Benchmarking: ./p99/within-limit/p99-tall-simple.bench.ts

[email protected] tsx
tsx ./p99/within-limit/p99-tall-simple.bench.ts

🏌️ 100 simple models with 1 field each
⛳ Result: 4048 instantiations
🎯 Baseline: 4048 instantiations
📊 Delta: 0.00%

🏌️ 100 simple models with 1 field each w/ client types
⛳ Result: 6064 instantiations
🎯 Baseline: 6064 instantiations
📊 Delta: 0.00%

Benchmarking: ./p99/within-limit/p99-wide-large.bench.ts

[email protected] tsx
tsx ./p99/within-limit/p99-wide-large.bench.ts

🏌️ 26 models w/ 215 fields each, 1 model with 4
⛳ Result: 3328 instantiations
🎯 Baseline: 3328 instantiations
📊 Delta: 0.00%

🏌️ 26 models w/ 215 fields each, 1 model with 4 w/ client types
⛳ Result: 5344 instantiations
🎯 Baseline: 5344 instantiations
📊 Delta: 0.00%

Benchmarking: ./p99/within-limit/p99-very-tall-simple.bench.ts

[email protected] tsx
tsx ./p99/within-limit/p99-very-tall-simple.bench.ts

🏌️ 1522 simple models with 1 field each
⛳ Result: 25753 instantiations
🎯 Baseline: 25753 instantiations
📊 Delta: 0.00%

🏌️ 1522 simple models with 1 field each w/ client types
⛳ Result: 27769 instantiations
🎯 Baseline: 27769 instantiations
📊 Delta: 0.00%

Benchmarking: ./p99/within-limit/operations/p99-very-tall-simple-CRUDL.bench.ts

[email protected] tsx
tsx ./p99/within-limit/operations/p99-very-tall-simple-CRUDL.bench.ts

🏌️ baseline
⛳ Result: 0 instantiations
🎯 Baseline: 0 instantiations
📊 Delta: NaN%

🏌️ 1522 simple models with 1 field each CRUDL
⛳ Result: 712884 instantiations
🎯 Baseline: 709359 instantiations
📊 Delta: +0.50%

Benchmarking: ./p99/within-limit/operations/p99-complex-sql-CRUDL.bench.ts

[email protected] tsx
tsx ./p99/within-limit/operations/p99-complex-sql-CRUDL.bench.ts

🏌️ complex SQL
⛳ Result: 3067498 instantiations
🎯 Baseline: 7942619 instantiations
📉 complex SQL was under baseline by 61.38%! Consider setting a new baseline.

Benchmarking: ./p99/within-limit/operations/p99-tall-simple-CRUDL.bench.ts

[email protected] tsx
tsx ./p99/within-limit/operations/p99-tall-simple-CRUDL.bench.ts

🏌️ baseline
⛳ Result: 0 instantiations
🎯 Baseline: 0 instantiations
📊 Delta: NaN%

🏌️ 70 simple models with 1 field each w/ client types
⛳ Result: 159672 instantiations
🎯 Baseline: 156147 instantiations
📊 Delta: +2.26%

Benchmarking: ./p99/within-limit/operations/p99-very-wide-large-CRUDL.bench.ts

[email protected] tsx
tsx ./p99/within-limit/operations/p99-very-wide-large-CRUDL.bench.ts

🏌️ baseline
⛳ Result: 0 instantiations
🎯 Baseline: 0 instantiations
📊 Delta: NaN%

🏌️ 1 model containing 2288 fields, 34 models w/ 215 fields each CRUDL
⛳ Result: 3652623 instantiations
🎯 Baseline: 3647386 instantiations
📊 Delta: +0.14%

Benchmarking: ./p99/within-limit/operations/p99-wide-small-selection-set.bench.ts

[email protected] tsx
tsx ./p99/within-limit/operations/p99-wide-small-selection-set.bench.ts

🏌️ 1 simple model w/ 43 fields CRUDL
⛳ Result: 870418 instantiations
🎯 Baseline: 884652 instantiations
📊 Delta: -1.61%

Benchmarking: ./p99/within-limit/operations/p99-tall-complex-CRUDL.bench.ts

[email protected] tsx
tsx ./p99/within-limit/operations/p99-tall-complex-CRUDL.bench.ts

🏌️ baseline
⛳ Result: 0 instantiations
🎯 Baseline: 0 instantiations
📊 Delta: NaN%

🏌️ 99 complex models CRUDL
⛳ Result: 396043 instantiations
🎯 Baseline: 574151 instantiations
📉 99 complex models CRUDL was under baseline by 31.02%! Consider setting a new baseline.

Benchmarking: ./p99/within-limit/operations/p99-tall-simple-selection-set.bench.ts

[email protected] tsx
tsx ./p99/within-limit/operations/p99-tall-simple-selection-set.bench.ts

🏌️ baseline
⛳ Result: 0 instantiations
🎯 Baseline: 0 instantiations
📊 Delta: NaN%

🏌️ 70 simple models with 1 field each w/ client types
⛳ Result: 171521 instantiations
🎯 Baseline: 174835 instantiations
📊 Delta: -1.90%

Benchmarking: ./p99/within-limit/operations/p99-wide-small-CRUDL.bench.ts

[email protected] tsx
tsx ./p99/within-limit/operations/p99-wide-small-CRUDL.bench.ts

🏌️ 1 simple model w/ 43 fields CRUDL
⛳ Result: 836322 instantiations
🎯 Baseline: 832461 instantiations
📊 Delta: +0.46%

Benchmarking: ./p99/within-limit/p99-very-wide-large.bench.ts

[email protected] tsx
tsx ./p99/within-limit/p99-very-wide-large.bench.ts

🏌️ 1 model containing 2288 fields, 34 models w/ 215 fields each
⛳ Result: 3478 instantiations
🎯 Baseline: 3478 instantiations
📊 Delta: 0.00%

🏌️ 1 model containing 2288 fields, 34 models w/ 215 fields each w/ client types
⛳ Result: 5494 instantiations
🎯 Baseline: 5494 instantiations
📊 Delta: 0.00%

Benchmarking: ./p99/within-limit/p99-wide-small.bench.ts

[email protected] tsx
tsx ./p99/within-limit/p99-wide-small.bench.ts

🏌️ 1 simple model w/ 1700 fields each
⛳ Result: 2938 instantiations
🎯 Baseline: 2938 instantiations
📊 Delta: 0.00%

🏌️ 1 simple model w/ 1700 fields each w/ client types
⛳ Result: 4954 instantiations
🎯 Baseline: 4954 instantiations
📊 Delta: 0.00%

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[changeset-bot]

🦋 Changeset detected

Latest commit: 106a5fe

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
@aws-amplify/data-schema	Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[iartemiev]

In linked issue we would see a TS error here due to the selection set type union length