deterministic order on query and filter on gsi

[Polandia94]

We need a deterministic order on scan and query operations on GSI.

With this two tables created the same, and added the same elements will have always the same order on GSI.

If we have a table:

GSI_PK | GSI_SK | PK
"a" | "a" | "a"
"a" | "a" | "b"

And we do a scan on gsi, we get two items.

If we do a new scan with ExclusiveStartKey with the key of the first item, now we will be sure that will return only one item. Previously the order to calculate exclusiveStartKey and to return on scan/query were different, now, are the same.

Also, on scan on GSI with the same PK and different SK, the order is correct, previously was using primarykey of table to order

[Polandia94]

@bblommers I think the test failed because a 503 on httpbin, not related to this PR

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 94.59%. Comparing base (361b4fc) to head (9a70e16).
Report is 1 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #8376   +/-   ##
=======================================
  Coverage   94.59%   94.59%           
=======================================
  Files        1159     1159           
  Lines      101415   101422    +7     
=======================================
+ Hits        95932    95941    +9     
+ Misses       5483     5481    -2     

Flag	Coverage Δ
servertests	28.86% <12.50%> (-0.01%)	⬇️
unittests	94.56% <100.00%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[bblommers]

Hi @Polandia94! Can you add a non-trivial test that shows the problem?

With a test it would be easier to see what AWS is doing, and then determine what the correct approach/fix is.

In an ideal world we always mimick AWS, but they do not guarantee order on scan, so the exact order that Moto returns should not matter (as long as it is deterministic).

We have quite a few (AWS verified) tests already for various scenarios, if you want some inspiration:
https://github.com/getmoto/moto/blob/master/tests/test_dynamodb/test_dynamodb_scan.py