Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

S3: API now also returns number of bytes processed #26

Merged
merged 1 commit into from
Dec 25, 2024

Conversation

bblommers
Copy link
Collaborator

Also simplifies the flow. The input is now processed one document at a time:

for document in source:
    # process FROM clause
    # check if WHERE clause is applicable
    # process SELECT clause
    add document

Which should make it much easier to add support for a LIMIT-clause later on.

Note that only the S3-part has been changed, the DynamoDB part is unchanged.

@bblommers bblommers force-pushed the s3-return-processed-bytes branch 3 times, most recently from 43ebd30 to 2e53ca1 Compare December 25, 2024 19:26
@bblommers bblommers force-pushed the s3-return-processed-bytes branch from 2e53ca1 to 78677ee Compare December 25, 2024 19:30
@bblommers bblommers merged commit 4be9ff7 into main Dec 25, 2024
14 checks passed
@bblommers bblommers deleted the s3-return-processed-bytes branch December 25, 2024 19:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant