Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Catch regex config errors in AWS proxy; fix logic for proxying S3 requests with host-based addressing #80

Merged
merged 5 commits into from
Sep 23, 2024

Conversation

whummer
Copy link
Member

@whummer whummer commented Sep 23, 2024

Adjust regex logic to catch config errors in AWS proxy.

Addresses an issue where we've recently seen errors like this:

  File "/var/lib/localstack/lib/extensions/python_venv/lib/python3.11/site-packages/aws_replicator/server/aws_request_forwarder.py", line 74, in select_proxy
    resource_name_matches = any(
                            ^^^^
  File "/var/lib/localstack/lib/extensions/python_venv/lib/python3.11/site-packages/aws_replicator/server/aws_request_forwarder.py", line 75, in <genexpr>
    self._request_matches_resource(context, resource_name_pattern)
  File "/var/lib/localstack/lib/extensions/python_venv/lib/python3.11/site-packages/aws_replicator/server/aws_request_forwarder.py", line 105, in _request_matches_resource
    return bool(re.match(resource_name_pattern, s3_bucket_arn))
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/re/__init__.py", line 166, in match
    return _compile(pattern, flags).match(string)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/re/__init__.py", line 294, in _compile
    p = _compiler.compile(pattern, flags)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/re/_compiler.py", line 745, in compile
    p = _parser.parse(p, flags)
        ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/re/_parser.py", line 989, in parse
    p = _parse_sub(source, state, flags & SRE_FLAG_VERBOSE, 0)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/re/_parser.py", line 464, in _parse_sub
    itemsappend(_parse(source, state, verbose, nested + 1,
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/re/_parser.py", line 691, in _parse
    raise source.error("nothing to repeat",
re.error: nothing to repeat at position 0

We are not changing the logic per se in here, but at least make the error message more human-readable.


The PR also enhances the logic for proxying S3 requests with host-based addressing. This is required for AWS SDK S3 clients that are using host-based bucket addressing (e.g., AWS JavaScript SDK v2/v3 by default). We are now passing a x-ls-host-original from the original request handler to the proxy handler, and then extract the bucket name from that header value, to form the final request path (/<bucket>/<path>).

The tests have been extended with additional parametrization to cover this case (host-based as well as path-based bucket addressing) as well.

@whummer whummer changed the title Adjust regex logic to catch config errors in AWS proxy Catch regex config errors in AWS proxy; fix logic for proxying S3 requests with host-based addressing Sep 23, 2024
@whummer whummer marked this pull request as ready for review September 23, 2024 23:27
@whummer whummer merged commit fe0c97e into main Sep 23, 2024
1 check passed
@whummer whummer deleted the catch-regex-error branch September 23, 2024 23:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant