Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove copy/replicate mode and focus on cloud proxy in replicator extension #81

Merged
merged 2 commits into from
Sep 30, 2024
Merged

Remove copy/replicate mode and focus on cloud proxy in replicator extension #81

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f5da821
remove "copy" mode and focus on cloud proxy in replicator extension
whummer Sep 23, 2024
4d93f85
add reference to last commit of old replicator CLI
whummer Sep 30, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 15 additions & 41 deletions aws-replicator/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,28 +1,28 @@
AWS Replicator Extension (experimental)
AWS Cloud Proxy Extension (experimental)
========================================
[![Install LocalStack Extension](https://localstack.cloud/gh/extension-badge.svg)](https://app.localstack.cloud/extensions/remote?url=git+https://github.com/localstack/localstack-extensions/#egg=localstack-extension-aws-replicator&subdirectory=aws-replicator)

A LocalStack extension to replicate AWS resources into your local machine.
A LocalStack extension to proxy and integrate AWS resources into your local machine.
This enables one flavor of "hybrid" or "remocal" setups where you can easily bridge the gap between LocalStack (local resources) and remote AWS (resources in the real cloud).

⚠️ Please note that this extension is experimental and currently under active development.
⚠️ Please note that this extension is experimental and still under active development.

⚠️ Note: Given that the scope of this extension has recently changed (see [below](#resource-replicator-cli-deprecated)), it may get renamed from `aws-replicator` to `cloud-proxy` in an upcoming release.

## Prerequisites

* LocalStack Pro
* Docker
* Python

## Overview

This extension currently offers two modes of operation: (1) the AWS connection proxy, and (2) the resource replicator CLI.

## AWS Connection Proxy
## AWS Cloud Proxy

The AWS connection proxy can be used to forward certain API calls in LocalStack to real AWS, in order to enable seamless transition between local and remote resources.
The AWS Cloud Proxy can be used to forward certain API calls in LocalStack to real AWS, in order to enable seamless transition between local and remote resources.

**Warning:** Be careful when using the proxy - make sure to _never_ give access to production accounts or any critical/sensitive data!

**Note:** The replicator CLI currently works only when installing the `localstack` CLI via `pip`. If you're downloading the `localstack` CLI as a [binary release](https://docs.localstack.cloud/getting-started/installation/#localstack-cli), then please use the proxy configuration UI described below.
**Note:** The Cloud Proxy CLI currently works only when installing the `localstack` CLI via `pip`.
If you're downloading the `localstack` CLI as a [binary release](https://docs.localstack.cloud/getting-started/installation/#localstack-cli), then please use the proxy configuration UI described below.

### Usage

Expand Down Expand Up @@ -66,7 +66,7 @@ EXTRA_CORS_ALLOWED_ORIGINS=https://aws-replicator.localhost.localstack.cloud:456

5. Now we can communicate with the real AWS cloud resources, directly via LocalStack.

To clean up the running proxy container simply click "disable" on the Replicator UI.
To clean up the running proxy container simply click "disable" on the Cloud Proxy UI.

### Resource-specific proxying

Expand Down Expand Up @@ -117,38 +117,12 @@ In addition to the proxy services configuration shown above, the following confi

**Note:** Due to some recent changes in the core framework, make sure to start up your LocalStack container with the `GATEWAY_SERVER=hypercorn` configuration enabled, for backwards compatibility. This will be fixed in an upcoming release.

## Resource Replicator CLI

The figure below illustrates how the extension can be used to replicate the state, e.g., an SQS queue and the messages contained in it, from AWS into your LocalStack instance.

![overview](etc/aws-replicate-overview.png)
## Resource Replicator CLI (deprecated)

To use the resource replicator, make sure that you have access to AWS configured in your terminal. Note: the extension will only talk to AWS in read-only mode, and will **not** make any changes to your real AWS account.
Note: Previous versions of this extension also offered a "replicate" mode to copy/clone (rather than proxy) resources from an AWS account into the local instance.
This functionality has been removed from this extension, and is now being migrated to a new extension (more details following soon).

The following command can be used to replicate SQS queues (incl. their messages) into your LocalStack instance:
```
$ localstack aws replicate -s sqs
```

Once the command has completed, you should be able to list and interact with the queue that was replicated into your local account:
```
$ awslocal sqs list-queues
...
$ awslocal sqs receive-message --queue-url ...
...
```

## Installing

To install the CLI extension, use the following `pip` command:
```bash
pip install "git+https://github.com/localstack/localstack-extensions/#egg=localstack-extension-aws-replicator&subdirectory=aws-replicator"
```

To install the extension itself (server component running inside LocalStack), use the following `extensions` command:
```bash
localstack extensions install "git+https://github.com/localstack/localstack-extensions/#egg=localstack-extension-aws-replicator&subdirectory=aws-replicator"
```
If you wish to access the deprecated instructions, they can be found [here](https://github.com/localstack/localstack-extensions/blob/fe0c97e8a9d94f72c80358493e51ce6c1da535dc/aws-replicator/README.md#resource-replicator-cli).

## Change Log

Expand Down
2 changes: 0 additions & 2 deletions aws-replicator/aws_replicator/server/ui/app.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -112,8 +112,6 @@ const App = () => {
</TableContainer>
<Button onClick={saveConfig} variant="contained" disabled={isLoading}>{isLoading ? 'loading ...' : 'Save configuration'}</Button>
</div>
<h2>AWS Resource Replicator</h2>
Configuration switches coming soon ...
</div>);
}

Expand Down
8 changes: 1 addition & 7 deletions aws-replicator/aws_replicator/server/ui/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,17 +15,11 @@
</head>
<body>
<div>
<h1>AWS Replicator - LocalStack Extension</h1>
<h1>AWS Cloud Proxy - LocalStack Extension</h1>
<p>
This extension allows you to <b>mirror resources from real AWS accounts into your LocalStack instance</b>, thereby "bridging the gap" between local and remote cloud resources.
</p><p>
Some <b>common use cases are</b>: developing a local Lambda function that accesses a remote DynamoDB table; running a local Athena SQL query in LocalStack accessing files in a real S3 bucket in AWS; seeding a local Terraform script with SSM parameters from a real AWS account
</p><p>
The extension currently offers <b>two modes of operation</b>:
<ol>
<li>AWS Connection Proxy</li>
<li>AWS Resource Replicator</li>
</ol>
</p>
<div id="root"></div>
</div>
Expand Down
Loading