Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: add more instructions for Pitr readme #446

Merged
merged 1 commit into from
Sep 24, 2023
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
98 changes: 69 additions & 29 deletions pitr/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,24 @@ This is a cli tool for point-in-time recovery of Apache ShardingSphere and OpenG

## Prerequisition

Before you start, you need to prepare at least three servers, set the running environment and deploy required softwares respectively. The topology is:


```shell
+------------------------------+ +------------------+
| | | OpenGauss Server |
| Apache ShardingSphere Proxy | | Pitr Agent |
| Apache Zookeeper | ----------> +------------------+
| Pitr Cli (aka `gs_pitr`) | | OpenGauss Server |
| | | Pitr Agent |
+------------------------------+ +------------------+
```


### Servers

You need to prepare at least three servers: one server for the Pitr commandline tool and Apache ShardingSphere, other two servers for the Pitr agent and OpenGauss:

| | Role | Components |
|:-:|:-:|:-:|
|1| Pitr cli operation server | Pitr Cli + ShardingSphere Proxy + Zookeeper|
Expand All @@ -14,18 +30,25 @@ This is a cli tool for point-in-time recovery of Apache ShardingSphere and OpenG

### Environment

- Apache ShardingSphere Proxy can access OpenGauss network
After the servers are ready, you should check and ensure the following items:

- Apache ShardingSphere Proxy is allowed to access OpenGauss Servers
- External access to Apache ShardingSphere Proxy
- External access to OpenGauss Server via port 18080
- Set below environment variables on OpenGauss Servers
- export PGDATABASE=13100
- export PGPORT=tpccdb
- OpenGauss has user `omm` and database `omm` which can be accessed
- OpenGauss enables `cbm tracking`
- SSL key pairs for Pitr cli-agent secure communication
- SSL key pairs. Any valid key pairs are acceptable, they will be used for Pitr cli-agent secure communication

#### Compilation (optional)

If you want to compile Pitr tools yourself, you should using a recommanded Golang version 1.20 with Linux 3.10.0-957.el7.x86_64. Following the steps below to compile both Pitr agent and cli.
Generally the Pitr command line tools, including cli binary and agent binary could be downloaded throught the [Apache ShardingSphere-on-Cloud release page](https://github.com/apache/shardingsphere-on-cloud/releases).

In case of if you want to compile Pitr tools yourself, you should using this recommanded Golang version 1.20 with Linux 3.10.0-957.el7.x86_64. Please follow the steps below to compile both Pitr agent and cli.

Step 1. Firstly clone the project
Step 1. Clone the project

```shell
git clone [email protected]:apache/shardingsphere-on-cloud.git
Expand All @@ -52,28 +75,48 @@ The communication of Pitr cli and Pitr agent is secured by TLS which needs a SSL
- tls.key
- tls.crt

The key pair need to be deployed on the servers where Pitr agent and OpenGauss are installed.

#### Generate new TLS keypair (Optional)

Make sure you have a usable OpenSSL environment, check environment variable OPENSSL_CONF, generally it is set to `/etc/pki/tls`.
If you want to generate a new key pair, please make sure you have a available OpenSSL environment, check environment variable OPENSSL_CONF, generally it is set to `/etc/pki/tls`.

Then using the script under Pitr agent code directory, execute the commands below:

```shell
git clone [email protected]:apache/shardingsphere-on-cloud.git
cd shardingsphere-on-cloud/pitr/agent
make openssl-local
```

After that, the keypair files will be write to `./certs` in the current directory.
After that, the keypair files will be write to `./certs` in the current directory.


## Deployment

Pitr cli (aka `gs_pitr`) and Pitr agent (aka `pitr-agent`) binaries could be downloaded at Apache ShardingSphere on Cloud release page, or just compiled in your local development environment.
Pitr cli (aka `gs_pitr`) and Pitr agent (aka `pitr-agent`) binaries could be downloaded at [Apache ShardingSphere on Cloud release page](https://github.com/apache/shardingsphere-on-cloud/releases), or just compiled in your local development environment according the previous instructions.

### Step 1: Get ShardingSphere Proxy Configurations
The whole deployment process consists of two parts:

Substitute the OpenGauss server address below:
1. Deploying Apache ShardingSphere Proxy, Zookeeper and Pitr Cli, refering to step 1 - step 2
2. Deploying OpenGauss and Pitr Agent, refering to step 3 - step 5.

### Step 1: Get Pitr tools

You can download pre-compiled Pitr tools binary release or compile them yourself from source code.

#### Get binary release

The binaries are packaged as .tar.gz file on [release page](https://github.com/apache/shardingsphere-on-cloud/releases). You can download expected version and uncompress the binary files `gs_pitr` and `pitr-agent`.

#### Compile it yourself

Please refer to the `Compilation` section in `Prerequsition` for detailed instructions.

After fetching the binaries successfully. You need to save the `gs_pitr` to the same server where Apache ShardingSphere Proxy is located. And save `pitr-agent` to the servers where OpenGauss is deployed.

### Step 2: Get ShardingSphere Proxy Configurations

Using the OpenGauss host to substitute the ${OPENGAUSS_SERVER_1} and ${OPENGAUSS_SERVER_2} below:

server.yaml

Expand Down Expand Up @@ -141,29 +184,23 @@ dataSources:
minPoolSize: 1
```

### Step 2: Get OpenGauss Configurations
And using the script `bin/start.sh` to start ShardingSphere Proxy. This script could be found in apache-shardingsphere-{version}-shardingsphere-proxy-bin.tar.gz

### Step 3: Set OpenGauss Configurations

Enable `cbm tracking` in postgres.conf
a. Enable `cbm tracking` in postgres.conf

```shell
enable_cbm_tracking = on
```
b. Execute `gs_probackup init -B ${backup-path}` to set the expected backup path.

### Step 3: Get Pitr tools

You can download pre-compiled Pitr tools binary release or compile them yourself from source code.

#### Get binary release

The binaries are packaged as .tar.gz file on [release page](https://github.com/apache/shardingsphere-on-cloud/releases). You can download expected version and uncompress the binary files `gs_pitr` and `pitr-agent`.
Then you can start both OpenGauss servers.

#### Compile it yourself

Please refer to the `Compilation` section in `Prerequsition` for detailed instructions.
### Step 4: Deploy SSL certs for Pitr Agent

### Step 4: Deploy Pitr Agent

1. Copy cert files
Before you start Pitr agent, you need to deploy SSL certs for Pitr agent:

If the TLS keypair is compiled yourself, the cert files are located at `shardingsphere-on-cloud/pitr/agent/certs`. You should change directory to the cert directory before executing the command below:

Expand All @@ -172,7 +209,11 @@ scp tls.crt tls.key root@${OPENGAUSS_SERVER_1}:/home/omm/
scp tls.crt tls.key root@${OPENGAUSS_SERVER_2}:/home/omm/
```

2. Copy binary files
Otherwise the key pairs need to be deployed to the same path on OpenGauss servers.

### Step 5: Start Pitr Agent

1. Copy binary files

```shell
cd shardingsphere-on-cloud/pitr/agent
Expand All @@ -181,9 +222,7 @@ scp pitr-agent root@${OPENGAUSS_SERVER_1}:/home/omm/
scp pitr-agent root@${OPENGAUSS_SERVER_2}:/home/omm/
```

### Step 5: Start Pitr Agent

1. Login OpenGauss servers and change directory to `/home/omm`
2. Login OpenGauss servers and change directory to `/home/omm`

Here are files under `/home/omm`:

Expand All @@ -197,7 +236,7 @@ drwx------ 29 omm omm 4.0K May 23 11:37 pgdata
-rwxr-xr-x 1 root root 1.7K May 16 18:26 tls.key
```

2. Start Pitr agent
3. Start Pitr agent

```shell
./pitr-agent -pgdata /data/data-glt/d1 -port 18080 -tls-crt tls.crt -tls-key tls.key -log-level debug
Expand Down Expand Up @@ -315,3 +354,4 @@ Verify data:
```SQL
select * from t_user;
```

Loading