Migration from AWS cloud to Azure cloud subscription fails with data loss

[celinaT]

Steps to reproduce:

1. Follow the migration guide as per the official documentation.
2. Compare the size of the tables in the source and target databases after migration.
3. Observe the data loss (30-40% missing).

Expected behaviour:

I have been following the official migration guide for InfluxDB v2 as outlined (https://docs.influxdata.com/influxdb/v2/write-data/migrate-data/migrate-cloud-to-oss/). I attempted the migration twice, using both methods provided in the documentation.

Although I did not encounter any migration errors during the process, I observed that the data was not fully migrated. Specifically, when comparing the table sizes between the source and target databases, I found that 30-40% of the data was missing in the target. There were no duplicates present in the source table, so that does not explain the discrepancy.

Actual behaviour:

Huge data losses in both cases, no errors in the logs

Environment info:

• Source bucket in the AWS cloud
• Destination bucket in the Azure cloud

Config:

• start: 2023-12-01T00:00:00Z,
• stop: 2024-08-31T00:00:00Z,
• batchInterval: 1h,
• option task = {
  name: "Migration",
  every: 30s,
  }

[celinaT]

Any updates here?