Add orca/orca_recovery_workflow modules, update docs #93
Conversation
|
Reviewers - please note this work is still in progress, as it's in review in the OBDAAC project. I'm posting here to start getting feedback as I believe it's in a form complete enough for additional feedback. |
|
@Jkovarik - thank you for this PR! Does this still reflect what OBDAAC is using (just checking to see whether there were any changes/updates after the OBDAAC review)? I'm planning on using much of this in NSIDC's use of ORCA. |
| @@ -0,0 +1,22 @@ | |||
| terraform { | |||
There was a problem hiding this comment.
I believe this second terraform block is redundant (since required_version is specified on line 5).
|
@mikedorfman thanks for the comment - I'm planning on reviewing where that finally landed in our merge PRs and this repo just to make sure things are up to date by the end of this sprint. |
|
@mikedorfman I checked the current state of the OB branch with Orca enabled against this PR and things look good. Please let me know if you run into issues integrating this PR/need help/etc. |
| ## -------------------------- | ||
|
|
||
| ## TODO - Decide if it's valueable to allow for an alternate cluster to the rds module cluster | ||
| ## OBDAAC implementation makes it YAGNI |
There was a problem hiding this comment.
Are these TODOs still relevant? Also I am not familiar with YAGNI?
| @@ -0,0 +1,2 @@ | |||
| orca_db_user_password="<db password goes here>" | |||
There was a problem hiding this comment.
I feel we should probably be promoting the use of ENV vars instead of tfvars using DB passwords stored in a repo. Maybe just a note?
There was a problem hiding this comment.
In general agree - that's more a function of how Orca is offering configuration as part of it's module. I believe we opted to generate the secrets file via script pulling from a secret store in the bamboo build, for what it's worth.
|
Thanks for the review @mattp0! Merging will need to be done via a maintainer - I'm considering my involvement with the PR done at this point without further engagement from the team. |
|
|
||
| variable "elb_account_id" { | ||
| type = string | ||
| default = "797873946194" |
There was a problem hiding this comment.
I don't think an account_id should have a default value. Am I missing something?
This PR adds two daac user modules:
orca-- this module deploys ORCA (https://nasa.github.io/cumulus-orca/docs/operator/restore-to-orca) using remote state information from therdsanddaacmodules for database/bucket configuration, and provides output to thecumulusmodule (dependent on Add orca DAAC module configuration hooks into CIRRUS-core CIRRUS-core#168 being merged) when configured to use orca. This allows for use of the Cumulus provided ORCA adapter lambda in workflows to archive data to ORCA.As this initial effort was in support of OBDAAC who are using the Cumulus default database module, the module is currently assuming use of the
rdsmodule. Improving this to allow for additional database clusters is beyond the scope of that work, but this module could be improved to accommodate other configurations.orca_recovery_workflow-- This module takes outputs from thecumulusandorcamodules to build a default 'recovery' workflow using the recovery adapter lambda Cumulus Core provides.I'd be happy to discuss further, provide demonstrations of the deployed modules, etc, as needed. Thanks for your consideration.