RFC: Seeding Data, Auth users and groups

[josefaidt]

Hey folks 👋 we’ve been thinking about seeding capabilities for the Data and Auth categories, and would love to get your feedback! In our explorations we’ve found seeding can be applied to two primary use cases:

1. seeding arbitrary data for testing (auth flows, UI rendering, etc.)
2. seeding preliminary data for environments (base tenant configurations, etc.)

Today, you can use the AWS SDK in conjunction with the Amplify (Gen 2) data client to seed sandbox and branch environments, however this involves writing the scripts and manually invoking at some point in time. This can be tedious to author, test, and orchestrate, where we would like to provide functionality out of the box to help make this easier.

Thoughts

At a high level we’re thinking of a TypeScript file that exports a seed function with a new command to invoke, amplify seed [path-to-file]. The thought highlights two notable attributes of the intended experience:

1. preconfigured clients for interacting with Auth and Data resources
2. an orchestration mechanism

Getting started, you will be able to create a seed.ts file in your amplify/ directory that exports a function with the SeedFunction type:

// amplify/seed.ts
import type { SeedFunction } from '@aws-amplify/backend'
import { backend } from './backend'

const seed: SeedFunction<typeof backend> = async ({ auth, data }) => {
  // create a test user
  const user = await auth.createUser('testuser', 'Testpassword1234!')
  // set the test user on the session
  auth.setCurrentUser(user)
  // create 10 posts as that user
  let posts: Schema['Post'][] = []
  for (let i = 0; i < 10; i++) {
    const post = await data.models.Post.create({
      title: `post ${i + 1}`,
      authMode: 'userPool'
    })
    posts.push(post)
  }

  // create an admin user
  const admin = await auth.createUser('testadmin', 'Testpassword1234!', {
    groups: ['ADMINS']
  })

  // use the admin user to update a post
  auth.setCurrentUser(admin)
  await data.models.Post.update({
    id: posts[0].id,
    title: 'updated title by admin'
  })
}

export default seed

By providing seed via a TypeScript file, this would also enable the use of flat files and create complex data model relationships. For seeding preliminary data to a new environment, we can provide some mechanism to ensure seed only runs once for that environment — this way you do not need to juggle amplify seed in your buildspec/amplify.yml file.

Acknowledgements

• Prisma’s seed functionality https://www.prisma.io/docs/orm/prisma-migrate/workflows/seeding

[josefaidt]

this is worth noting for the auth client https://github.com/aws-amplify/amplify-js/blob/main/packages/aws-amplify/src/adapterCore/authProvidersFactories/cognito/createUserPoolsTokenProvider.ts#L21-L37

[AnaCoda]

Is there any ETA on data seeding capabilities or current examples/best practices for data seeding in Amplify Gen 2?

[ideen1]

@josefaidt Seconding @AnaCoda comment. This or a temporary workaround would be really useful.

[renebrandel]

I think we also need a mechanism to dynamically specify which type of branches should do seeding or if it's only available in sandbox environments.

[renebrandel]

@josefaidt Seconding @AnaCoda comment. This or a temporary workaround would be really useful.

@ideen1 / @AnaCoda - for non-sandbox environments, you can go to the Data Manager in the console and click "Actions" > "Generate seed data"

[renebrandel]

@josefaidt I was thinking maybe we could use a createSeedFunction. That way we can allow customers to pass in a set of patterns, in which we should run the seed logic.

// amplify/seed.ts
import { createSeedFunction } from '@aws-amplify/backend'
import { backend } from './backend'

const seed = createSeedFunction(['feature/*'], async ({ data }) => {
  // create 10 posts as that user
  let posts: Schema['Post']['type'][] = []
  for (let i = 0; i < 10; i++) {
    const post = await data.models.Post.create({
      title: `post ${i + 1}`,
    })
  }
})

export default seed

[josefaidt]

@renebrandel I like where you're headed but I think this would best be handled by something like #1094

[JDMathew]

It would also be useful if one could seed data with a specific data set say from a json or js file. - I see this could be potentially done in a loop in the seed function but it might be useful to have a dedicated api for it.

[sschibli97]

@josefaidt , are there any updates if these seed data capabilities will be coming at some point?
Also, in the meantime, are there any resources to how I could do this: "Today, you can use the AWS SDK in conjunction with the Amplify (Gen 2) data client to seed sandbox and branch environments"?

Would be a huge help!

[JGamboa]

@josefaidt any update on this?

[josefaidt]

Hey @sschibli97 and @JGamboa apologies for the delay on the reply! We are actively working towards a solution to enable seeding in sandbox environments 🙂

@sschibli97 we do not currently have a documentation page walking through seeding auth users, however you can use the CognitoIdentityProviderClient and its commands to create users, modifying profiles, emulating device CRUD, etc.. Information for your user pool can be found in amplify_outputs.json to interact. For the data client, you can use an API key or an IAM role to seed data, including user details (sub::username) for owner auth as necessary.