add dependabot.yml #2353

rtpascual · 2024-12-18T23:32:17Z

Problem

We can be more proactive in keeping our dependencies up to date so we can avoid problem statements like in #2303.

Issue number, if available:

Changes

Adds dependabot.yml which will trigger Dependabot to check for version updates of our dependencies.

Corresponding docs PR, if applicable:

Validation

Forked the repo and used dependabot.yml to create PRs to update dependencies, see https://github.com/rtpascual/amplify-backend-fork/pulls.

Checklist

If this PR includes a functional change to the runtime behavior of the code, I have added or updated automated test coverage for this change.
If this PR requires a change to the Project Architecture README, I have included that update in this PR.
If this PR requires a docs update, I have linked to that docs PR above.
If this PR modifies E2E tests, makes changes to resource provisioning, or makes SDK calls, I have run the PR checks with the run-e2e label set.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

changeset-bot · 2024-12-18T23:32:20Z

🦋 Changeset detected

Latest commit: a79bfee

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 0 packages

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

.github/dependabot.yml

Co-authored-by: Kamil Sobol <[email protected]>

Amplifiyer · 2024-12-19T09:36:39Z

.github/dependabot.yml

+      # Runs every Monday
+      interval: 'weekly'


why weekly and not say daily?

I was thinking weekly to start so we aren't constantly bombarded with version update PRs, especially since we are using increase-if-necessary the PRs will most likely be for new MVs so we need to be thorough on checking for breaking changes.

Or we make sure E2E runs on these PRs and we assume if E2E passes then there are no breaks, in that case I'm fine with changing it to daily.

Amplifiyer · 2024-12-19T09:46:09Z

.github/dependabot.yml

+      # Runs every Monday
+      interval: 'weekly'
+    # Update package.json files if new version is outside of version range specified there. Otherwise lock file only.
+    versioning-strategy: increase-if-necessary


If this strategy can update package.json files, how will we accommodate changesets in those PRs?

That's a good point. Best if this could be automated.

If it can't:
PR won't pass checks until changeset is added (if our check works)
Maintainer adds changeset (this is doable using UI here #2353 (comment) +/- quotes style but we could exclude changeset files from prettier.)

Good point on changesets, I'll see if I can automate it.

With change in d826612, I used the new script locally for this PR rtpascual#20 and this is the resulting changeset file that will be force pushed to the PR (had to do it locally to simulate health_checks running on PR):

sobolk

The idea for automation looks good.

sobolk · 2024-12-20T15:04:10Z

.github/dependabot.yml

+    # Update package.json files if new version is outside of version range specified there. Otherwise lock file only.
+    versioning-strategy: increase-if-necessary
+    labels:
+      - 'dependency'


What's the purpose of dependency label?

Just keeping that label from past dependabot PRs, I'll remove the labels here since we will be adding it through the script

sobolk · 2024-12-20T15:09:56Z

scripts/dependabot_generate_changesets.ts

+const getVersionUpdates = async (files: string[]) => {
+  const updates = new Map<string, string>();
+  for (const file of files) {
+    const changes = await gitClient.getFileChanges(file);
+    for (const change of changes.split(EOL)) {
+      if (!change.startsWith('+ ')) {
+        continue;
+      }
+      // This will capture lines in git diff like:
+      // +  "<dependency>": "^<version>"
+      const match = change.match(/"(.*)": "(.*)"/);
+
+      if (!match) {
+        continue;
+      }
+      // Add dependency name (match[1]) and new version (match[2]) to Map
+      updates.set(match[1].replace(/"/g, ''), match[2].replace(/"/g, ''));
+    }
+  }
+
+  return updates;
+};


One other source to consider is parse PR title or description. I'm not sure if this will be any simple - please evaluate and adopt or reject this suggestion.

Sometimes dependabot will group version updates together so the title can be something like Bump the npm_and_yarn group across 1 directory with 3 updates.

I think description will be best, since it gives us the version dependabot updated from since there will be a line like Updates/Bumps <dependency> from <old-version> to <new-version>.

sobolk · 2024-12-20T15:12:00Z

scripts/dependabot_generate_changesets.ts

+// Get modified package.json files that we care about for changesets
+const packageJsonFiles = changedFiles.filter(
+  (changedFile) =>
+    changedFile.startsWith('packages/') &&
+    !['packages/integration-tests', 'packages/eslint-rules'].some(
+      (packageName) => changedFile.startsWith(packageName)
+    ) &&
+    changedFile.endsWith('package.json')
+);


You can use private instead of names, see

amplify-backend/scripts/components/api-changes-validator/api_changes_validator.ts

Line 49 in b574bce

if (latestPackageJson.private) {

sobolk · 2024-12-20T15:17:26Z

scripts/dependabot_generate_changesets.ts

+await gitClient.commitAllChanges('add changeset');
+await gitClient.push({ force: true });


How does the stop condition work?
I.e. this will presumably push a change to PR and trigger new workflow. Would that trigger this script again or not ? If this is handled already - please drop a comment somewhere - it's not easy to catch this detail.

Other thing is e2e tests.
In case script adds extra commit then we'll be kicking off 2 e2e runs next to each other. If possible we should kick off one.
For example this script could add run-e2e label before pushing. And always commit something (even if it's only pacakge-lock update).

Good catch I don't have anything in place to stop pushing the changeset from triggering the script again, I'll see where I can handle that (possibly using the fact that the script may be adding the run-e2e label?).

That's a good idea on the script adding the run-e2e label to prevent the initial e2e run, then we can push an empty commit or empty changeset (in case of only package-lock update).

sobolk · 2024-12-20T15:18:45Z

scripts/dependabot_generate_changesets.ts

@@ -0,0 +1,93 @@
+import fsp from 'fs/promises';


I'd rename this script to dependabot_handle_version_update or something like this.
It's doing a bit more than generating changeset (i.e. it's also pushing it to repo).

Also - consider writing a happy case test case for this.
Something like https://github.com/aws-amplify/amplify-backend/blob/main/scripts/components/release_lifecycle.test.ts .

scripts/components/dependabot_version_update_handler.ts

add dependabot.yml

7de5e5a

rtpascual requested review from a team as code owners December 18, 2024 23:32

sobolk reviewed Dec 18, 2024

View reviewed changes

.github/dependabot.yml Show resolved Hide resolved

add increase-if-necessary strategy

7a7659d

sobolk previously approved these changes Dec 19, 2024

View reviewed changes

.github/dependabot.yml Outdated Show resolved Hide resolved

Update .github/dependabot.yml

072eb0b

Co-authored-by: Kamil Sobol <[email protected]>

rtpascual dismissed sobolk’s stale review via 072eb0b December 19, 2024 01:27

sobolk previously approved these changes Dec 19, 2024

View reviewed changes

Amplifiyer reviewed Dec 19, 2024

View reviewed changes

dependabot adds e2e label and action to add changesets to dependabot PRs

d826612

rtpascual dismissed sobolk’s stale review via d826612 December 20, 2024 01:27

rtpascual added 5 commits December 19, 2024 17:33

fix health checks

781d50b

fix this

184fd36

try this

9c8db99

add to eslint dictionary

bc872ff

Merge branch 'main' into dependabot-version-updates

6f17062

sobolk reviewed Dec 20, 2024

View reviewed changes

rtpascual added 6 commits December 20, 2024 15:14

PR feedback

2597451

add testing

2720df0

try this

296722e

try that

afe0907

mock git push

a632b8f

fix changeset file path

0114e26

github-advanced-security bot found potential problems Dec 24, 2024

View reviewed changes

scripts/components/dependabot_version_update_handler.ts Fixed Show fixed Hide fixed

update way ghContext is mocked

f6bc68f

github-advanced-security bot found potential problems Dec 24, 2024

View reviewed changes

scripts/components/dependabot_version_update_handler.ts Fixed Show fixed Hide fixed

rtpascual added 2 commits December 23, 2024 20:20

try this

4031cb0

try this

a79bfee

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add dependabot.yml #2353

add dependabot.yml #2353

rtpascual commented Dec 18, 2024

changeset-bot bot commented Dec 18, 2024 •

edited

Loading

Amplifiyer Dec 19, 2024

rtpascual Dec 19, 2024

Amplifiyer Dec 19, 2024

sobolk Dec 19, 2024

rtpascual Dec 19, 2024

rtpascual Dec 20, 2024 •

edited

Loading

sobolk left a comment

sobolk Dec 20, 2024

rtpascual Dec 20, 2024 •

edited

Loading

sobolk Dec 20, 2024

rtpascual Dec 20, 2024

sobolk Dec 20, 2024

sobolk Dec 20, 2024

rtpascual Dec 20, 2024

sobolk Dec 20, 2024

		await gitClient.commitAllChanges('add changeset');
		await gitClient.push({ force: true });

add dependabot.yml #2353

Are you sure you want to change the base?

add dependabot.yml #2353

Conversation

rtpascual commented Dec 18, 2024

Problem

Changes

Validation

Checklist

changeset-bot bot commented Dec 18, 2024 • edited Loading

🦋 Changeset detected

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rtpascual Dec 20, 2024 • edited Loading

Choose a reason for hiding this comment

sobolk left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rtpascual Dec 20, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

changeset-bot bot commented Dec 18, 2024 •

edited

Loading

rtpascual Dec 20, 2024 •

edited

Loading

rtpascual Dec 20, 2024 •

edited

Loading