Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bundle CyberChef into a container and publish to GCHR #1699

Merged
merged 16 commits into from
Feb 13, 2024

Conversation

AshCorr
Copy link
Contributor

@AshCorr AshCorr commented Jan 30, 2024

Bundle CyberChef into a container and publish to GCHR

Yet another attempt at bundling CyberChef into a container! There have been various attempts made before this one but they all seem to have been abandoned:

But as the famous saying goes "8th time's the charm" and maybe this PR will be the one that finally does it.

This PR adds a:

  • Containerfile which uses simple-web-server a lightweight HTTP server built in rust for serving static files built in rust to serve CyberChef.
    • EDIT: Now using Nginx, it results in a slightly larger image but with the benefit that we're using tried and tested software.
  • New steps to the release.yml workflow
    • A step using docker-metadata-action which generates tags to be applied to the image from the release version (1, 1.0, 1.0.0 and latest)
    • A step using buildah to build an OCI compliant container image. It builds a both an ARM64 image and a AMD64 image.
    • And finally a step to push the built container to GHCR using the GITHUB_TOKEN built into github actions.
  • A step to the pull-requests.yml workflow to verify that the Containerfile builds correctly

and a couple of flyby fixes that can be dropped if needs be:

  • A version bump to ChromeDriver, fixing the failing UI tests being experience by all PRs. This was caused by a bump to Chrome in the ubuntu-latest runner. It looks like another user has also attempted this in Update chromedriver to v119 so that it is compatible with the chrome … #1656
  • Switch the Release workflow to use npm ci instead of npm install. npm ci respects the package-lock.json file whereas npm install will try and bump dependencies which is not ideal in a CI workflow as it adds uncertainty into which dependencies are being used in your production bundle.

Related to #1280

Why?

Theres various un-official images floating around of CyberChef, although I'm sure most of them are probably fine, but for a tool such as CyberChef where I'm inputting potential sensitive data I'd rather not take any chances.

Testing

I've published a version built on my own fork to ghcr.io/ashcorr/cyberchef:latest which may be used for testing.

podman run -p 8080:80 -it ghcr.io/ashcorr/cyberchef:latest or docker run -p 8080:80 -it ghcr.io/ashcorr/cyberchef:latest

image

@CLAassistant
Copy link

CLAassistant commented Jan 30, 2024

CLA assistant check
All committers have signed the CLA.

Containerfile Outdated Show resolved Hide resolved
@AshCorr AshCorr marked this pull request as ready for review January 30, 2024 15:48
@a3957273
Copy link
Member

a3957273 commented Feb 2, 2024

Fixed some merge conflicts from another PR that updated chromedriver.

@AshCorr
Copy link
Contributor Author

AshCorr commented Feb 2, 2024

Fixed some merge conflicts from another PR that updated chromedriver.

Thanks!

.github/workflows/pull_requests.yml Show resolved Hide resolved
.github/workflows/releases.yml Outdated Show resolved Hide resolved
.github/workflows/releases.yml Show resolved Hide resolved
.github/workflows/releases.yml Show resolved Hide resolved
@AshCorr AshCorr force-pushed the ash/Containers! branch 2 times, most recently from 5e1686f to 73144c5 Compare February 8, 2024 10:14
@AshCorr AshCorr requested a review from a3957273 February 8, 2024 12:21
Copy link
Member

@a3957273 a3957273 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one minor change!

.github/workflows/releases.yml Outdated Show resolved Hide resolved
@a3957273
Copy link
Member

a3957273 commented Feb 8, 2024

When building this container locally, I notice it comes with a lot of context. Could we add a .dockerignore file so we transfer less other stuff? E.g.

node_modules
npm-debug.log
travis.log
build
.vscode
.idea
.*.swp
**/*.DS_Store
tests/browser/output/*
.node-version

@SebastianThorn
Copy link

This is relevant as well for this issue: #1582
Not sure if that is fixed or not, but for a production-build, there should be as few unnecessary components as possible.

@a3957273
Copy link
Member

a3957273 commented Feb 9, 2024

@SebastianThorn A fair point. I'm inclined to merge this PR in even if it does require internet to compile, then accept other PRs to improve the process? A single PR doesn't necessarily need to be perfect, and I'm happy that a future improvement to allow offline building wouldn't impact existing users of this feature.

@maxstanley
Copy link

maxstanley commented Feb 11, 2024

As you've reference my closed PR, I thought I would drop a comment and say this will be a great addition. Feel free to copy anything from my PR into this branch e.g. dockerignore.

I think for me as a user the main thing that is missing is an update to the README that demonstrates how to build with docker/podman and how an end-user might deploy this.

tyvm

README.md Outdated
docker run -it -p 8080:80 ghcr.io/gchq/cyberchef:latest
```

This image is built and published through our [GitHub Workflows](.github/workflows/docker.yml)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to update the workflow file reference ;)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, good catch!

with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=semver,pattern={{major}}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason for the omission of:

                      type=raw,value=latest

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its added automatically when using a type=semver pattern! See https://github.com/docker/metadata-action?tab=readme-ov-file#latest-tag

.github/workflows/releases.yml Outdated Show resolved Hide resolved
.github/workflows/releases.yml Outdated Show resolved Hide resolved
@SebastianThorn
Copy link

The switch to Nginx is good.

@AshCorr AshCorr force-pushed the ash/Containers! branch 2 times, most recently from 704ff8f to 879fd1f Compare February 12, 2024 10:46
@@ -33,6 +33,20 @@ jobs:
if: success()
run: npx grunt prod

- name: Production Image Build
if: success()

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this still required?

@a3957273
Copy link
Member

I'm going to merge this as is, because this conversation already has over three dozen comments. Happy to accept other PRs for improvements! 👍

Thanks so much for this @AshCorr. I love how you saw the other 7 PRs and decided to make an 8th 😛, persistence is the key!

@a3957273 a3957273 merged commit 4b9d5a7 into gchq:master Feb 13, 2024
4 checks passed
@a3957273
Copy link
Member

@AshCorr Seems to be erroring on permissions, see: https://github.com/gchq/CyberChef/actions/runs/7879757258/job/21500616862

@AshCorr
Copy link
Contributor Author

AshCorr commented Feb 13, 2024

@AshCorr Seems to be erroring on permissions, see: https://github.com/gchq/CyberChef/actions/runs/7879757258/job/21500616862

Odd, I didn't touch that action thats erroring. I wonder if something wonky has happened where defining an explicit packages: write permission has caused default permissions to not be applied? My earlier statement of "Github permissions always confuse me" I think is relevant again 😅

By the looks of it you should be able to add an explicit contents: write permission to fix this, see svenstaro/upload-release-action#70 (comment)

@AshCorr
Copy link
Contributor Author

AshCorr commented Feb 13, 2024

Ah yes, that does appear to be the case as per https://docs.github.com/en/actions/security-guides/automatic-token-authentication#modifying-the-permissions-for-the-github_token

You can use the permissions key in your workflow file to modify permissions for the GITHUB_TOKEN for an entire workflow or for individual jobs. This allows you to configure the minimum required permissions for a workflow or job. When the permissions key is used, all unspecified permissions are set to no access, with the exception of the metadata scope, which always gets read access.

My apologies for missing that! I believe I commented out this action on my fork as I couldn't get it working, I assumed it was due to it being run on a fork, not due to lack of permissions.

@a3957273
Copy link
Member

No worries, super not clear. Removing all permissions and just using the defaults appear to have worked well!

Package now available here! https://github.com/gchq/CyberChef/pkgs/container/cyberchef

@AshCorr

@AshCorr
Copy link
Contributor Author

AshCorr commented Feb 13, 2024

No worries, super not clear. Removing all permissions and just using the defaults appear to have worked well!

Package now available here! https://github.com/gchq/CyberChef/pkgs/container/cyberchef

@AshCorr

image

Looks to be working! Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants