Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: cmake asan build #4048

Merged
merged 16 commits into from
Jan 12, 2024
Merged

ci: cmake asan build #4048

merged 16 commits into from
Jan 12, 2024

Conversation

jmayclin
Copy link
Contributor

@jmayclin jmayclin commented Jun 9, 2023

Resolved issues:

Description of changes:

Currently the address sanitizer job can't be run through CMake. This PR

  • adds new self-contained buildspec to run asan jobs
  • adds short docker readme of fun facts that I learned while putting together my buildspec

I added the asan buildspec as a separate codebuild job because it feels simpler and a little bit more self-documenting. It also reduces maintenance because now we can use the buildspec as the actual specification for the codebuild job rather than copying and pasting everytime we update it.

Call-outs:

After this PR is merged

  1. enable webhook to run the ASAN job on PRs
  2. change github config to make passing the ASAN job mandatory
  3. remove old ASAN jobs

Testing:

Here is a codebuild job confirming that we fail when there is a memory leak present: link This was the result of running the codebuild job on mainline s2n-tls with the mem fix for ossl3 reverted.

Here is the successful asan codebuild job for the PR: link

All other existing CI should pass.

Is this a refactor change? If so, how have you proved that the intended behavior hasn't changed?

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@github-actions github-actions bot added the s2n-core team label Jun 9, 2023
@jmayclin jmayclin force-pushed the cmake-asan-build branch 4 times, most recently from 3a8fba0 to e581e12 Compare June 13, 2023 01:32
@jmayclin jmayclin changed the title Cmake asan build ci: cmake asan build Jun 13, 2023
@jmayclin jmayclin requested a review from lrstewart June 13, 2023 22:43
@jmayclin jmayclin marked this pull request as ready for review June 13, 2023 22:44
@jmayclin jmayclin requested a review from dougch as a code owner June 13, 2023 22:44
- |
cmake . -Bbuild \
-DCMAKE_C_COMPILER=/usr/bin/clang \
-DCMAKE_PREFIX_PATH=/usr/local/$S2N_LIBCRYPTO \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for dropping the ./test-deps/$S2N_LIBCRYPTO pattern.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ya, the fact that the codebuild src folder is always different, and therefore test-deps is always a different path felt like a bit of forbidden magic 😄 . I also hope that this pattern will reduce the delta between "how we run things in CI" and "how we build things locally"

@@ -120,43 +120,6 @@ batch:
S2N_LIBCRYPTO: 'awslc-fips'
BUILD_S2N: 'true'

- identifier: s2nAsanOpenSSL111Coverage
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just update these with the new job?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. I guess that depends on how exactly we are using the omnibus job. I usually think of it as "approximate list of the CI jobs that we run", and since the Asan jobs are now documented in the Asan buildspec, I figured it made sense to remove it.

But perhaps your point is that we do run the omnibus job, in which case we should keep the asan jobs in there? I'd be fine with that in the short term to unblock the PR, but long term it feels a bit odd to duplicate all of our CI build specs. Could whatever is using the omnibus just switch the using the normal CI jobs that we have?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've had some drift since the original setup, so it's worth a review. Internal releases still use the Omnibus job, but it no longer contains all the jobs (it was originally the source of truth that the other spec files came from).

@jmayclin jmayclin requested a review from dougch June 15, 2023 23:33
CMakeLists.txt Outdated Show resolved Hide resolved
utils/s2n_random.c Outdated Show resolved Hide resolved
codebuild/spec/buildspec_asan.yml Show resolved Hide resolved
codebuild/spec/buildspec_asan.yml Outdated Show resolved Hide resolved
jmayclin added 3 commits July 18, 2023 10:31
* move ASAN ignore to s2n_safety
* remove newline from CMakeLists.txt
@jmayclin jmayclin requested a review from lrstewart July 18, 2023 18:40
CMakeLists.txt Outdated
Comment on lines 514 to 515
set(UNIT_TEST_ENVS ${UNIT_TEST_ENVS} S2N_ADDRESS_SANITIZER=1)
endif()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't there now a line in the next if that's a duplicate? Line 511/521?

codebuild/README.md Outdated Show resolved Hide resolved
@github-actions
Copy link

This PR has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@@ -568,8 +568,11 @@ S2N_RESULT s2n_set_private_drbg_for_test(struct s2n_drbg drbg)
/*
* volatile is important to prevent the compiler from
* re-ordering or optimizing the use of RDRAND.
*
* This is marked with ASAN_IGNORE because address sanitizer is unable to deal
* with the inline assembly and emits false positives.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not a false positive, it's an actual bug
#4310

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for catching this! I'll go ahead and update the PR.

@jmayclin jmayclin requested a review from lrstewart January 10, 2024 23:29
@jmayclin jmayclin enabled auto-merge (squash) January 12, 2024 01:38
@jmayclin jmayclin merged commit 7c471bb into aws:main Jan 12, 2024
28 checks passed
dougch pushed a commit to dougch/s2n-tls that referenced this pull request Jan 17, 2024
@jmayclin jmayclin deleted the cmake-asan-build branch July 1, 2024 07:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants