Feature Request: Batch processing of sbom file with count of components more then 2000 (AWS Inspector limit)

[iriabov]

Description

We catch the limitation error in the AWS Inpector CLI command to scan SBOM files. Error log message:
An error occurred (ValidationException) when calling the ScanSbom operation: Input SBOM failed validation: Max number of components exceeded (2000)

You can find the package limit in AWS documentation:
https://docs.aws.amazon.com/inspector/v2/APIReference/API_scan_ScanSbom.html#API_scan_ScanSbom_RequestSyntax

The JSON file for the SBOM you want to scan. The SBOM must be in CycloneDX 1.5 format. This format limits you to passing 2000 components before throwing a ValidException error.

Expected Behavior

Divide the SBOM file by chunks (with the same metadata and max count of components - 2000). The next step is scanning these files independently. The last step is calculating vulnerabilities from all files.

Actual Behavior

We have an error in the scanning GH action:
An error occurred (ValidationException) when calling the ScanSbom operation: Input SBOM failed validation: Max number of components exceeded (2000)

Steps to Reproduce

Build the image with more the 2000 of components in SBOM file.

Other Information

Please provide any additional information such as logs or screenshots.

[s-kenji]

Hello, thank you for reaching out.
We are confirming receipt of your issue.
We will begin to triage the problem, and will report back when we have pertinent updates.

[bluesentinelsec]

Hello, thank you again for creating this issue.
We understand the 2,000 component limit is preventing you from completing Inspector scans.
Splitting the SBOM into smaller chunks is new functionality that we will have to implement.
We have this feature request in our backlog, but we do not have an ETA as to when work will begin.

I'm going to mark this issue as on-hold until we begin work on the feature.

[bluesentinelsec]

@iriabov Hello, we recently increased the permitted component count to 5,000.
Regarding chunking, that is still being discussed internally as an enhancement.

[iriabov]

Hi @bluesentinelsec
Awesome, thank you for update and thank you so much for your time and effort—I really appreciate it!
If it's alright with you, I could try to implement chunking and create a PR to review.

Thank you!

[bluesentinelsec]

Hi @bluesentinelsec Awesome, thank you for update and thank you so much for your time and effort—I really appreciate it! If it's alright with you, I could try to implement chunking and create a PR to review.

Thank you!

Contributions are certainly welcome. :)
I would actually recommend that you write a stand alone script for this purpose, that way, you don't need to worry about any of the GitHub Actions integration. This is the basic workflow I have in mind:

#1. First generate a large sbom
./inspector-sbomgen container --image large_image:latest -o large_sbom.json

#2. Now scan the SBOM; the program will need to handle chunking the scan into multiple requests,
# and then reassembling the responses into a single JSON document.
./scan_sbom_in_chunks.py --f your_large_sbom.json --profile your_aws_profile

# You'll need to use the AWS SDK to interact with Inspector's ScanSbom API.

Be advised, I think this is a non-trivial program to write; no worries if you need to back out because this is likely a moderate amount of work. Otherwise, feel free to take a stab and we'll provide feedback if you open a pull request.