Speed up STF tests by only compiling P4 program once for all test packets

[jafingerhut]

No description provided.

[jafingerhut]

One possible way to address #4571

The .p4.test script for an STF test currently does this:

(1) run p4testgen to create multiple STF tests, one packet per file
(2) for each such STF test, compile the P4 program and run BMv2 on it

These changes modify step (2) to be:

(2a) for the first STF test, compile the P4 program, copy the BMv2 JSON file to a place outside the tmpXXXXX directory provided on the run-bmv2-test.py command line as a new option, and run BMv2 on it
(2b) for the second and later STF tests, copy the BMv2 JSON file from the place outside the tmpXXXXX directory provided ont eh run-bmv2-test.py command line as a new option, skip running the P4 compiler because the BMv2 JSON file is already present, and run BMv2 on it

Basically we avoid running p4c-bm2-ss on the 2nd and later test packets. For some of the larger P4 test programs, they typically generate the maximum of 10 test packets, and today take 4-5 minutes on a 5-year-old x86_64 laptop I use. With these changes, that same test takes about 2 minutes. I think there are about 512 STF tests in the entire test suite, but most of them are shorter than that today, but I would not be surprised if this saved 30 minutes or so across all of them.

[fruffy]

Great thanks! This makes STF tests much more practical.

Personally, I would also consider implementing this by using a parameter like --stf-files [FILES] that either contains a list of files or a glob pattern. Then restructure the logic of the script to only compile once and reuse the json for all subsequent tests.

[jafingerhut]

@fruffy Is this something you think is worth merging as written? Or would you prefer that your suggested --stf-files [FILES] technique were implemented instead, and abandon the approach in this PR?

[fruffy]

@fruffy Is this something you think is worth merging as written? Or would you prefer that your suggested --stf-files [FILES] technique were implemented instead, and abandon the approach in this PR?

I have not been able to work on the other approach, so I can not say how involved it is. Although I think it requires only few changes. If the other approach is simple enough I'd prefer it because it requires fewer changes on the P4Testgen script side. Until then we can keep this open.

[fruffy]

Working on a rewrite of the run-bmv2-stf.py script, which has bitrotted a fair bit. After that rewrite I intend to extend the script to support running multiple files at once.