nrunner cannot reuse the env file in a loop

[PaulYuuu]

After we switched to using the new runner, the env file cannot be reused, which means some resources will be generated in each case, this further increases the consumption of the resource pool, such as DHCP. In the past, the first case will generate a MAC address and save it into env file, and then the second case can reuse this, but for now, every VM has a different MAC. If the DHCP pool is small, VM sometimes fails to get an IP address in some cases because the pool is eaten up.

IIUC, the nrunner will create a sub-process for each case, so the env file will be deleted when the case is finished. To fix this issue, I thought of two ways:

1. Have an enhancement in avocado to make nrunner have the same behavior as the legacy runner when max_parallel_tasks is set to 1.
2. Fix this on avocado-vt side, save the MAC address cache somewhere, but this can only fix the MAC issue.

$ rg "env"
1-io-github-autotest-qemu.unattended_install.cdrom.extra_cdrom_ks.default_install.aio_threads.arm64-pci/debug.log
81:[stdlog] 2023-02-22 05:12:26,710 avocado.test DEBUG|     env_cleanup = no
137:[stdlog] 2023-02-22 05:12:26,717 avocado.test DEBUG|     job_env_cleanup = no
348:[stdlog] 2023-02-22 05:12:27,909 avocado.virttest.utils_env WARNI| Creating new, empty env file
...
...
2-io-github-autotest-qemu.numa_opts.nodes.64.arm64-pci/debug.log
56:[stdlog] 2023-02-22 05:30:27,843 avocado.test DEBUG|     env_cleanup = no
106:[stdlog] 2023-02-22 05:30:27,849 avocado.test DEBUG|     job_env_cleanup = no
313:[stdlog] 2023-02-22 05:30:28,640 avocado.virttest.utils_env WARNI| Creating new, empty env file

Hello @luckyh @clebergnu, do you have any opinion about this issue?

[clebergnu]

Thanks for reporting this @PaulYuuu. @richtja mentioned that this is similar to the requirements of reusing containers in the LXC spawner effort, so maybe we can think of a common solution.

[luckyh]

I haven't dived deeper into the root cause of this issue, but I feel like the cache facility would be something might help on that. Maybe we can have a try.