Dev: Provisioning logic debugging

This page provides information about known issues that may cause provisioning to fail.

In addition to failures, in some situations make all might take a very long time or even hang. When it's the first time provisioning, this is normal, because it is pulling 3 different template VMs (> 2GB of packages, which could take 1h or more). Once the first make all successfully completes, it is possible that bugs in the provisioning logic make subsequent runs hang. The following will provide helpful debugging information.

Known issues that may cause failures

• make all may conflict with updates of the legacy 24h cron job updater. You may wish to apply the changeset to remove the legacy updater prior to provisioning. Details about the issue: https://github.com/freedomofpress/securedrop-workstation/issues/378

• During a make clean operation, sd-proxy may fail to shut down: https://github.com/freedomofpress/securedrop-workstation/issues/429

• You may sometimes need to manually run dpkg --configure -a in individual templates, or package-related provisioning logic will fail for those templates: https://github.com/freedomofpress/securedrop-workstation/issues/428

What to do when make all hangs

Identify VMs causing issue(s)

[user@dom0]$ qvm-ls --running

Observe the VMs that are running that are prefixed by disp-mgmt-. Those will indicate which VM provisioning steps are slow/handing.

Recovering and re-running make all

To recover, you can qvm-kill the management machine:

[user@dom0]$ qvm-kill disp-mgmt-sd-whatever

If whatever-vm is not automatically shut down afterwards, kill that one too:

[user@dom0]$ qvm-kill sd-whatever

You can then safely run make all or make sd-whatever again