Arion tries to start a service before podman is up

[pedorich-n]

Hi!

I've just encountered an error where after reboot arion on NixOS was trying to start the containers before the podman was ready:

2024-04-21T22:24:15+0900 arion-home-automation-start[2418281]: docker compose file: /nix/store/4d2i95x5rhb55dyncssr4xfz87sl8lyh-docker-compose.yaml
2024-04-21T22:24:15+0900 arion-home-automation-start[2418291]: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
2024-04-21T22:24:15+0900 arion-home-automation-start[2418284]: arion: readCreateProcess: docker "images" "--filter" "dangling=false" "--format" "{{.Repository}}:{{.Tag}}" (exit 1): failed

It looks like it auto-restarted eventually.

But this makes me think: would it make sense to add something like After = [ "podman.service" ] (or docker, depending on the backend) or ConditionPathExists=/var/run/docker.sock to the systemd service definition to ensure it starts correctly from the first try?

Edit: More info
NixOS stable 23.05, nixpkgs commit bc194f70731cc5d2b046a6c1b3b15f170f05999c

$ podman --version
podman version 4.7.2

$ arion version
docker-compose version 1.29.2, build unknown
docker-py version: <module 'docker.version' from '/nix/store/7420rvz9fw7cjqkjf5i62zarv8s4p21c-python3.11-docker-6.1.3/lib/python3.11/site-packages/docker/version.py'>
CPython version: 3.11.8
OpenSSL version: OpenSSL 3.0.13 30 Jan 2024

[roberth]

It does have

arion/nixos-module.nix

Line 38 in 1886d25

after = [ "sockets.target" ];

So that suggests that the podman socket is not registered with systemd.

Maybe it needs to run after tmpfiles too?

https://github.com/NixOS/nixpkgs/blob/ff03bc83894ca42d93f80ec6ea82b9e4eaff02b9/nixos/modules/virtualisation/podman/default.nix#L244

Ideally systemd would know that the docker socket is an alias for the podman socket. I think this could be achieved with multiple ListenStreams in podman: one for each location. That makes the tmpfiles solution seem like a hack.

[pedorich-n]

Ah, I didn't realize after = [ "sockets.target" ]; is there to ensure it starts after the socket is present. This makes sense now.

Thanks for the quick fix!
Like you said, race conditions are hard to test, but with your change applied after a couple of reboots, I haven't seen the issue come up again.