Paralel processing of incoming hook calls

[mikicz]

So today I got a first errors report on Sentry from the app, specifically that the pulling of the repository failed because there was already a pull in progress in a different process - two requests came at a very similar time (two seconds apart) and because I configured Apache to have two workers, it processed them in parallel.

It happened for the first time since the app started to be used, so we might not need to solve it yet, but eventually it might become more of a problem.

Two possible solutions come into mind - reducing the number of workers to one (not really ideal) or some kind of locking mechanism around the pulling of the repository.

The error message:

Cmd('git') failed due to: exit code(1)
  cmdline: git pull -v origin
  stderr: 'fatal: Unable to create '<project_path>/.arca/repos/github.com_pyvec_naucse.python.cz/master/.git/refs/remotes/origin/master.lock': File exists.

If no other git process is currently running, this probably means a
git process crashed in this repository earlier. Make sure no other git
process is running and remove the file manually to continue.'

[hroncok]

AFAIK that second request can be safely discarded because it will be included in the Travis rebuild. All that needs to be done is to properly handle the error (without screwing up a case where git previously in fact died).

[mikicz]

Not necessarily though. Imagine the following situation.

Branch mikicz/requestA is not linked to in the main repository. Branch mikicz/requestB is.

I push to mikicz/requestA, triggering request A, and then immediately to branch mikicz/requestB which triggers request B.

A: Starts pulling the repository
B: Tries to pull the repository, dies because A is still working
A: Goes through the repository, finds that mikicz/requestA is not used and therefore does not trigger a build.

However since mikicz/requestB is used in the main repo, request B should have triggered a build, but thanks to it failing it doesn't.

[hroncok]

Oh, right.