TestDrive is a handy script for developers working on The MarkUs Project.
The MarkUs Project practices pre-commit code review with Review Board in order to reduce the number of defects committed to the repository, and also as a way of quickly bringing new developers up to speed.
But looking at the code is only half of the story. It’s just as important to try the code that’s up for review, just to make sure it’s up to snuff. That’s where TestDrive comes in.
TestDrive has its own clone of the MarkUs repository. When you TestDrive a review request, TestDrive will make sure that repository is up to date, and then try to splat the patch from the review request onto the repository.
TestDrive will even run the MarkUs test suites for you, and compile it into a report, if you’d like.
Call it pre-commit continuous integration. I hope it’ll help you find more bugs.
Every review request up on Review Board has a unique ID. You can see this ID in the URL when you’re viewing the review request. For example, if I was looking at a review request, and the URL looked like:
review.markusproject.org/r/210
Then the review request number is, clearly, 210.
That’s really all you need to know to get started. Once you’ve got TestDrive, run it like this:
./TestDrive -r 210
And it’ll pretty much do all of the rest. TestDrive might ask you to specify a branch to grab, in case it wasn’t made clear in the review request.
TestDrive can also run the test suites, or start the server, as soon as the new instance is ready to go.
./TestDrive -r 210 -a -s
The -a parameter tells TestDrive to auto-test. The -s tells TestDrive to run the server when it’s ready to go. Pretty simple.
To get a full list of parameters, type in:
./TestDrive -h
Once you’ve downloaded TestDrive, do the following:
-
Clone a fresh copy of the MarkUs Github repository
-
Copy config/database.yml.sqlite3 to config/database.yml (or otherwise configure MarkUs to work with your current database layout)
Open up TestDrive.conf.yml, and make sure td_clone points to that cloned repositories path. That’s it!
Review Board currently has a bug that distorts git diffs that create new files. Until this is fixed, it is not possible for us to use the apply-patch method for TestDriving those review requests.
Similarly, text diffs cannot contain binary files. So if your review request includes things like images, or other binary data, it’s not going to be included in the TestDrive.
To deal with this, we have a work-around:
-
Have the review requester push their patch to a branch on their Github account
-
Have the review requester include “@branch <branch-name>” in the description of their review request
When TestDrive sees @branch <branch-name>, it’ll attempt to grab/merge that branch from the review requester’s Github account (assuming they are listed in github_accounts.yml).
TestDrive uses the git-apply command to apply the review request diff. If git-apply has problems…well, then you’ve got problems. At least for now, TestDrive doesn’t detect when patch fails, so keep an eye out. This may mean you’ll have to do some manual merging to make things work…
You’ll need the JSON Ruby gem. Oh, and Git - but as a MarkUs developer, you probably already have that. ;)
Written by Mike Conley as an experiment, first mentioned on my blog here.
Other contributors:
-
Evan Browning
-
Benjamin Vialle
Want to help contribute to The MarkUs Project? Read more about it here!