Pitest CI Integration
Pitest is a mutation testing product written by Henry Coles. It essentially mutates your code to see how many mutations are caught by your unit tests. It’s a great tool for analysing the robustness of unit tests.
I have an open source product that I’m working on, an emulator called EmuRox. I created it as a larger product I could test technologies and approaches out on. I’ve been running Pitest on it for a while to improve my pretty comprehensive tests. The problem is that no one that I’ve found has created a Pitest CI integration, let alone one that provides a GitHub shield and I want one.
What was required?
I needed a very small application server and something I could write small web scripts pretty quickly in. After some R&D I decided to go with Python and a free app server called Python Anywhere. I needed my Travis CI hooked build to include a Pitest report then send it to my new Pitest CI, which means my CI needs endpoints
- To receive Pitest reports
- To view Pitest reports
- To convert Pitest results into GitHub shields
Lastly, I needed the actual GitHub shield attached to my project.
### The CI
This is a pretty menial task in Python. Create some endpoints with the web library, contact shield.io for a shield image and return it. As I developed this fast, it initially has a few limitations:-
- There’s no security
- I only store the last build
- I can only display the last build
- I’m returning the Pitest report raw
Since I’m initially using the free tier of Python Anywhere, that also imposes certain restraints:-
- I have limited bandwith
- I have limited processing
But all in all, it works. I can:-
- Take a Pitest XML output and store it
- Extract data from Pitest XML
- Convert data to a single coverage percentage
- Return summerised data
- Compile this summerised data into a GitHub shield by a call to shield.io
### The Hook
I use Travis CI to do my build, this uses a yml file to do pre and post build actions. I can just add a
pitest command to my
after_success block in there and a
curl post to my Pitest CI endpoint containing the pitest XML in order to load the most recent run.
- ./gradlew check jacocoTestReport pitest ... - bash <(curl -X POST -d @build/reports/pitest/mutations.xml http://myPitestCI/report)
### The Display
Then it’s just a case of calling the
/shield endpoint in the README.md of my project to get a nice, proxied GitHub shield containing summerized Pitest data, as you can see below.