Avoiding the horrors of testing with Codebots

by Rosie Odsey, Sep 29, 2017

If you had asked me what testing was two years ago, I would have responded with a look of mild terror. “Testing? I should know what that is. Everyone talks about it.” (disclaimer: that would almost definitely have been internal monologue). Thankfully, I’m not a software developer and I haven’t built programs. Having spent time with a lot of people who have, it’s amazing that there is still debate about whether you should test, and how.

What can go wrong?

I shudder at just how easy it is for things to go wrong with software creation. Imagine developing a new ERP payroll system, only to find it overpays employees $53 million. That was the case for one large urban school system, who was tied up in lawsuits up to two years later attempting to recover the overpayments. The cause of the problem? Inadequate testing.  
British retailer J Sainsbury’s horror story is the perfect example of testing gone wrong. The company had to write off its $526 million investment in an automated supply-chain management system when merchandise was stuck in company depots and warehouses instead of getting through to stores. 

So... what do people do?

Option 1: no tests

This is a cowboy.
This is your friend who texts and drives.
This is the time you started your assignment the night before it’s due.
This is recklessness.
If you think this doesn’t apply to you, then please tell me the best thing you’ve pulled off without tests and I will be truly impressed at your luck.
(side note: Haskell programmers, before you argue, yes you have an expressive type-system, but there is a reason HUnit exists).

Option 2: manual tests

This is old school.
It’s less thorough because human error is involved.
The person who built the thing can’t do the QA of the thing.
There is a lot of fatigue because the tasks can be tedious and repetitive.
It’s usually not reusable for future versions, let alone for other apps.

Option 3: automated testing

Automated testing is the way the world is going? This is because:
  • It finds more bugs. Automating the testing allows the bots to find the bugs and the humans to focus on the important work: getting rid of them.
  • It’s faster. “I could have a few days off in the time it would take to rewrite and run unit tests by hand every time we update the app.” - Naveen Kumar (chief tester).
  • You can create reusable code. We rewrite a lot of the same tests for newer iterations and different products. This way, the bot can pull a lot of that for us.

...and Codebots does even more!

The Codebots platform also adds traceability, better ways to find issues, and makes automated testing easy. In addition to this, there are dashboards so that project managers can have a helicopter view. Lastly, everything is automatically committed to a repository so you can interact there and use your own tooling.


Codebots uses a traceability matrix to create a relationship between a user story and a test. Based on that relationship, the platform shows you which tests are passing and which user stories have been successfully implemented. It’s powerful at a managerial level as managers are able to gain an overview of the software’s health.

Finds the issue

When a test fails the developer knows there’s a problem. What they won’t necessarily know is what that problem is. The beauty of Codebots is that it drills down to the exact stage that the test fails. This creates transparency and allows developers to rectify the problem much faster than before.

Testing made easy

A key feature of the platform’s testing function is its use of Cucumber testing. Cucumber reads code written in plain English and tests the application based on defined scenarios. Given I have X, when I do Y, then I expect Z to happen. These tests are translated from the project’s user stories. Codebots uses Blockly to create these testing scenarios. Blockly is a visual code editor, where you can move the blocks around to alter the test. The best part? It’s incredibly simple to use. It was initially targeted at kids (Scratch games), so anyone can use it.


This is an awesome feature for all the project managers out there. Codebots’ testing dashboard displays the number of tests run, the number of tests failing and the number of user scenarios without tests assigned to them. It also gives a representation of the software’s health based on the data the platform collects.


Everything is automatically committed to a repository. This allows developers to work on the code in their preferred application (Atom, Eclipse, or IntelliJ, for example). It also shows the flexibility of Codebots. A developer can use the platform while still retaining familiarity with their existing applications.
The Codebots platform will be released in 2018. Click here for early access.