​

Improving Code Coverage With Devin

Most applications in production don’t have 100% code coverage. Engineering teams have to ruthlessly prioritize, and tend to focus on writing tests for critical paths to get the most benefit out of their time spent. Devin can bridge this gap by analyzing a codebase’s existing test suite style and increasing coverage of less critical but still important functions.

For this example, we’re going to take an open source Typescript sample application based on the RealWorld spec which already has some tests but not comprehensive coverage: https://github.com/SeuRonao/realworld-express-prisma

You can follow along with the full Run here or you can read through the rest of this post highlighting each step of Devin’s process. We’ll be highlighting various parts of Devin’s Workspace like the Editor or Shell. If you’re curious, you can read more about how you can interact with Devin’s Machine over in our docs.

We start off our Run with a simple prompt directing Devin to set up our dev environment and assess the existing code coverage of our application:

In its Shell Devin first clones our repo from GitHub, parses the README file, and installs our project dependencies in its local environment.

Devin then uses its built-in Editor to create our .env file so that the application can run locally. When working with Devin, you also have the ability to open VSCode yourself to perform manual edits or review the files Devin is working on.

Devin then runs the test suite in its Shell. One of the major benefits of Devin is that it not only can modify or create code for your team, it can also work directly in its Browser and Terminal to perform higher level interactions with your codebase, its dev environment, or even your running application.

We can see below how Devin interpreted and translated the results from its Shell into our chat so that it is more human readable and distilled down.

Now we can move on to the actual task at hand, increasing code coverage. It looks like profileViewer.ts is a good place for us to start, given its essentially nonexistent coverage by default. We prompt Devin on next steps, and it reports back to us on what is going on throughout the process.

Devin is able to read the existing file, determine what test cases we might want to implement based on the functionality that exists, and then actually write the new tests for us without any intervention. After it finishes implementing the new test cases, Devin gives us a final rundown of its changes:

It also shares the new file with us so that we can download and review it. Alternatively, we could have also asked Devin to create a Pull Request directly on GitHub with this new file so that it could be looked at as part of our normal code review process. I decide to just review it in Devin’s built-in Editor so that I don’t have to download the file.

Generally the changes look good but we want to make sure there’s no runtime bugs that we overlooked, so we ask Devin to run the test suite again and report back.

It runs as expected and we’ve increased our overall coverage from 28.57% of functions to 57.14% of functions, which is exactly what we wanted to achieve. The entire process took Devin less than 10 minutes of working time to implement even though we stepped away in the middle before giving Devin our next set of instructions. You can increase test coverage and remove a tedious task from your engineering team’s backlog just by delegating it to Devin.

If you find yourself frequently coming back to Devin to increase code coverage, you could even turn this prompt into a detailed Playbook so that you can easily kick off new Runs for different parts of your application. If you want to learn more about what kinds of Prompts Devin works well for, read up on some of our examples in the Docs.