How to fail a build on failing unit tests

Goal

Every time a push is made, run all unit tests. If tests fail, the build should fail and not be deployed.

Assumptions

During the build phase no services are available. That means only tests that can run with only your code (no database, no web requests, etc.) can be run. Most testing frameworks have a way to define test suites and run only a selected set of them. This guide assumes you have defined a test suite that is safe to run in isolation.

Although this example uses PHP the process is essentially identical for any language, provide its test runner follows standard Unix error conventions.

Steps

1. Ensure that your test framework is included in your list of project dependencies

For a PHP application that would mean composer.json. For a Node.js application it would mean package.json. For PHP:

{
    "requires-dev": {
        "phpunit/phpunit": "^7.5"
    }
}

(By default Platform.sh installs dev dependencies as well, so it will be downloaded by default.)

2. Add set -e as the first line of your build hook

In .platform.app.yaml, locate your build hook and add set -e as the first line. If one does not exist go ahead and create it.

hooks:
    build:
        set -e

That tells the system to fail the build if any command in the build hook returns an error code.

3. Add your test command as the last step of your build hook

After all other tasks in your build hook have run, add a line to run your tests. For PHP your build hook will look like this:

hooks:
    build:
        set -e
        # Possibly other stuff here
        vendor/bin/phpunit

For Node.js it will look like this:

hooks:
    build:
        set -e
        # Possibly other stuff here
        npm test

4. Commit the result and push

git add composer.json .platform.app.yaml
git commit -m "Enable build tests"

Conclusion

If the test command fails a test it will return a non-0 error code to the shell. The set -e flag means that error will cause the whole build process to fail and the container will not be deployed. If no tests fail then the test command will return 0, allowing the build to proceed.

The output of the test runner will be available in the build log.

1 Like

But what if my tests need access to a service, specifically a database?

Those don’t work in the build environment. (Also, they’re not technically unit tests at that point.)

If you need tests that run on a service, you could use a post-deploy hook and have it run only on non-master branches. That wouldn’t block the deploy of course, but it would still give you an automated test run.