Using platform.sh environments for continuous integration / functional tests

#1

What would the pros / cons of using platform to run integration / functional tests, instead of a typical Continuous Integration service? And is using platform to run functional tests possible / a bad idea?

My thinking is this:

  • Unit tests, static analysis, code linting etc. are easy to run in isolation, so that should remain in a typical CI environment
  • Functional tests require a functioning environment, which platform.sh can automatically provide
  • Sometimes you want to run your functional tests against production data, which is also something platform.sh can automatically provide

An example implementation could be:

  1. On build, detect if this is an environment you want to run tests in (eg. non-production, pull request environment etc.)
  2. Install any tools you need to run functional tests (eg. a headless browser and testing framework)
  3. Run the tests – fail the build if the tests fail
  4. Send notification of the passed/failed build (I think the standard github pull request integration would report this back, but I’m not sure – an email or slack notification could be sent)
1 Like
#2

Hi Daniel.

The build environment is branch-agnostic. Since build images can be reused on multiple branches there’s no way to detect the branch during build.

This is not something we currently support directly; you can run unit tests (that do not require access to the database) during build. More on that here: How to fail a build on failing unit tests

For end to end tests, you would need to ensure the tools are installed in all environments. Then in a post_deploy hook, check if you’re on a branch where you want to run tests (at minimum, non-master), and run whatever commands you need. It’s better to do in post_deploy than deploy because deploy blocks the environment being publicly accessible, which may be necessary depending on the nature of your tests.

The report-back in case of failure is the big challenge. At present we have no mechanism available to do that. If you have some external notification system that can accept a webhook, you can have your test code ping that URL when it’s done. We have no built-in support for it at this time, though.