Using environments for continuous integration / functional tests

What would the pros / cons of using platform to run integration / functional tests, instead of a typical Continuous Integration service? And is using platform to run functional tests possible / a bad idea?

My thinking is this:

  • Unit tests, static analysis, code linting etc. are easy to run in isolation, so that should remain in a typical CI environment
  • Functional tests require a functioning environment, which can automatically provide
  • Sometimes you want to run your functional tests against production data, which is also something can automatically provide

An example implementation could be:

  1. On build, detect if this is an environment you want to run tests in (eg. non-production, pull request environment etc.)
  2. Install any tools you need to run functional tests (eg. a headless browser and testing framework)
  3. Run the tests – fail the build if the tests fail
  4. Send notification of the passed/failed build (I think the standard github pull request integration would report this back, but I’m not sure – an email or slack notification could be sent)
1 Like

This seems like a reasonable approach, rather that what I’ve suggested: