How to run an Anaconda/Miniconda Python stack on Platform.sh

How-to Guides
, ,
#1

Goal

To run Anaconda (the full data science Python stack) or Miniconda (just the Python package manager) on Platform.sh.

Assumptions

You will need:

Problems

Platform.sh offers several different versions of Python by default, but some applications require different environments, such as data science applications running on the Anaconda stack or packages installed with Miniconda. Running the Anaconda/Miniconda installer in the build hook of a Platform.sh application allows for the activation of conda virtual environments for script execution.

Steps

1. Set git remote to Platform.sh project

$ git init
$ platform project:set-remote <project id>

2. Create an installation script file

The Platform.sh build hook runs Dash, not Bash, so the installation script is executed separately in install_conda.sh:

#!/bin/bash

# Download an Anaconda3 release and name the file `conda.sh`

curl https://repo.anaconda.com/archive/Anaconda3-2019.03-Linux-x86_64.sh -o conda.sh

# If you wish to install Miniconda3 instead, comment out the line above and uncomment the line below:
# curl https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -o conda.sh

# Run the downloaded `conda.sh` script using bash
# — Use the `-b` flag to run the installer in batch mode
# — Use the `-p` flag to specify where the package manager will actually be installed
bash conda.sh -b -p $PLATFORM_APP_DIR/conda

# Source the following `conda.sh` file to put the `conda` command in our path for the duration of this script
. /app/conda/etc/profile.d/conda.sh

# Add the above command to `.bash_profile` so that it is available during SSH sessions
echo ". /app/conda/etc/profile.d/conda.sh" >> ~/.bash_profile
echo "conda activate" >> ~/.bash_profile

# Enter the base conda environment and prepare to configure it
conda activate base

# Update conda itself
conda update -n base -c defaults conda

# OPTIONAL: Print out debugging information in the build hook
conda info

3. Add a start script for running scripts with Anaconda

start_command.sh:

#!/bin/bash

# Source the `conda.sh` file to make this particular shell session able to run the `conda` command
. /app/conda/etc/profile.d/conda.sh

# Activate the conda environment that was created in the build hook
conda activate base

# Put any commands needed to run a web app here

Make sure to put any commands needed to run a web app at the end of this file.

4. Add .platform.app.yaml

Create the .platform.app.yaml configuration file:

# The name of this app. Must be unique within a project.
name: app

# The runtime the application uses.
type: "python:3.7"

# The hooks executed at various points in the lifecycle of the application.
hooks:
    build: |
      set -e

      # The build hook uses `dash` rather than `bash`.
      # Thus, the installation occurs in separate script that runs with `bash`.
      bash install_conda.sh

    deploy: |
      set -e

      # Make the start script executable
      chmod +x start_command.sh

# The size of the persistent disk of the application (in MB).
disk: 1024

# The configuration of app when it is exposed to the web.
web:
    commands:
        start: bash ./start_command.sh
    upstream:
        socket_family: tcp
        protocol: http
    locations:
        "/":
            passthru: true

5. Define routes

./.platform/routes.yaml

https://{default}/:
    type: upstream
    upstream: app:http

6. Add empty services

./.platform/services.yaml

# empty

7. Add, commit, and push these files to your Platform.sh project

git add .
git commit -m "Adding configuration to install conda environment"
git push platform master

Conclusion

By executing the Anaconda or Miniconda installation Bash scripts in the build hook, a project is able to install an Conda-based execution environment on Platform.sh.

#2
#3