Interactively debug python applications running on Platform.sh.
- An active Python application running on Platform.sh
- A configured SSH key on the project account
- Familiarity with the Python debugger (pdb)
- Optional: a Platform.sh Django template application
Effectively debugging web applications isn’t trivial, especially when an HTTP request goes through multiple layers until it reaches the web app, which is usually the case in cloud platforms.
Debugging in a local environment is easier, but the bug might only trigger when interacting with data present on production.
Note: The debugging procedure should only be performed on a non-production environment. So, if necessary clone the production environment, with the data too if need be, depending on the preconditions that trigger the bug you’re investigating.
The existence of bugs is sometimes manifested by an unhandled exception being raised and bubbling up to the top of your application, or in other cases, there are no exceptions, but the actions that a request handler is supposed to perform do not occur the way you’d expect.
Let’s explore the different set of debugging tools we can use and learn when to use what.
Sometimes identifying a bug is possible by just looking at the stack trace of the exception or getting the values of some variables at some stack in the backtrace.
This is exactly what the DebuggedApplication WSGI wrapper from the werkzeug library does. To use it:
$ pipenv install werkzeug
Add the script
wsgi-debug.py to the project directory. It wraps our WSGI application with
DebuggedApplication and exposes it.
import werkzeug.debug # For the template, it is assumed wsgi.py is in project/myapp/, # edit to match your project accordingly. import myapp.wsgi application = werkzeug.debug.DebuggedApplication( myapp.wsgi.application, evalex=True, )
Add the script
gunicorn-debug.sh that stops the “app” service in the environment and spawns a new
gunicorn instance that’s pointed to the wrapped WSGI application.
#!/usr/bin/env sh et -ex # Stop the application. sv stop app # When this script exits, start it again. trap "exit 0" INT trap "sv start app" EXIT # Run a gunicorn and point it towards the wrapped WSGI application. # It's important to run in sync mode, and a single worker, because werkzeug # DebuggedApplication doesn't like forking. # Set a high enough timeout that gives us time to debug. gunicorn \ --worker-class sync \ --workers 1 \ --timeout 3600 \ --bind unix:$SOCKET \ wsgi-debug:application
And make it executable.
$ chmod +x gunicorn-debug
Tell Django to let exceptions bubble up, by setting these in your
DEBUG = False DEBUG_PROPAGATE_EXCEPTIONS = True
After the deployment succeeds, ssh into the environment and run the
Now issue the request that makes it raise the unhandled exception, and lo and behold, a beautiful debugging interface appears.
The stack trace of the exception can be seen when a line of code is hovered over, and a “shell” button appears at the right that will spawn a Python shell at that line where variables (or anything else can be run, for that matter) can be printed.
Note: A PIN code is required to enable the shell, which is printed in the console of the
gunicorn-debugscript. This is for security reasons, to prevent unauthorized access in case it is accidentally enabled on production.
The previous approach will help to understand why an exception is raised so that it can be fixed, but if the bug occurs at an intermediate stage before the exception is raised, this will not be enough. The
werkzeug debugging interface only captures the latest state of the app and the values of each variable (i.e., it cannot go back in time).
To gain extra insight into what code path is entered when the request is handled, a pdb trace point can be set at some point in the code, and then interactively execute it line by line and inspect variables as you go. See the pdb official documentation for more information.
While still running the application with our
gunicorn-debug script, add a pdb trace (
import pdb; pdb.set_trace()) at some point in the code, which can even be inside a conditional.
For example, to test this methodology, the following can be included in a Django application’s
if request.GET.get('pdb', False) == '1': import pdb; pdb.set_trace()
Issue a request that makes it reach the
pdb.set_trace() and then code execution will pause and the PDB shell will appear in the terminal where the
gunicorn-debug script is running.
From there, pdb commands can be issued, for example, to execute code line by line, to print variables, or run any arbitrary code desired.
Debugging complex bugs that are only reproduced with the data that are present on production is non trivial, but thanks to development environments data cloning capabilities, and the dynamic nature of Python, we have great tools at our disposal to properly investigate these kind of bugs.