Using Docker Compose with web apps
Inspiration
I recently got to set up a multi-service web app with Docker Compose. I found that Docker Compose handled most of the problems that I've ran across when trying to "dockerize" web apps that have multiple components. One example of that is Pulldasher, a GitHub pull request dashboard written by iFixit. Pulldasher is made up of an Express (Node.js) application and a MySQL database. Making a container around the Express application seemed simple enough, but creating a setup such that I can have a MySQL container spin up and tear down in sync with the Pulldasher container seemed like something I would have to automate myself using a shell script.
Introduction to Docker Compose
Up and Running
I wasn't aware of Docker Compose until I had to set up a CallPower instance for Repair.org. CallPower is an open source service that lets advocacy groups help constituents contact each of their representatives at a State or Federal level. The service ties together data from the OpenStates API and Sunlight Foundation and uses Twilio to connect constituents with their representatives. CallPower includes a Dockerfile
which lets you build a container around the main application, which is a Flask app. It also includes a lightweight docker-compose.yml
file, which specifies a single service named calltool, along with how to build that service's Docker image, which ports to forward, and which environment varibles to start the service with. As is, the docker-compose.yml
will build an image for you with the name <current-directory>_<service-name>
, start a container from that image named <current-directory>_<service-name>_1
, forward the container's port 5000
to your host machine's 5000
and start the Flask app with the environment variables given. This all happens by running docker-compose up
in the same directory as the docker-compose.yml
file. Pretty cool.
Here's a copy of the example docker-compose.yml
given with CallPower.
version: '2'
services:
calltool:
build: . # tells docker-compose where to look for the Dockerfile to build the calltool Docker image.
environment: # Environment variables for the Flask app
FLASK_ENV: development # or development-expose or production
SUNLIGHT_API_KEY:
TWILIO_DEV_ACCOUNT_SID:
TWILIO_ACCOUNT_SID:
TWILIO_DEV_AUTH_TOKEN:
TWILIO_AUTH_TOKEN:
SECRET_KEY:
ADMIN_API_KEY:
CALLPOWER_CONFIG: call_server.config:DevelopmentConfig # or call_server.config:ProductionConfig
APP_HOST: 0.0.0.0
ports:
- "5000:5000" # Map the container's port 5000 to the host's port 5000
Going Full Container
While that's pretty neat for starters, Callpower requires a Redis server for caching and a SQL database for use with SQLAlchemy in the Flask app. Connection URLs for these servers can easily be passed to the calltool container through environment variables, so if you have a Redis and MySQL/Postgres/sqlite server somewhere on the internet or your LAN to connect to, that's all you need to get started. However, if you'd like to be able to set up everything the application needs on your own host, you'll need to add Redis and SQL database services to the docker-compose.yml
.
MariaDB with Volumes
Callpower requires a SQL database to store Admin credentials, call campaign information, call statistics, and voice recordings to play in each campaign phone call. For this example, we'll add another service to the docker-compose.yml
file to tie together with our calltool service defined previously. We'll need to specify a service name, Docker image, and list of environments variables to start the database with. For this example we'll use MariaDB/MySQL.
version: '2'
services:
calltool:
build: . # tells docker-compose where to look for the Dockerfile to build the calltool Docker image.
environment: # Environment variables for the Flask app
FLASK_ENV: development # or development-expose or production
SUNLIGHT_API_KEY:
TWILIO_DEV_ACCOUNT_SID:
TWILIO_ACCOUNT_SID:
TWILIO_DEV_AUTH_TOKEN:
TWILIO_AUTH_TOKEN:
SECRET_KEY:
ADMIN_API_KEY:
CALLPOWER_CONFIG: call_server.config:DevelopmentConfig # or call_server.config:ProductionConfig
APP_HOST: 0.0.0.0
ports:
- "5000:5000" # Map the container's port 5000 to the host's port 5000
db:
image: "mariadb:latest" # Use the latest MariaDB Docker image
volumes:
- db_data:/var/lib/mysql # Mount the db_data volume onto /var/lib/mysql
environment: # Set the MySQL environment variables
MYSQL_ROOT_PASSWORD:
MYSQL_DATABASE:
MYSQL_USER:
MYSQL_PASSWORD:
volumes:
db_data: # Define the db_data volume
There are a couple things of note in the new docker-compose.yml
file.
- db is defined as another service
- db has an image attribute instead of a build attribute. This is because we can grab the default MariaDB image.
- db has a volumes attribute that specifies that a db_data volume is mounted to
/var/lib/mysql
in the container - There is a new top level attribute called volumes, which contains a simple volume called db_data.
Now that we've defined a db service, it will be brought up and shut down whenever we run docker-compose up/down
. Another very useful aspect is that the calltool service can contact the db service using the service name as a domain name. For example, if you defined a MySQL connection string to tell the Flask app where to find the MySQL database, you could give it mysql://<user>:<pass>@db/<db_name>
. Through some DNS magic (probably DNS?), the db
domain name will resolve to the container's IP address. Something else that's important when using a database is to make sure it's storage doesn't disappear when you shut it down. Without defining a volume, the filesystem used by the db service would be deleted on shutdown and all of your data would be gone. To specify that we long storage that exists outside of a container's lifespan, we add a volume. The volume configuration above is all you need to define a simple volume and mount it on your container when it starts up. If you'd like to see where that volume actually exists on your host filesystem, you can run docker volume <volume_name>
. The volume name in this case should be callpower_db_data
.
The output should look like this:
[
{
"Name": "callpower_db_data",
"Driver": "local",
"Mountpoint": "/var/lib/docker/volumes/callpower_db_data/_data",
"Labels": null,
"Scope": "local"
}
]
The filepath described in Mountpoint
is where the volume exists outside of your container. At this point running docker-compose.yml
will bring up the callpower_calltool_1
container and the callpower_db_1
container.
Redis
The last running service needed is Redis. For development purposes, you can add a service named redis with and image value of redis:alpine
and your Flask application will be able to reach it at redis://redis:6379
, without any further configuration. Just like with the db service, your Flask app will be able to reach this container over the network by the domain name redis
. With this configuration, the container won't be exposed to the internet, but if you're planning on running this service anywhere other than your laptop you should take measures to secure it with a strong password.
The final docker-compose.yml
looks like:
version: '2'
services:
calltool: # calltool is the container that runs the Flask application
build: . # tells docker-compose where to look for the Dockerfile to build the calltool Docker image.
environment: # Environment variables for the Flask app
FLASK_ENV: development # or development-expose or production
SUNLIGHT_API_KEY:
TWILIO_DEV_ACCOUNT_SID:
TWILIO_ACCOUNT_SID:
TWILIO_DEV_AUTH_TOKEN:
TWILIO_AUTH_TOKEN:
SECRET_KEY:
ADMIN_API_KEY:
CALLPOWER_CONFIG: call_server.config:DevelopmentConfig # or call_server.config:ProductionConfig
APP_HOST: 0.0.0.0
ports:
- "5000:5000" # Map the container's port 5000 to the host's port 5000
redis: # redis is the container that runs redis
image: "redis:alpine"
db: # db is the container that runs MariaDB/MySQL
image: "mariadb:latest" # Use the latest MariaDB Docker image
volumes:
- db_data:/var/lib/mysql # Mount the db_data volume onto /var/lib/mysql
environment: # Set the MySQL environment variables
MYSQL_ROOT_PASSWORD:
MYSQL_DATABASE:
MYSQL_USER:
MYSQL_PASSWORD:
volumes:
db_data: # Define the db_data volume
Running docker-compose up
should start three containers named callpower_calltool_1
, callpower_redis_1
, and callpower_db_1
. Running docker-compose down
will stop each of the containers.
Takeaways
docker-compose
is a pretty simple way of specifying and organizing multiple services under a single configuration. Since most of the web apps I work on have at least a backend application and SQL server component, I'm planning on using this to more easily do web development and testing on my laptop, without needing to stay connected to a development server in the cloud. I'll probably do a follow up to my Dockerizing Pulldasher post with a how-to Pulldasher setup with docker-compose
.