Distributed Python

Build your Celery (the distributed task queue, not the vegetable) skills with weekly tutorials,
how-tos and articles about common gotchas.

October 13, 2018

Estimated reading time: 5 minutes

Monitoring a Dockerized Celery Cluster with Flower

Posted by Bjoern Stiel

A flower, sometimes known as bloom or blossom, is the reproductive structure found in flowering plants. Celery is a marshland plant in the family in the family Apicaeae that has been cultivated as a vegetable since antiquity. A docker is a waterfront manual laborer who is involved in loading and unloading ships, trucks, trains or airplanes – Wikipedia.

Read more...


October 01, 2018

Estimated reading time: 4 minutes

Celery, docker and the missing startup banner

Posted by Bjoern Stiel

Have you ever dockerized your Celery app and wondered where the Celery worker bannker in the docker log output went? Usually, when you start your Celery worker, it comes up with a startup screen that displays useful information like version, broker, result backend, concurrency and subscribed queues.

Read more...


September 28, 2018

Estimated reading time: 6 minutes

Custom Celery task states

Posted by Bjoern Stiel

Celery tasks always have a state. If a task finished executing successfully, its state is SUCCESS. If a task execution resulted in an exception, its state is FAILURE. Celery knows six built-in states:

Read more...


September 25, 2018

Estimated reading time: 5 minutes

Parallel Monte Carlo with Dask

Posted by Bjoern Stiel

Today, we will look at how to implement a Monte Carlo simulation to value an Asian option in a Jupyter notebook; and how to distribute that Monte Carlo simluation across a cluster using Dask. Don’t worry if you are not a Quantitative Analyst, this blog is more about Dask, from dask import delayed and exciting cluster stuff than about financial...

Read more...


September 14, 2018

Estimated reading time: 3 minutes

Concurrency and Parallelism

Posted by Bjoern Stiel

Concurrency is often misunderstood and mistaken for parallelism. However, concurrency and parallelism are not the same thing. But why should you care? All you want is to make your Python application fast and responsive. Which you can achieve by distributing it across many CPUs. What difference does it make whether you call it concurrency or parallelism?

Read more...


Get distributed Python tips straight to your inbox