Distributed Python

Build your Celery (the distributed task queue, not the vegetable) skills with weekly tutorials,
how-tos and articles about common gotchas.

November 06, 2018

Estimated reading time: 4 minutes

Quick Guide: Custom Celery Task Logger

Posted by Bjoern Stiel

I previously wrote about how to customise your Celery log handlers. But there is another Celery logger, the celery.task logger. The celery.task logger is a special logger set up by the Celery worker. Its goal is to add task-related information to the log messages. It exposes two new parameters:


October 26, 2018

Estimated reading time: 8 minutes

Celery Execution Pools: What is it all about?

Posted by Bjoern Stiel

Have you ever asked yourself what happens when you start a Celery worker? Ok, it might not have been on your mind. But you might have come across things like execution pool, concurrency settings, prefork, gevent, eventlet and solo. So, what is it all about? How does it all fit together? And how is it related to the mechanics of...


October 13, 2018

Estimated reading time: 5 minutes

Monitoring a Dockerized Celery Cluster with Flower

Posted by Bjoern Stiel

A flower, sometimes known as bloom or blossom, is the reproductive structure found in flowering plants. Celery is a marshland plant in the family in the family Apicaeae that has been cultivated as a vegetable since antiquity. A docker is a waterfront manual laborer who is involved in loading and unloading ships, trucks, trains or airplanes – Wikipedia.


September 28, 2018

Estimated reading time: 6 minutes

Custom Celery task states

Posted by Bjoern Stiel

Celery tasks always have a state. If a task finished executing successfully, its state is SUCCESS. If a task execution resulted in an exception, its state is FAILURE. Celery knows six built-in states:


Get distributed Python tips straight to your inbox