Prerequisites
To follow along, you will need:
- Python installed on both the client and worker machines.
- Celery and a message broker (RabbitMQ) installed. Redis will be used as the result backend.
- Basic knowledge of Python and familiarity with Celery.
Step 1: Setup the Worker
First, let’s set up the Celery worker. On the worker machine, create a file named tasks.py:
from celery import Celery
app = Celery("tasks", broker='amqp://username:password@localhost',
backend='redis://localhost:6379/0')
@app.task(name='celery_project.tasks.add')
def add(x, y):
return x + y
Here, we define a simple task named add that takes two arguments and returns their sum. Adjust the broker and backend URLs to point to your actual RabbitMQ and Redis services.
Step 2: Start the Celery Worker
Run the following command on the worker machine to start the Celery worker:
.venv\Scripts\python.exe -m celery -A tasks worker --loglevel=info -E --pool=solo
This command starts a Celery worker that listens for tasks to execute.
Step 3: Setup the Client
On the client machine, you don’t need the full task definitions—only the Celery app configuration and the task signatures. Create a file named main.py:
from celery import Celery
app = Celery("tasks", broker='amqp://username:password@localhost',
backend='redis://localhost:6379/0')
result = app.send_task('celery_project.tasks.add', args=[4, 4])
print(result.get())
Here, send_task is used to dispatch the task. It requires the name of the task (which must match the name given in the worker’s task decorator) and the arguments for the task.
Step 4: Calling the Task from the Client
Run the main.py script on the client machine:
python main.py
This script sends the add task to the worker machine via the message broker, and then fetches the result using result.get().
Or Use Minimal Task Definitions approach
On the client side, you only need a minimal definition of the tasks to send them. You can redefine the tasks in a simple module that just includes the task names, without their implementations:
client_tasks.py:
from celery import Celery
app = Celery('client_tasks', broker='pyamqp://guest@your_broker_ip//')
@app.task(name='your_module_name.tasks.add')
def add(x, y):
pass # Implementation is not needed on the client
Then on the client:
from client_tasks import add result = add.delay(4, 4) print(result.get(timeout=10))