AMQP/RabbitMQ - Process messages sequentially

Kanarie Piet

I have one direct exchange. There is also one queue, bound to this exchange.

I have two consumers for that queue. The consumers are manually ack'ing the messages once they've done the corresponding processing.

The messages are logically ordered/sorted, and should be processed in that order. Is it possible to enforce that all messages are received and processed sequentially accross consumer A and consumer B? In other words, prevent A and B from processing messages at the same time.

Note: the consumers are not sharing the same connection and/or channel. This means I cannot use <channel>.basicQoS(1);.

Rationale of this question: both consumers are identicall. If one goes down, the other queue starts processing messages and everything keeps working without any required intervention.

cantSleepNow

Usually the point of a MQ system is to distribute workload. Of course, there are some situations where processing of message N depends on result of processing the message N-1, or even the N-1 message itself.

If A and B can't process messages at the same time, then why not just have A or just B? As I see it, you are not saving anything with having 2 consumers in a way that one can work only when the other one is not...

In your case, it would be best to have one consumer but to actually do the parallelisation (not a word really) on the processing part.

Just to add that RMQ is distributing messages evenly to all consumers (in round-robin fashion) regardless on any criteria. Of course this is when prefetch is set to 1, which by default it is. More info on that here, look for "fair dispatch".

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at
0

Comments

0 comments
Login to comment

Related

Python prints messages not sequentially

If i send multiple messages to the same webworker, does it queue them up and process them sequentially?

Process request sequentially

ActiveMQ Artemis - handle messages sequentially

Is it possible to process two topics sequentially in Kafka?

How to process the results of a series of AJAX requests sequentially?

multiprocessing targets are run in the parent process instead, sequentially

socket.io how to send multiple messages sequentially?

Write to stdin of process listening for messages

Unable to process messages concurrently in MDB

Sending Messages with sleekxmpp hangs in process()

Process FIFO SQS Messages In Order

Process an array of API calls sequentially with Bluebird.mapSeries

Is there a way to process all client requests sequentially on a .NET Framework WebAPI?

How to read and process data from a set of text files sequentially?

How to process Avro messages while reading a stream of messages from Kafka?

Bash: receive messages from background process

How to use TestKit to not process messages sent to Self

Send messages from child process to parent

Read and process a batch of messages from Kafka

Process all messages before posting result

Excel and RabbitMQ - Process RabbitMQ Messages in Excel?

implement message handler in rabbitmq to process received messages

GCP Pub Sub: process messages by batches

TPL dataflow process N latest messages

How to process Websocket messages from client in Java?

Flink capture messages based on time window and process

Bash: receive messages from background process

Pipe kernel messages to another process as they occur