Questions tagged [message-queue]
A message queue is a service that receives events or messages from producers and provides them to consumers.
259 questions
3 votes
3 answers
557 views
Design question for exactly-once processing in a message-driven system using a unique ID
To achieve exactly-once processing where messages are consumed from a queue with at-least-once delivery, many sources (e.g. here and here and here) suggest attaching a unique ID to messages in the ...
- 139
0 votes
5 answers
170 views
How can I measure which messages are causing queue backups?
Consider the following multi-producer, multi-consumer system: For simplicity, assume that we only have one queue Messages are dequeued on a first-in-first-out basis Oldest-message-pickup-latency has ...
- 119
3 votes
3 answers
259 views
Handling rate limits / delays in consumers without affecting performance of other operations
I have a producer that generates a batch job that consists of multiple operations (approx. 100 - 10000). These operations can be processed in any order, ideally as fast as possible. The processing of ...
- 141
7 votes
5 answers
446 views
What can I do to get a message processor to slow down the rate of writes that it is making to a database?
We have this architecture: queue -> message processor (horizontal scaling) -> RDBMS Sometimes external systems dump 10k messages onto the queue and the message processor of course dutifully ...
- 229
2 votes
4 answers
311 views
Ensure concurrent processing based on a message key
I've been wondering about the following: Say you have an async message/task and you want that message only processed one at the time. for example: Only process one order for each customer at the time. ...
- 174
0 votes
1 answer
94 views
Design a sequential processing of records
About 10 to 15 records are processed per day with the time interval of 5 minutes between each record. System A inserts a record in DB and sends id of that record to active mq. System B Listener ...
- 119
2 votes
0 answers
139 views
parent correlation id in message with rabbit queue and open telemetry
I have a rabbit queue that I use to communicate between applications. I have a chain of three message. App 1 generate and push a message in rabbit(message 1) App 2 catch the message and generate ...
- 307
0 votes
1 answer
152 views
Is RabbitMQ suitable for allowing SaaS customers to subscribe to events?
I work on a SaaS system which is generally a single-tenant environment. Customers (whom the SaaS instances are for) would like a way to subscribe to events, such as updates of certain data entities. ...
- 101