Message Queues are just like queue data structures, it works on the principle of FIFO i.e. First in first out. Order is essential inside the queue.

Let us take a practical example of an E-commerce website like Amazon suppose we perform two operations i.e. place an order and send an email.
So we have millions of orders to process and send emails. So we can see the use of queue as in whenever we place an order in that same order, it will send out the email.

Technical Definition

A message queue is a form of asynchronous service-to-service communication used in serverless and microservices architectures. Messages are stored on the queue until they are processed and deleted. Each message is processed only once, by a single consumer. Message queues can be used to decouple heavyweight processing, to buffer or batch work, and to smooth spiky workloads. [Source]

Benefits of Message Queues

• Better Performance

• Increased Reliability

• Granular Scalability

• Simplified Decoupling

Use Case

Notification System -> So let's suppose we have many users interacting with a particular server so many events will be generated and each event will pushed to Queues. So now we will have a worker or we can say a processor that will process each of these events/messages and send notifications in the form of email, push notification or any form of notification channel.

What is a dead-letter queue?

So let us assume we have a producer and a worker that will consume these messages. Now worker will pick one message at a time and will process it. So suppose it picks up the first message and then processes it similarly it picks up the 2nd message and processes it now while processing 3rd message somehow the worker goes down due to some error now after a while worker comes up and starts processing 4th message and the processes it successfully. So finally we have processed 1st, 2nd and 4th messages.

So now we see a problem here i.e. we don't see the 3rd message which means we have lost this message as we don't have the 3rd message inside the queue to process it anymore.

So to tackle this problem one approach is we can handle it through our code whenever the worker gets an error while processing a message we will push back the message into the queue again so that whenever the worker comes up again it will now process the 4th message first and then process the 3rd message so in this way our message is not getting lost.

But in this case, also there is one problem that is somehow if message 3 is corrupted and every time the worker tries to process the 3rd message but fails and then again pushes the 3rd message back into the queue so in this way we are stuck in an infinite loop kind of situation which is also not good.

So to get rid of this infinite loop kind of situation we will have the same setup of a producer and a worker and will process the message as it's just in case the message is corrupted or worker throws an error while processing the message we will have another queue with the name as error queue and 3rd message will be pushed into this error queue and worker will continue its processing and finally once the worker is done processing the main queue it will try to process 3rd message inside the error queue and if 3rd message is processed successfully well and good otherwise if it still fails to process the 3rd message it will simply discard the message.

so we are doing at least one retry for a failed message and it's not going into an infinite loop kind of situation as well.

So to sum up this error queue is known as a dead-letter queue.

Technical Definition

A dead-letter queue (DLQ) is a special type of message queue that temporarily stores messages that a software system cannot process due to errors. Message queues are software components that support asynchronous communication in a distributed system. They let you send messages between software services at any volume and don’t require the message receiver to always be available. A dead-letter queue specifically stores erroneous messages that have no destination or which can’t be processed by the intended receiver. [Source]

Benefits of Dead-Letter Queue

Reduced communication costs - Regular or standard message queues keep processing messages until the retention period expires. This helps ensure continuous message processing and minimizes the chances of your queue being blocked. However, if your system processes thousands of messages, a large number of error messages will increase communication overhead costs and burden the communication system. Instead of trying to process failing messages until they expire, it’s better to move them to a dead-letter queue after a few processing attempts. [Source]

Improved troubleshooting - If you move erroneous messages to the DLQ, this lets your developers focus on identifying the causes of the errors. They can investigate why the receiver couldn't process the messages, apply the fixes, and perform new attempts to deliver the messages. For example, a banking software might send thousands of credit card applications daily to its backend system for approval. From there, the backend system receives the applications but cannot process all of them because of incomplete information. Instead of making endless attempts, the software moves the messages to the DLQ until the IT team resolves the problem. This allows the system to process and deliver the remaining messages without performance issues. [Source]