Message Return Mechanism

This is a proposal for replacing the current message return-to-publisher mechanism with a mechanism based on the exchange-binding-queue model.

Basic proposal

1. Drop Basic.Return from the protocol.
2. Add 'return_key' property to the message.
3. Add new 'return' exchange type routing on return_key instead of routing_key.
4. Add new system exchange called 'amq.return' of 'return' type.
5. Specify that all undeliverable messages will be redirected to 'amq.return' exchange.

Request/reply use case

In this use case application sends a message and want to get either a reply from other party or a rejected message returned in case other party is offline/unfunctional.

1. Create a temporary queue for rejected messages (thus you will get broker-wide unique name)
2. Bind the queue to 'amq.return' exchange with condition 'return-key = return-queue-name'
3. Consume from the queue
4. When sending a message set its return key to return-queue-name.

Dead-letter queue

This use case shows how a queue that stores undeliverable messages can be implemented.

1. Declare a persistent queue.
2. Bind it to 'amq.return' exchange with condition 'return-key = *'

Discussion

Mandatory flag

IMO this flag is misplaced. What it's saying is 'turn return delivery of undeliverable messages on/off'. This should not be a part of Basic.Publish command, in fact it shouldn't be visible from message publishing application at all. Consider dead-letter queue scenario. Message publisher shouldn't be able to say "don't pass this message anywhere in case it cannot be delivered (including dead-letter queue)." Specifying what should be stored in dead-message queue is an administrative task and is not a responsibility of message sender.

In fact, in current proposal there's no need for the flag at all. You either want undeliverable messages back, in which case you'll subscribe to amq.return exchange, or you don't, in which case you simply don't subscribe.

Immediate flag

Immediate flag is a different story. It specifies how a queue should treat the message, so it should be grouped with properties like 'expiration' and 'priority', i.e. basic message properties. (Maybe we can consider 'immediate' being zero expiration, however, expiration is a property ported directly from JMS so it would require some investigation.)

However, it should be specified more clearly, what does 'immediate' mean.

  • Does it mean that that the message will be sent back when it cannot be delivered to any consumer system-wide.
  • Or does it mean that every queue that gets the message and cannot deliver it to a consumer will send it back separately ?

I, personally, would vote for b. option.

How should 'return' exchange work

Consider a scenario where you would like to subscribe for all rejected messages from a specific exchange. That would require to create a binding that matches the messages not only on 'return_key' property, but also on 'exchange' property.

This example demostrates a need for more complex matching then one on 'return_key'. Rejected message should contain several properties and 'return' exchange should allow matching on these. Here are some examples:
1. exchange name
2. rejection reason code (unroutable, no-consumers, expired)
3. queue name
etc.

Question: What happens with message properties when it is routed to 'return' exchange? Are they left untouched or modified somehow? (immediate, expiration, priority…)

Strip-content property

I would suggest to add one more property to a message. Property called 'strip-content' that would be used by return exchange. It would mean that message body has to be truncated when passing it through return exchange.
This way we can avoid returning large amount of data to sender in cases where the only thing sender cares about is whether the message was delivered or not.

Meta-rejection

What happens when a message is rejected by 'return' exchange? Dropped in all cases? Anything else that can be done with it?

Identifying AMQP broker world-wide

In our clustering architecture there's a need to identify sender application instance uniquelly so that returned messages can be delivered to it unambiguosly. Same issue will probably arrise in other use cases as well.

We would like to use '<application-instance-name>@<globally-unique-broker-name>' syntax for this. Maybe even some global registration of server names should be considered. However, this is an issue for a separate discussion.

Comments

Add a New Comment

Add a New Comment