6-DMP - Direct Message Protocol

This document describes an AMQP extension protocol that improves the performance of message transfer in common high-volume scenarios.

  • Name: gro.pqma.ikiw|PMD-6#gro.pqma.ikiw|PMD-6
  • Editor: Pieter Hintjens <moc.xitami|hp#moc.xitami|hp>
  • Contributors: none.
  • State: stable.

License

Copyright (c) 2008 iMatix Corporation.

This Specification is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version.

This Specification is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, see <http://www.gnu.org/licenses>.

Change Process

This document is governed by the gro.pqma.ikiw|SSOC-1#gro.pqma.ikiw|SSOC-1 specification.

Goals

The goals of the DMP framework are:

  • To improve the performance of common messaging scenarios.
  • To be simple to implement by client stacks.

Architecture

Overview

The Direct Message Protocol (DMP) connects client applications directly to message sinks and feeds on a server. In AMQP a sink is an exchange and a feed is a server-side private queue. The goal of DMP is to optimise the path through a server through eliminating various intermediaries:

  1. No parsing or protocol processing for messages sent to a sink; the messages can be passed to the sink at full speed.
  2. No internal queueing for outgoing messages; the broker feed can forward these at full speed to the client application.

DMP works as an out-of-band transfer for AMQP connections. It works in certain common high-performance scenarios:

  1. When applications publish to a low number (usually 1) of exchanges.
  2. When applications consume from low number (usually 1) of private queues.

DMP is not applicable for workload distribution scenarios (which depend on shared queues).

Operations

DMP is initiated by the client, which requests to connect to a sink or a feed on the server. Connections to sinks are non-exclusive, i.e. many clients can connect to the same sink. Connections to feeds are exclusive, i.e. only one client can connect to a feed at once.

Each connection is implemented as a physical TCP/IP connection. DMP is not designed for applications that work with large numbers of sinks or feeds at once.

DMP connections are independent and can be opened and closed at any time. They are forked off a principle protocol connection (the AMQP connection in the case of AMQP) which performs authentication and wiring, i.e. creation of sinks and feeds on the server. DMP connections do not close with the principle protocol connection. Any server-side entities that are lifespan-tied to the application will die when the last connection (principle or DMP) is closed.

Implementation

Integration with AMQP

For AMQP, DMP is implemented as an ASL class called Direct. The Direct class has two methods:

  • Put, which addresses a sink on the server. Put has one field, the sink name.
  • Get, which addresses a feed on the server. Get has one field, the feed name.

Both methods are synchronous. Their reply methods provide one field, a lease.

When the server responds with Put-Ok or Get-Ok, the client opens a new socket connection and performs this dialog:

C:AMQP%d10.1.0.1                Protocol-ID = 10, instance = 1, version = 0.1
S:OCTET 200 OK                  200 OK DMP/0.1
C:OCTET lease                   lease
S:OCTET 200 OK                  200 OK Ready to write to "somesink"

Following the protocol header, all commands and responses are short strings (1-octet length followed by 0-255 characters). All server responses are formatted as a 3-digit response code, a short response indicator, and optionally a textual explanation.

Server-side configuration

The server MAY announce its preference for DMP connections when sending a Connection.Start-Ok response. It does this by setting an integer field called "direct" in the server-properties table, with the value 1. Clients SHOULD interpret this field as a request that DMP be used where possible. This allows server-side configuration of DMP for a network of applications. Individual clients MAY ignore the server's stated preference.

Response codes

DMP uses standard IETF-style response codes:

  • 200 OK - The command completed successfully
  • 402 BAD-LEASE - Lease not valid for this client
  • 403 NO-ACCESS - Command not allowed for this client
  • 404 NOT-FOUND - Sink or feed not found
  • 502 SYNTAX-ERROR - Illegal values for one or more fields
  • 503 INVALID - Command was invalid
  • 530 NOT-ALLOWED - Command prohibited by server
  • 540 NOT-IMPLEMENTED - Command is not implemented by server
  • 541 INTERNAL-ERROR - Server is unable to complete command

Sending messages

For Put, the dialog then continues:

C:message-data

Receiving messages

For Get, the dialog then continues:

S:message-data

Message envelope and format

Message data is formatted on the wire as follows:

[nnnn]                          size of following message data, in network order
[n][exchange]                   exchange name (AMQP short string)
[n][routing key]                routing key (AMQP short string)
[ff][property...]               header flags + properties (AMQP format)
[n]                             options octet (Basic.Publish flags)
[nnn][body]                     content body (length + data)

The envelope for a DMP message is 12 octets plus content header fields. We plan a thinner envelope for high-performance (small message, market data) scenarios.

The options octet allows the encoding of options for Basic.Publish methods. The format of this byte is:

[0][0][0][0][0][0][mandatory][immediate]

Where 'immediate' is the lowest bit. An encoding of the options byte in C is as follows:

options = ((mandatory & 1) << 1) | (immediate & 1);

Heartbeating

For heartbeating purposes, peers can send null messages consisting of:

[0000]                          four zero octets

Peers MUST ignore null messages. Note that to effectively heartbeat a TCP/IP connection, peers must at a minimum attempt regular reads or writes to the connection. Normally either a read or a write attempt will fail immediately if the other peer has closed their end of the connection. A server that implements DMP needs to heartbeat only outgoing (feed) connections, and it can do this by writing null messages at regular intervals.

Appendix

This is the Direct class used for DMP:

<?xml version="1.0"?>
<class
    name    = "direct"
    handler = "connection"
    index   = "61500"
  >
  Direct Message Protocol class.
<doc>
    Implements the Direct Message Protocol specifications <4-DMP@wiki.amqp.org>.
    For more information see wiki.amqp.org.
</doc>

<doc name = "grammar">
    direct              = C:PUT S:PUT-OK
                        / C:GET S:GET-OK
</doc>

<chassis name = "server" implement = "MAY" />
<chassis name = "client" implement = "MAY" />

<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->

<method name = "put" synchronous = "1" index = "10">
  request a connection to a sink
  <doc>
    Requests non-exclusive access to the specified sink.
  </doc>
  <chassis name = "server" implement = "MUST" />
  <response name = "put-ok" />
  <field name = "sink" type = "shortstr" >
    Sink name
    <doc>
    Specifies the name of the sink, which must exist on the server.
    </doc>
  </field>
</method>

<method name = "put-ok" synchronous = "1" index = "11">
  grant access to the sink
  <doc>
  </doc>
  <chassis name = "client" implement = "MUST" />
  <field name = "lease" type = "shortstr">
    Lease for connection
    <doc>
    Holds a lease which the client must use when opening the direct 
    protocol connection.
    </doc>
  </field>
</method>

<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->

<method name = "get" synchronous = "1" index = "20">
  request a connection to a feed
  <doc>
    Requests exclusive access to the specified feed.
  </doc>
  <chassis name = "server" implement = "MUST" />
  <response name = "get-ok" />
  <field name = "feed" type = "shortstr" >
    Feed name
    <doc>
    Specifies the name of the feed, which must exist on the server and be
    already owned by the current connection.
    </doc>
    <assert check = "notnull" />
  </field>
</method>

<method name = "get-ok" synchronous = "1" index = "21">
  grant access to the feed
  <doc>
  </doc>
  <chassis name = "client" implement = "MUST" />
  <field name = "lease" type = "shortstr">
    Lease for connection
    <doc>
    Holds a lease which the client must use when opening the direct 
    protocol connection.
    </doc>
  </field>
</method>

</class>

Comments

Add a New Comment

Edit | Files | Tags | Source | Print | Talk

Use one of these tags to define the specification's state:

  • raw - new specification
  • draft - has at least one implementation.
  • stable - has been deployed to real users.
  • legacy - is being replaced by newer specifications.
  • retired - has been replaced and is no longer used.
  • deleted - abandoned before becoming stable.