7 KiB
Synapse Server-to-Server API
A description of the protocol used to communicate between Synapse home servers; also known as Federation.
Overview
The server-server API is a mechanism by which two home servers can exchange Synapse event messages, both as a real-time push of current events, and as a historic fetching mechanism to synchronise past history for clients to view. It uses HTTP connections between each pair of servers involved as the underlying transport. Messages are exchanged between servers in real-time by active pushing from each server's HTTP client into the server of the other. Queries to fetch historic data for the purpose of back-filling scrollback buffers and the like can also be performed.
- { Synapse entities } { Synapse entities }
^ | ^ | | events | | events | | V | V
+------------------+ +------------------+ | | | Home Server | | Home Server | | | +------------------+ +------------------+
Transactions and PDUs
The communication between home servers is performed by a bidirectional exchange of messages. These messages are called Transactions, and are encoded as JSON objects with a dict as the top-level element, passed over HTTP. A Transaction is meaningful only to the pair of home servers that exchanged it; they are not globally-meaningful.
Each transaction has an opaque ID and timestamp (UNIX epoch time in miliseconds) generated by its origin server, an origin and destination server name, a list of "previous IDs", and a list of PDUs - the actual message payload that the Transaction carries.
- {"transaction_id":"916d630ea616342b42e98a3be0b74113",
"ts":1404835423000, "origin":"red", "destination":"blue", "prev_ids":["e1da392e61898be4d2009b9fecce5325"], "pdus":[...]}
The "previous IDs" field will contain a list of previous transaction IDs that the origin server has sent to this destination. Its purpose is to act as a sequence checking mechanism - the destination server can check whether it has successfully received that Transaction, or ask for a retransmission if not.
The "pdus" field of a transaction is a list, containing zero or more PDUs.[*] Each PDU is itself a dict containing a number of keys, the exact details of which will vary depending on the type of PDU.
(* Normally the PDU list will be non-empty, but the server should cope with receiving an "empty" transaction, as this is useful for informing peers of other transaction IDs they should be aware of. This effectively acts as a push mechanism to encourage peers to continue to replicate content.)
All PDUs have an ID, a context, a declaration of their type, a list of other PDU IDs that have been seen recently on that context (regardless of which origin sent them), and a nested content field containing the actual event content.
[[TODO(paul): Update this structure so that 'pdu_id' is a two-element [origin,ref] pair like the prev_pdus are]]
- {"pdu_id":"a4ecee13e2accdadf56c1025af232176",
"context":"#example.green", "origin":"green", "ts":1404838188000, "pdu_type":"m.text", "prev_pdus":"blue","99d16afbc857975916f1d73e49e52b65", "content":... "is_state":false}
In contrast to the transaction layer, it is important to note that the prev_pdus field of a PDU refers to PDUs that any origin server has sent, rather than previous IDs that this origin has sent. This list may refer to other PDUs sent by the same origin as the current one, or other origins.
Because of the distributed nature of participants in a Synapse conversation, it is impossible to establish a globally-consistent total ordering on the events. However, by annotating each outbound PDU at its origin with IDs of other PDUs it has received, a partial ordering can be constructed allowing causallity relationships to be preserved. A client can then display these messages to the end-user in some order consistent with their content and ensure that no message that is semantically in reply of an earlier one is ever displayed before it.
PDUs fall into two main categories: those that deliver Events, and those that synchronise State. For PDUs that relate to State synchronisation, additional keys exist to support this:
- {...,
"is_state":true, "state_key":TODO "power_level":TODO "prev_state_id":TODO "prev_state_origin":TODO}
[[TODO(paul): At this point we should probably have a long description of how State management works, with descriptions of clobbering rules, power levels, etc etc... But some of that detail is rather up-in-the-air, on the whiteboard, and so on. This part needs refining. And writing in its own document as the details relate to the server/system as a whole, not specifically to server-server federation.]]
Protocol URLs
For active pushing of messages representing live activity "as it happens":
- PUT /send/:transaction_id/
Body: JSON encoding of a single Transaction
Response: [[TODO(paul): I don't actually understand what ReplicationLayer.on_transaction() is doing here, so I'm not sure what the response ought to be]]
The transaction_id path argument will override any ID given in the JSON body. The destination name will be set to that of the receiving server itself. Each embedded PDU in the transaction body will be processed.
To fetch a particular PDU:
GET /pdu/:origin/:pdu_id/
Response: JSON encoding of a single Transaction containing one PDU
Retrieves a given PDU from the server. The response will contain a single new Transaction, inside which will be the requested PDU.
To fetch all the state of a given context:
GET /state/:context/
Response: JSON encoding of a single Transaction containing multiple PDUs
Retrieves a snapshot of the entire current state of the given context. The response will contain a single Transaction, inside which will be a list of PDUs that encode the state.
To paginate events on a given context:
- GET /paginate/:context/
Query args: v, limit
Response: JSON encoding of a single Transaction containing multiple PDUs
Retrieves a sliding-window history of previous PDUs that occurred on the given context. Starting from the PDU ID(s) given in the "v" argument, the PDUs that preceeded it are retrieved, up to a total number given by the "limit" argument. These are then returned in a new Transaction containing all off the PDUs.
To stream events all the events:
- GET /pull/
Query args: origin, v
Response: JSON encoding of a single Transaction consisting of multiple PDUs
Retrieves all of the transactions later than any version given by the "v" arguments. [[TODO(paul): I'm not sure what the "origin" argument does because I think at some point in the code it's got swapped around.]]