Skip to content

Wire Protocol

Jump to bottom Edit New page
josephg edited this page Jan 26, 2012 · 16 revisions

Wire protocol

Protocol version 2 - This protocol is used by the not-yet-released ShareJS 0.5. Some of this stuff (the metadata operations) are not implemented yet.

ShareJS has three wire protocols you can use to create, view and edit documents. These are:

  • REST frontend
  • Socket.IO frontend. The socket.io frontend currently only supports v1 of this protocol.
  • BrowserChannel frontend.

The RESTful protocol is simpler to use, but does not support live streaming operations.

Think of ShareJS as a key-value store which maps string names to document data. Documents are versioned. To edit a document, you must apply an operation to the document.

The wire protocols let clients:

  • Fetch a document's current version and contents
  • Submit operations
  • Receive operations as they are submitted (socket.io only)
  • Perminantly delete documents (This is experimental, only available in the REST protocol and disabled by default).

Documents

ShareJS is a key-value store mapping a document's name to a versioned document. Documents must be created before they can be used. Each document has a type, a version, a snapshot and metadata.

The document's type specifies a set of functions for interpreting and manipulating operations. You must set a document's type when it is created. After a document's type is set, it cannot be changed. Each type has a unique string name -- for example, the plain text type is called 'text' and the JSON type is called 'json'. More types will be added in time - in particular rich text. For details of how types work, see Types.

Document versions start at 0. The version is incremented each time an operation is applied.

The document's snapshot is the contents of the document at some particular version. For text documents, the snapshot is a string containing the document's contents. For JSON documents, the snapshot is a JSON object.

Document metadata is a JSON object containing some extra information about the document, including a list of active sessions and a list of contributors. See Document Metadata for details.

For example, a text document might have the snapshot 'abc' at version 100. An operation [{i:'d', p:3}] is applied at version 100 which inserts a 'd' at the end of the document. After this operation is applied, the document has a snapshot at version 101 of 'abcd'.

When a document is first created, it has:

  • Version is set to 0
  • Type set to the type specified in the create request
  • Snapshot set to the result of a call to type.initialVersion(). For text, this is an empty string ''. For JSON, this is null.

RESTful web protocol

Documents can be accessed through a RESTful interface at /doc/DOCNAME. The RESTful interface doesn't support live streaming of operations, but you can do everything else with it.

The RESTful frontend can be disabled by setting rest:null in the options. The sharejs client javascript does not use the REST protocol.

The server uses standard HTTP error codes to respond:

  • 200 if the operation succeeds
  • 403 if the auth function rejects the operation
  • 404 if the requested document does not exist
  • 400 for all other errors

Create a document

PUT /doc/DOCNAME

This creates the named document. The body should be {"type":"TYPE"} (text or json).

Get a document snapshot

GET /doc/DOCNAME

Gets the named document. The document snapshot is returned in the body of the HTTP response.

  • Text documents are returned in the body of the HTTP response and have Content-Type: text/plain
  • JSON documents have Content-Type: application/JSON

The document also has the following headers set:

  • X-OT-Type: The type of the document (Usually 'text' or 'json')
  • X-OT-Version: The version of the document. (Counted from 0 when the document is created)

Submit an operation

POST /doc/DOCNAME?v=VERSION

Submit an operation to the document. The body contains the operation encoded as JSON (eg [{"p":2,"i":"hi there"}] to insert text at position 2.

The version is specified either as a GET parameter (?v=VERSION) or using a X-OT-Version: VERSION header.

Streaming Protocol (Socket.io and BrowserChannel)

The streaming protocol uses JSON messages sent over a standard Socket.io or BrowserChannel connection. A single connection can be used to edit many documents at the same time.

Here is an example of a normal client and server interaction:

(1) S: {auth:'90b657dc1498061fb7b974740c21395d'}
(2) C: {doc:'holiday', open:true, create:true, type:'text', snapshot:null}
(3) S: {doc:'holiday', open:true, create:true, v:0, meta:{creator:'Sam', creationTime:1327379131999}}
(4) C: {v:0, op:[{i:'Hi!', p:0}]}
(5) S: {v:0}
(6) C: {v:1, op:[{i:' there', p:2}]}
(7) S: {v:1, op:[{i:'Oh, ', p:0}], meta:{...}}
(8) S: {v:2}
  1. The server sends the client its authenticated session ID once the client connects and is accepted by user access control. This ID is sent regardless of whether or not you use an auth function.
  2. Open the 'holiday' document. (Documents must be open in order to receive ops sent by other clients.) Create the document if it doesn't exist with type:'text'. If it already exists, send a snapshot.
  3. The 'holiday' document has been opened at version 0. It was created by (1). The type and snapshot are not included because they can be inferred from the context.
  4. Apply an op to the holiday document. (doc:'holiday' is inferred because that was the last doc the client named). The op is [{i:'Hi!', p:0}] which inserts 'Hi!' at position 0 (the start of the document).
  5. The op was applied at version 0. The document is now at version 1.
  6. The client sends another op on the document.
  7. The server tells the client that somebody else has sent an op at version 1 which inserted 'Oh, ' at the start of the document.
  8. The server confirms that it received the client's op. The op was applied at version 2. In order to apply at version 2, the op must have been transformed by the v:1 op. In this case, the transformed operation would have been [{i:' there', p:5}]. The document is now at version 3 and has contents 'Oh, Hi there!'

After this has taken place, another client connects and requests a document snapshot:

(1) C: {doc:'holiday', snapshot:null}
(2) S: {doc:'holiday', v:3, type:'text', snapshot:'Oh, Hi there!', meta:{...}}
  1. Request a snapshot for the 'holiday' document.
  2. The 'holiday' document is a text document at version 3. Its contents are 'Oh, Hi there!'.

Spec

Each message is a JSON object. Messages are interpreted based on which fields they contain.

Each message refers to a particular ShareJS document. Regardless of message type, the relevant document is specified using a doc:DOCNAME field. Both the server and client can leave this field out of a message if the message refers to the same document as the previous message.

In the example above, once the client sent doc:'holiday', the client no longer needed to specify the document name as all subsequent messages the client sends refer to doc:'holiday'. The same is true for the server (though the server still needs to send doc:'holiday' in its first message).

These are all the different messages the client & server can send to each other:

Authenticated

New in 0.5!

Server:

{auth:"<random ID>"}

or

{auth:null, error:'forbidden'}

This message is sent by the server when a client connects. It contains a single field containing the client's session ID.

If a custom auth function is used, the connection attempt can be rejected. In this case, the server responds with auth:null and disconnects the client's session. The client should not reconnect.

The client should store its session ID.

Create a document

This operation can be combined with open and get snapshot, below.

Client:

{doc:DOCNAME, create:true, type:TYPENAME, <meta:{...}>}

Server response:

{doc:DOCNAME, create:true, meta:{creator:'Sam', ctime:<current unix time, eg: 1327379132000>}}

or

{doc:DOCNAME, create:false}

Create a document with the specified type. The document will only be created if it does not already exist. This operation will never destroy data.

Client supplied metadata is currently ignored. The server will set the initial document metadata using the agent's name field and unix time. The creator is set using agent.name, which can be set by setting the server's auth function.

Request a document snapshot

This operation can be combined with create and open.

Client:

{doc:DOCNAME, <type:TYPE>, snapshot:null}

Server response:

{doc:DOCNAME, snapshot:SNAPSHOT, meta:{creator:'sam', ctime:..., mtime:...}, v:VERSION, <type:TYPE>}

or

{doc:DOCNAME, snapshot:null, error:ERRORMESSAGE}

Request a document snapshot. If a type is specified, a document snapshot will only be returned if the real document's type matches.

The format of the snapshot object is type dependant. For text, the snapshot is a string containing the contents of the document.

If the document does not exist or any other error occurred, snapshot:null will be set in the response. However, a null snapshot does not mean an error occurred. Check the error:ERRORMESSAGE field to check for errors.

Valid error messages:

  • 'Type mismatch': The requested document does not have the specified type
  • 'Document does not exist': The requested document does not exist

Snapshot requests can be combined with open and create requests. If a document does not exist, and a client request contains snapshot:null, type:TYPENAME, create:true, the document will be created by the request. In this case, the server simply responds with create:true, meta:{...}. The document snapshot is inferred by calling <TYPE>.create().

NOTE: Requesting a snapshot at a specified version may be added in a future version. Add an issue in the issue tracker if this is important to you.

Open a document (Start streaming operations)

This operation can be combined with create and get snapshot.

Client:

{doc:DOCNAME, open:true, <v:VERSION>, <type:TYPE>}

Server response:

{doc:DOCNAME, open:true, v:VERSION}

Then, repeated:

{doc:DOCNAME, v:APPLIEDVERSION, op:OPERATION, <meta:{...}>}
{doc:DOCNAME, v:APPLIEDVERSION, mop:META OPERATION}

or

{doc:DOCNAME, open:false, error:ERRORMESSAGE}

Open the specified document. Opening a document makes the server send the client all operations applied to the document. Operations sent by the client itself are excluded from this stream.

If a type is specified, the document will only be opened if the document's type matches.

If the server already has operations since VERSION, they are sent immediately.

Version is optional. If not specified, the most recent version is opened.

The version specified in the server response will be the same as the version specified in the client's request, or the most recent version if the client's request did not include a version.

Valid error messages:

  • 'Type mismatch': The requested document does not have the specified type
  • 'Document does not exist': The requested document does not exist

NOTE: You can use this as a ghetto way to get the history of a document. Its kind of awful - I'll add a special API for getting historical operations later. You can make this happen sooner by filing a ticket.

Close a document

Client:

{doc:DOCNAME, open:false}

Server response:

{doc:DOCNAME, open:false}

Close a document. No more operations will be sent to the client.

NOTE: For a short window, the client may receive a few more operations from the server.

Submit an op

Client:

{doc:DOCNAME, v:VERSION, op:OP, <meta:META>, <dupIfSource:[ids]>}

Server response:

{doc:DOCNAME, v:APPLIEDVERSION}

or

{doc:DOCNAME, v:null, error:ERRORMESSAGE}

Submit the operation OP to the server. The op must be valid given the type of the document.

The op must be 'reasonably' recent. (To prevent denial-of-service attacks, The server can reject ops which are too old).

The version specified by the client is the version at which the operation is applied. This is the version the document has before it is applied, not after it has been applied. Generally, this should be the most recent version the client knows about. For example, if you are sending an operation to a new document (version 0), the version specified in your request is 0.

The server responds with the version at which the operation was actually applied. This is usually the same as the version specified in the operation.

If multiple clients send operations at the same time, they are applied in the order they are received by the server. Your operation may be transformed by other operations before it is applied. If your client has the the document open, you will be sent the other operation before being sent confirmation that your operation was applied. The example near the start of this document shows this happening.

For text documents, operations are a list of operation components. Each component either inserts or deletes text at a particular location in the document. For example, this inserts 'hi' at position 50 in the document: [{i:'hi', p:50}] and this deletes it again: [{d:'hi', p:50}].

Refer to (not written yet) documentation on the op type for specifics on what valid operations look like.

The dupIfSource field is used to prevent an op from being submitted multiple times. If a client is disconnected while it has an operation in-flight, it doesn't know whether or not the server received the operation. When it reconnects and resends the op, its previous session ID is added to the op. The server will detect the duplicate operation and reject it if necessary. (error: 'Op already submitted')

Add a custom footer
Add a custom sidebar
Clone this wiki locally