From 641feef0fd02839c1855e84862630a2a344502b1 Mon Sep 17 00:00:00 2001 From: Boutade Date: Sun, 11 Aug 2019 12:58:06 -0500 Subject: [PATCH 01/10] WebSockets tutorial --- websockets.md | 222 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 222 insertions(+) create mode 100644 websockets.md diff --git a/websockets.md b/websockets.md new file mode 100644 index 00000000..166619e6 --- /dev/null +++ b/websockets.md @@ -0,0 +1,222 @@ +--- +title: WebSockets +--- + +The Common Lisp ecosystem boasts a few approaches to building WebSocket servers. +First, there is the excellent +[Hunchensocket](https://github.com/joaotavora/hunchensocket) that is written as +an extension to [Hunchentoot](https://edicl.github.io/hunchentoot/), the classic +web server for Common Lisp. I have used both and I find them to be wonderful. + +Today, however, you will be using the equally excellent +[websocket-driver](https://github.com/fukamachi/websocket-driver) for use with +[Clack](https://github.com/fukamachi/clack). The community has expressed a +slight prefernce for the Clack eco system, as it provides a uniform interface to +a variety of backends, including Hunchentoot. You can pick and choose the +backend. + +In what follows, you will build a one-room chat server and connect to it from a +web browser. The tutorial is written so that you can enter the code into your +REPL as you go. The full code listing is repeated on the end. + +As a first step, you should load the needed libraries via quicklisp: + +~~~lisp + +(ql:quickload '(clack websocket-driver alexandria)) + +~~~ + + +## The websocket-driver Concept + +In websocket-driver, a WebSocket connection is an instance of the `ws` class, +which exposes an event-driven API. You register event handlers by passing your +WebSocket instance the method `on`, e.g. `(on :message my-websocket #'some-message-handler)`, +where `some-message-handler` would be evoked whenever a new message arrives. + +The `websocket-driver` API provides for handlers on the following events: + +- `:open`: When a connection is opened. Expects a thunk as its handler. +- `:message` When a message arrives. Expects a handler of one argument, the message received. +- `:close` When a connection closes. Expects a handler with two keyword args, a + "code" and a "reason" for the dropped connection. +- `:error` When some kind of protocol level error occurs. Expects a handler of + one argument, the error message. + +For the purposes of your chat server, you will want to handle the case when a +new user arrives to the channel, when the user sends a message to the channel, +and when a user leaves. + +## Defining Handlers for Chat Server Logic + +In this section you will define the functions that your event handlers will +eventually call. These are helper functions that manage the chat server logic. +You will actually define the server in the next section. + +First, when a user connects to the server, you need to give that user a nickname +so that other users know whose chats belong to whom. You will also need a data +structure to map individual WebSocket connections to nicknames: + +~~~lisp + +;; make a hash table to map connections to nicknames +(defvar *connections* (make-hash-table)) + +;; and assign a random nickname to a user upon connection +(defun handle-new-connection (con) + (setf (gethash con *connections*) + (format nil "user-~a" (random 100000)))) + +~~~ + +Next, when a user sends a chat to the room, the rest of the room should be +notified. The message that the server receives is prepended with the nickname of +the user who sent it. + +~~~lisp + +(defun broadcast-to-room (connection message) + (let ((message (format nil "~a: ~a" + (gethash connection *connections*) + message))) + (loop :for con :being :the :hash-key :of *connections* :do + (websocket-driver:send con message)))) +~~~ + +Finally, when a user leaves the channel, by closing the browser tab or +navigating away, the room should be notified of that change, and the user's +connection should be dropped from the `*connections*` table. + +~~~lisp +(defun handle-close-connection (connection) + (let ((message (format nil " .... ~a has left." + (gethash connection *connections*)))) + (remhash connection *connections*) + (loop :for con :being :the :hash-key :of *connections* :do + (websocket-driver:send con message)))) +~~~ + +## Defining A Server + +Using Clack, a server is started by passing a function to `clack:clackup`. You +will define a function called `chat-server` that you will start by +calling `(clack:clackup #'chat-server :port 12345)`. + +A Clack server function accepts a single plist as its argument. That plist +contains environment information about a request and is provided by the system. +Your chat server will not make use of that environment, but if you want to learn +more you can check out Clack's documentation. + +When a browser connects to your server, a websocket will be instantiated, +handlers defined on it for the events you want to support, and then a websocket +"handshake" will be sent back to notify the browser that the connection has been +made. Here's how it works: + +~~~lisp +(defun chat-server (env) + (let ((ws (websocket-driver:make-server env))) + + (websocket-driver:on :open ws + (lambda () (handle-new-connection ws))) + + (websocket-driver:on :message ws + (lambda (msg) (broadcast-to-room ws msg))) + + (websocket-driver:on :close ws + (lambda (&key code reason) + (declare (ignore code reason)) + (handle-close-connection ws))) + + (lambda (responder) + (declare (ignore responder)) + (websocket-driver:start-connection ws)))) + +~~~ + +You may now start your server, running on port `12345`: + +~~~lisp +;; keep the handler around so that you can stop your server later on + +(defvar *chat-handler* (clack:clackup #'chat-server :port 12345)) +~~~ + + +## A Quick HTML Chat Client + +So now you need a way to talk to your server. Using Clack, define a simple +application that serves a webpage to display and send chats. First the web page: + +~~~lisp + +(defvar *html* + " + + + + + LISP-CHAT + + + + +
+ +
+ + + +") + + +(defun client-server (env) + (declare (ignore env)) + `(200 (:content-type "text/html") + (,*html*))) + +~~~ + +You might prefer to put this in a file, as escaping quotes is kind of annoying. +Keeping the page data in a `defvar` was simpler for the purposes of this +tutorial. + +You can see that the `client-server` function just serves the HTML content. Go +ahead and start it, this time on port `8080`: + +~~~lisp +(defvar *client-handler* (clack:clackup #'client-server :port 8080)) +~~~ + +## Check it out! + +Now open up two browser tabs and point them to `http://localhost:8080` and you +should see your chat app! + + + + + From a8fcf6626ea8e39ac32df5b5b440b16afb2bef0a Mon Sep 17 00:00:00 2001 From: thegoofist <49315797+thegoofist@users.noreply.github.com> Date: Sun, 11 Aug 2019 15:49:45 -0500 Subject: [PATCH 02/10] Update websockets.md Co-Authored-By: Moritz <13287984+mohe2015@users.noreply.github.com> --- websockets.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/websockets.md b/websockets.md index 166619e6..4f2da7db 100644 --- a/websockets.md +++ b/websockets.md @@ -33,7 +33,7 @@ As a first step, you should load the needed libraries via quicklisp: In websocket-driver, a WebSocket connection is an instance of the `ws` class, which exposes an event-driven API. You register event handlers by passing your WebSocket instance the method `on`, e.g. `(on :message my-websocket #'some-message-handler)`, -where `some-message-handler` would be evoked whenever a new message arrives. +where `some-message-handler` would be invoked whenever a new message arrives. The `websocket-driver` API provides for handlers on the following events: From a9fcbce57e5c9304ea227d8916f4d3a5f93b2925 Mon Sep 17 00:00:00 2001 From: thegoofist <49315797+thegoofist@users.noreply.github.com> Date: Sun, 11 Aug 2019 15:50:02 -0500 Subject: [PATCH 03/10] Update websockets.md Co-Authored-By: Moritz <13287984+mohe2015@users.noreply.github.com> --- websockets.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/websockets.md b/websockets.md index 4f2da7db..50d4f4b5 100644 --- a/websockets.md +++ b/websockets.md @@ -35,7 +35,7 @@ which exposes an event-driven API. You register event handlers by passing your WebSocket instance the method `on`, e.g. `(on :message my-websocket #'some-message-handler)`, where `some-message-handler` would be invoked whenever a new message arrives. -The `websocket-driver` API provides for handlers on the following events: +The `websocket-driver` API provides handlers for the following events: - `:open`: When a connection is opened. Expects a thunk as its handler. - `:message` When a message arrives. Expects a handler of one argument, the message received. From 987881deb0aed689743f2e1a2cdb90b19ca631fe Mon Sep 17 00:00:00 2001 From: thegoofist <49315797+thegoofist@users.noreply.github.com> Date: Sun, 11 Aug 2019 15:50:18 -0500 Subject: [PATCH 04/10] Update websockets.md Co-Authored-By: Moritz <13287984+mohe2015@users.noreply.github.com> --- websockets.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/websockets.md b/websockets.md index 50d4f4b5..9a1d3d54 100644 --- a/websockets.md +++ b/websockets.md @@ -38,7 +38,7 @@ where `some-message-handler` would be invoked whenever a new message arrives. The `websocket-driver` API provides handlers for the following events: - `:open`: When a connection is opened. Expects a thunk as its handler. -- `:message` When a message arrives. Expects a handler of one argument, the message received. +- `:message` When a message arrives. Expects a handler with one argument, the message received. - `:close` When a connection closes. Expects a handler with two keyword args, a "code" and a "reason" for the dropped connection. - `:error` When some kind of protocol level error occurs. Expects a handler of From f43dcd0255739013d0117a91c89380d33f9dff72 Mon Sep 17 00:00:00 2001 From: thegoofist <49315797+thegoofist@users.noreply.github.com> Date: Sun, 11 Aug 2019 15:50:34 -0500 Subject: [PATCH 05/10] Update websockets.md Co-Authored-By: Moritz <13287984+mohe2015@users.noreply.github.com> --- websockets.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/websockets.md b/websockets.md index 9a1d3d54..e1f7893e 100644 --- a/websockets.md +++ b/websockets.md @@ -41,7 +41,7 @@ The `websocket-driver` API provides handlers for the following events: - `:message` When a message arrives. Expects a handler with one argument, the message received. - `:close` When a connection closes. Expects a handler with two keyword args, a "code" and a "reason" for the dropped connection. -- `:error` When some kind of protocol level error occurs. Expects a handler of +- `:error` When some kind of protocol level error occurs. Expects a handler with one argument, the error message. For the purposes of your chat server, you will want to handle the case when a From 26048cfe4a96603d8a848dad96f296b236604b8b Mon Sep 17 00:00:00 2001 From: thegoofist <49315797+thegoofist@users.noreply.github.com> Date: Sun, 11 Aug 2019 15:50:46 -0500 Subject: [PATCH 06/10] Update websockets.md Co-Authored-By: Moritz <13287984+mohe2015@users.noreply.github.com> --- websockets.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/websockets.md b/websockets.md index e1f7893e..1be46989 100644 --- a/websockets.md +++ b/websockets.md @@ -45,7 +45,7 @@ The `websocket-driver` API provides handlers for the following events: one argument, the error message. For the purposes of your chat server, you will want to handle the case when a -new user arrives to the channel, when the user sends a message to the channel, +new user arrives to the channel, when a user sends a message to the channel, and when a user leaves. ## Defining Handlers for Chat Server Logic From a0d53543a328c6f3a77a1bf7a9a6a2992b7b04a7 Mon Sep 17 00:00:00 2001 From: thegoofist <49315797+thegoofist@users.noreply.github.com> Date: Sun, 11 Aug 2019 15:53:28 -0500 Subject: [PATCH 07/10] Update websockets.md --- websockets.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/websockets.md b/websockets.md index 1be46989..db9db98d 100644 --- a/websockets.md +++ b/websockets.md @@ -37,7 +37,7 @@ where `some-message-handler` would be invoked whenever a new message arrives. The `websocket-driver` API provides handlers for the following events: -- `:open`: When a connection is opened. Expects a thunk as its handler. +- `:open`: When a connection is opened. Expects a handler with zero arguments. - `:message` When a message arrives. Expects a handler with one argument, the message received. - `:close` When a connection closes. Expects a handler with two keyword args, a "code" and a "reason" for the dropped connection. From c2239dbbfbc8d608db5318169ca89f344088563a Mon Sep 17 00:00:00 2001 From: thegoofist <49315797+thegoofist@users.noreply.github.com> Date: Sun, 11 Aug 2019 15:59:26 -0500 Subject: [PATCH 08/10] Reworded, fixed a typo --- websockets.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/websockets.md b/websockets.md index db9db98d..d77ccb24 100644 --- a/websockets.md +++ b/websockets.md @@ -9,11 +9,11 @@ an extension to [Hunchentoot](https://edicl.github.io/hunchentoot/), the classic web server for Common Lisp. I have used both and I find them to be wonderful. Today, however, you will be using the equally excellent -[websocket-driver](https://github.com/fukamachi/websocket-driver) for use with -[Clack](https://github.com/fukamachi/clack). The community has expressed a -slight prefernce for the Clack eco system, as it provides a uniform interface to -a variety of backends, including Hunchentoot. You can pick and choose the -backend. +[websocket-driver](https://github.com/fukamachi/websocket-driver) to build a Websocket server with +[Clack](https://github.com/fukamachi/clack). The Common Lisp web development community has expressed a +slight prefernce for the Clack ecosystem because it provides a uniform interface to +a variety of backends, including Hunchentoot. That is, with Clack, you can pick and choose the +backend you prefer. In what follows, you will build a one-room chat server and connect to it from a web browser. The tutorial is written so that you can enter the code into your From 7f35c8cdb7e929868af63dbf89a4ec36223489e9 Mon Sep 17 00:00:00 2001 From: thegoofist <49315797+thegoofist@users.noreply.github.com> Date: Sun, 11 Aug 2019 16:06:06 -0500 Subject: [PATCH 09/10] Oops. Forgot to reproduce the code listing! --- websockets.md | 98 +++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 95 insertions(+), 3 deletions(-) diff --git a/websockets.md b/websockets.md index d77ccb24..d5aff72e 100644 --- a/websockets.md +++ b/websockets.md @@ -9,15 +9,15 @@ an extension to [Hunchentoot](https://edicl.github.io/hunchentoot/), the classic web server for Common Lisp. I have used both and I find them to be wonderful. Today, however, you will be using the equally excellent -[websocket-driver](https://github.com/fukamachi/websocket-driver) to build a Websocket server with +[websocket-driver](https://github.com/fukamachi/websocket-driver) to build a WebSocket server with [Clack](https://github.com/fukamachi/clack). The Common Lisp web development community has expressed a slight prefernce for the Clack ecosystem because it provides a uniform interface to a variety of backends, including Hunchentoot. That is, with Clack, you can pick and choose the backend you prefer. -In what follows, you will build a one-room chat server and connect to it from a +In what follows, you will build a simple chat server and connect to it from a web browser. The tutorial is written so that you can enter the code into your -REPL as you go. The full code listing is repeated on the end. +REPL as you go, but in case you miss something, the full code listing can be found at the end. As a first step, you should load the needed libraries via quicklisp: @@ -218,5 +218,97 @@ should see your chat app! +## All The Code + +~~~lisp +(ql:quickload '(clack websocket-driver alexandria)) + +(defvar *connections* (make-hash-table)) + +(defun handle-new-connection (con) + (setf (gethash con *connections*) + (format nil "user-~a" (random 100000)))) + +(defun broadcast-to-room (connection message) + (let ((message (format nil "~a: ~a" + (gethash connection *connections*) + message))) + (loop :for con :being :the :hash-key :of *connections* :do + (websocket-driver:send con message)))) + +(defun handle-close-connection (connection) + (let ((message (format nil " .... ~a has left." + (gethash connection *connections*)))) + (remhash connection *connections*) + (loop :for con :being :the :hash-key :of *connections* :do + (websocket-driver:send con message)))) + +(defun chat-server (env) + (let ((ws (websocket-driver:make-server env))) + (websocket-driver:on :open ws + (lambda () (handle-new-connection ws))) + + (websocket-driver:on :message ws + (lambda (msg) (broadcast-to-room ws msg))) + + (websocket-driver:on :close ws + (lambda (&key code reason) + (declare (ignore code reason)) + (handle-close-connection ws))) + (lambda (responder) + (declare (ignore responder)) + (websocket-driver:start-connection ws)))) + +(defvar *html* + " + + + + + LISP-CHAT + + + + +
+ +
+ + + +") + + + +(defun client-server (env) + (declare (ignore env)) + `(200 (:content-type "text/html") + (,*html*)))) + +(defvar *chat-handler* (clack:clackup #'chat-server :port 12345)) +(defvar *client-handler* (clack:clackup #'client-server :port 8080)) +~~~ From 3ccb147189fbad9c3dffc64f8346872303685f62 Mon Sep 17 00:00:00 2001 From: Boutade Date: Mon, 12 Aug 2019 08:13:41 -0500 Subject: [PATCH 10/10] typos --- websockets.md | 25 +++++++++++++------------ 1 file changed, 13 insertions(+), 12 deletions(-) diff --git a/websockets.md b/websockets.md index d5aff72e..ea85bc1a 100644 --- a/websockets.md +++ b/websockets.md @@ -11,7 +11,7 @@ web server for Common Lisp. I have used both and I find them to be wonderful. Today, however, you will be using the equally excellent [websocket-driver](https://github.com/fukamachi/websocket-driver) to build a WebSocket server with [Clack](https://github.com/fukamachi/clack). The Common Lisp web development community has expressed a -slight prefernce for the Clack ecosystem because it provides a uniform interface to +slight prefernce for the Clack ecosystem because Clack provides a uniform interface to a variety of backends, including Hunchentoot. That is, with Clack, you can pick and choose the backend you prefer. @@ -32,8 +32,9 @@ As a first step, you should load the needed libraries via quicklisp: In websocket-driver, a WebSocket connection is an instance of the `ws` class, which exposes an event-driven API. You register event handlers by passing your -WebSocket instance the method `on`, e.g. `(on :message my-websocket #'some-message-handler)`, -where `some-message-handler` would be invoked whenever a new message arrives. +WebSocket instance as the first argument to a method called `on`. For example, +calling `(on :message my-websocket #'some-message-handler)` would invoke +`some-message-handler` whenever a new message arrives. The `websocket-driver` API provides handlers for the following events: @@ -44,15 +45,15 @@ The `websocket-driver` API provides handlers for the following events: - `:error` When some kind of protocol level error occurs. Expects a handler with one argument, the error message. -For the purposes of your chat server, you will want to handle the case when a -new user arrives to the channel, when a user sends a message to the channel, +For the purposes of your chat server, you will want to handle three cases: when +a new user arrives to the channel, when a user sends a message to the channel, and when a user leaves. ## Defining Handlers for Chat Server Logic In this section you will define the functions that your event handlers will eventually call. These are helper functions that manage the chat server logic. -You will actually define the server in the next section. +You will define the WebSocket server in the next section. First, when a user connects to the server, you need to give that user a nickname so that other users know whose chats belong to whom. You will also need a data @@ -108,10 +109,10 @@ contains environment information about a request and is provided by the system. Your chat server will not make use of that environment, but if you want to learn more you can check out Clack's documentation. -When a browser connects to your server, a websocket will be instantiated, -handlers defined on it for the events you want to support, and then a websocket -"handshake" will be sent back to notify the browser that the connection has been -made. Here's how it works: +When a browser connects to your server, a websocket will be instantiated and +will be handlers defined on it each of the the events you want to support. +Finally a WebSocket "handshake" will be sent back to the browser, indicating +that the connection has been made. Here's how it works: ~~~lisp (defun chat-server (env) @@ -130,7 +131,7 @@ made. Here's how it works: (lambda (responder) (declare (ignore responder)) - (websocket-driver:start-connection ws)))) + (websocket-driver:start-connection ws)))) ; send the handshake ~~~ @@ -199,7 +200,7 @@ application that serves a webpage to display and send chats. First the web page ~~~ -You might prefer to put this in a file, as escaping quotes is kind of annoying. +You might prefer to put the HTML into a file, as escaping quotes is kind of annoying. Keeping the page data in a `defvar` was simpler for the purposes of this tutorial.