Chat System

A messaging backend that lives in your namespace. Rooms hold peers, peers append messages, messages carry text and files. It is the layer underneath a messenger — the part everyone rebuilds and nobody enjoys rebuilding.

Implementation in progress. The design below is settled. The runtime is being built against it.

A peer is whoever you say it is

When a room is created it mints its own peer ids — one per seat. A peer id is not an email, not a username, and not an account uid. It is an opaque handle for a seat in a conversation, and your application decides what sits in it.

That is deliberate, and it is what makes this usable as the messenger underneath something else. If you are building a social network, peer 0 is one of your account holders and you store the mapping. If you are building a support desk, peer 0 is a customer who never signed up for anything. If you are building a game, a peer is a character. The chat system does not need to know, so it does not ask.

Nothing here depends on the accounts system, and nothing stops you using both together. Keep a column on your own users table holding their peer id for a room, and the two compose without either knowing about the other.

Creating a room

NEW CHAT WITH n PEERS creates a room with n seats and hands back the room and its peer ids in one value.

A Two-Person Conversation

NEW CHAT WITH 2 PEERS SET ?chatroom

AFTER EMIT ?chatroom("id")
AFTER EMIT ?chatroom("peers")(0)
AFTER EMIT ?chatroom("peers")(1)

Keep those three values. The room id addresses the conversation; each peer id addresses one seat in it. A room with ten seats is written the same way — NEW CHAT WITH 10 PEERS — and the peers array simply has ten entries.

Storing the Mapping Alongside Your Own Users

NEW CHAT WITH 2 PEERS SET ?chatroom
AFTER INSERT INTO DB "app" TABLE "conversations"
ROW "room"   AS ?chatroom("id")
AND "user_a" AS ?alice_uid
AND "peer_a" AS ?chatroom("peers")(0)
AND "user_b" AS ?bob_uid
AND "peer_b" AS ?chatroom("peers")(1)
SET ?row

Appending a message

Messages are appended to a room, always from a peer. The timestamp, the message id, and everything else on the record are set for you.

Text

CHAT ?chatroom("id") APPEND MESSAGE ?msgstring FROM ?chatroom("peers")(0)

Text With a File

CHAT ?chatroom("id") APPEND MESSAGE ?msgstring WITH FILE ?fileurl FROM ?chatroom("peers")(1)

A file is carried by reference, not copied into the conversation. ?fileurl is whatever address your application will serve it from — a shared link, a path on a subdomain, an external URL. The file stays wherever you put it and the message points at it.

Sending an Uploaded Image

FILE WRITE !FILES("photo")("data") TO "/root/uploads/" & !FILES("photo")("name")
AFTER CHAT ?chatroom("id") APPEND MESSAGE `sent you a photo`
WITH FILE "/root/uploads/" & !FILES("photo")("name")
FROM ?chatroom("peers")(0)

Several Files at Once

NEW ARRAY SET ?attachments
AFTER APPEND "/root/uploads/floorplan.pdf" TO ?attachments
AFTER APPEND "/root/uploads/site-photo.jpg" TO ?attachments
AFTER APPEND "/root/uploads/quote.xlsx" TO ?attachments

AFTER CHAT ?chatroom("id") APPEND MESSAGE `everything for the site visit`
WITH FILES ?attachments
FROM ?chatroom("peers")(0)

WITH FILE takes one path, WITH FILES takes an array of them. Both store the same thing — a message carries a list of files, and one file is a list of one. Reading a message back always gives you files as an array, so a client never has to branch on how many were sent.

A peer can only append to the room it belongs to. Passing a peer id from one room to another is refused — a seat is a seat in one conversation, not a global identity.

Adding a peer

A room grows by appending a seat. The new peer id comes back and the room now has one more entry in its peers list.

A Third Person Joins

CHAT ?chatroom("id") APPEND PEER SET ?newpeerid
AFTER CHAT ?chatroom("id") APPEND MESSAGE `hello both` FROM ?newpeerid

Every Seat in a Room

CHAT PEER LIST ?chatroom("id") SET ?peers
AFTER COUNT ?peers SET ?n
AFTER EMIT ?n & " people in this conversation"

Reading messages

Every Room, Then Its Messages

CHAT LIST SET ?chatrooms
AFTER CHAT ?chatrooms(0)("id") LIST MESSAGES SET ?msgs

AFTER EMIT ?msgs(0)("text")
AFTER EMIT ?msgs(0)("peer")
AFTER EMIT ?msgs(0)("date")
AFTER EMIT ?msgs(0)("files")

(* files is an array — ?msgs(0)("files")(0), ?msgs(0)("files")(1), and so on.
   A message sent with no attachments has an empty one. *)

Rendering a Message and Its Attachments

CHAT ?room LIST MESSAGES SET ?msgs
AFTER FOREACH ?msgs SET ?m
OPEN
  EMIT `<p>` & ?m("text") & `</p>`
  AFTER COUNT ?m("files") SET ?n
  AFTER IF ?n IS GREATER THAN 0
  OPEN
    FOREACH ?m("files") SET ?f
    OPEN
      FILE SHARE ?f SET ?link
      AFTER EMIT `<a href="` & ?link("url") & `">attachment</a>`
    CLOSE
  CLOSE
CLOSE

A message stores the file's namespace path, not a link. The file stays where FILE WRITE put it and nothing is copied into the conversation. FILE SHARE is how you hand one to a browser when it is time to display it — that step is deliberately yours, because who may see an attachment is your application's decision, not the chat system's.

The message object

Field	Holds
`id`	The message id, ascending in the order messages were appended
`text`	What was said
`peer`	The peer id that appended it — the message's owner
`date`	When it was appended, set by the runtime
`files`	An array of namespace paths — empty when nothing was attached

Only what is new

Reading a whole conversation to find the last three messages is wasteful once a room has history. SINCE takes the last id you saw and returns what has arrived after it, which is all a polling client needs.

Polling for New Messages

CHAT ?room LIST MESSAGES SINCE ?lastseen SET ?new
AFTER COUNT ?new SET ?n
AFTER IF ?n IS GREATER THAN 0
OPEN
  HEADER "Content-Type" AS "application/json"
  AFTER CAST ?new AS JSON SET ?json
  AFTER EMIT ?json
CLOSE

The Most Recent Page

CHAT ?room LIST MESSAGES LIMIT 50 SET ?recent

A worked example — a support inbox

One room per enquiry. The customer never signs in; the agent does. Both are peers.

Opening the Enquiry

NEW CHAT WITH 2 PEERS SET ?room
AFTER INSERT INTO DB "support" TABLE "tickets"
ROW "room"     AS ?room("id")
AND "customer" AS ?room("peers")(0)
AND "agent"    AS ?room("peers")(1)
AND "email"    AS !POST("email")
AND "status"   AS "open"
SET ?ticket

AFTER CHAT ?room("id") APPEND MESSAGE !POST("message") FROM ?room("peers")(0)
AFTER EMAIL SEND TO "support@example.com" SUBJECT "New enquiry"
  BODY `A new ticket was opened.`

The Agent Replies

SELECT ROWS FROM DB "support" TABLE "tickets"
WHERE "room" IS IDENTICAL TO !POST("room")
SET ?t
AFTER CHAT ?t(0)("room") APPEND MESSAGE !POST("reply") FROM ?t(0)("agent")
AFTER UPDATE ROWS FROM DB "support" TABLE "tickets"
WHERE "room" IS IDENTICAL TO ?t(0)("room")
SET "status" AS "answered"

The Customer's Thread, From a Link in Their Email

SELECT ROWS FROM DB "support" TABLE "tickets"
WHERE "room" IS IDENTICAL TO !GET("room")
SET ?t
AFTER CHAT ?t(0)("room") LIST MESSAGES SET ?msgs
AFTER FOREACH ?msgs SET ?m
OPEN
  IF ?m("peer") IS EQUAL TO ?t(0)("agent")
  OPEN
    EMIT `<p class="agent">` & ?m("text") & `</p>`
  CLOSE
  OR
  OPEN
    EMIT `<p class="you">` & ?m("text") & `</p>`
  CLOSE
CLOSE

Full Verb Reference

Verb	Description
`NEW CHAT WITH n PEERS SET ?room`	Create a room with n seats. Returns `id` and `peers`
`CHAT ?id APPEND MESSAGE ?text FROM ?peer`	Append a message. Timestamp and id are set for you
`CHAT ?id APPEND MESSAGE ?text WITH FILE ?path FROM ?peer`	Append a message carrying one file
`CHAT ?id APPEND MESSAGE ?text WITH FILES ?array FROM ?peer`	Append a message carrying several
`CHAT ?id APPEND PEER SET ?peer`	Add a seat, returning its new peer id
`CHAT PEER LIST ?id SET ?peers`	Every peer id in a room, in seat order
`CHAT LIST SET ?rooms`	Every room in the namespace
`CHAT ?id LIST MESSAGES SET ?msgs`	Messages in order — `id`, `text`, `peer`, `date`, `files`
`CHAT ?id LIST MESSAGES SINCE ?id SET ?msgs`	Only what arrived after that message id
`CHAT ?id LIST MESSAGES LIMIT n SET ?msgs`	At most n messages