protocol

overview

The goval protocol is built around isolated synchronous channels for all communication. The channels act as a namespacing mechanism and have a 1 to 1 mapping to instances of a service inside pid1. Creating a channel will construct a new instance of the desired service and attaching to a channel Will share an existing service. A global channel (channel with id 0) is always present and intended for channel management (and a few global state messages). Each service decides the semantics around its channel's usage (e.g. events, request/response, etc.). All messages within a single channel are guaranteed to arrive in order but across channels there is no guarantee.

Channels are assigned int IDs given by pid1 upon opening a channel. A channel can optionally be named allowing other clients to share the channel by way of attaching. An unnamed channel is called an anonymous channel and can never be attached to, it is bound the creating client. All anonymous channels are automatically destroyed when the client leaves taking the associated service with it.

There is also a built in request/response style system to make life easier for clients. A service can decide to "respond" to a incoming message through its passed in response writer instead of the broadcast writer. A response will automatically have the corresponding request message's ref (if it has one) burned into it. This way a client can make a request without needing to keep track of order.

Goval is setup roughly like this:

+------+       +------+       +----+
|client| <---> |conman| <---> |pid1|
+------+       |      |       +----+
+------+       |      |
|client| <---> |      |
+------+       +------+

channel 0, a channel for channels

It all starts with channel 0. Every client automatically "has" channel 0 and can use it to open more channels.

All messages on channel 0 are synchronous and globally locking across all channels. This means time spent opening/closing a channel should be minimal as it blocks messages to all other channels. The advantage of this is guaranteeing all messages after a chan 0 message will be dispatched after that chan 0 message has been processed and global state updated. This makes it possible to open a channel and sent initial messages before receiving a chanOpenRes.

opening channels

To open a channel the client sends one of:

// Create a new named channel of the given service and with the given name. An
// error is returned if a service with that name already exists.
{
  channel: 0,
  openChan: {
    service: "<service name>",
    name: "<channel name>",
    action: CREATE,
  },
  ref: "[ref]",
} |

// Create a new anonymous channel of the given service. This service is bound
// to the creating client and cannot be shared.
{
  channel: 0,
  openChan: {
    service: "<service name>",
    name: "",
    action: CREATE,
  },
  ref: "[ref]",
} |

// Attach to an existing channel with the provided name. If it does not exist
// an error is returned.
//
// TODO: there should be a variant of this in which you can specify the service
//       resulting in an error if the service names mismatch
{
  channel: 0,
  openChan: {
    name: "<channel name>",
    action: ATTACH,
  },
  ref: "[ref]",
} |

// Attach to the privided channel if an existing channel by the same name
// exists. Otherwise create a new named channel with the given service.
//
// Note: this will attach you to an existing channel even if the service names
//       differ
{
  channel: 0,
  openChan: {
    service: "<service name>",
    name: "<channel name>",
    action: ATTACH_OR_CREATE,
  },
  ref: "[ref]",
}

Which can include an optional ref.

And should can expect the next openChanRes to correspond to this open request. It's worth noting that: although openChanRes will correspond to the order in which openChans are sent, other messages (i.e. state notifications) can occur in-between on the global channel. If successful the openChanRes will include the channel's numeric id. All messages destined for this channel/service should have channel set to this id and all messages from this channel will have the same id.

Opening a channel will launch the service. Some services might start sending back responses immediately (i.e. interp will send back the prompt as soon as it starts).

The openChan command has an optional (as in the default value indicates to ignore) id allowing the client to choose the channel's id ahead of time. This makes it possible to request an open channel and send messages before receiving a openChanRes. Channel IDs are namespaced to the client (session) preventing collisions across sessions. A collision within a session (trying to assign a channel an existing id) is an error and will prevent the channel from opening.

There is a weird case where an error opening a channel will cause any inflight messages destined for that channel to become invalid. It's unclear what to do here, so we'll panic :/.

closing channels

Channel closing provides a flexible way for a client to stop talking to a service. This service can either be destroy from the close or kept alive in the background. It's important to note that a channel will never be closed out from under a client who holds a reference to that channel.

A service instance will receive a closeChan message when its channel is closed. Any services which require internal cleanup should perform that here. As with openChan, the passed in writer will be the broadcast writer. The broadcast writer will be held open until the handle function returns, after that it is an error to write to the broadcast writer. For most simple services it is safe to ignore this message as the service instance will dropped and garbage collected.

Anonymous channels will also be automatically closed when the corresponding client is disconnected. This sends a closeChan message to the service the same way a explicitly closing the channel would. In this case the action of the closeChan sent to the service is DISCONNECT.

A closeChan command takes a few different forms depending on the provided action:

A channel close message should look like:

{
  channel: 0,
  closeChan: {
    id: <chan id>,
    action: DISCONNECT | TRY_CLOSE | CLOSE,
  },
  ref: "[ref]"
}

which will send back:

{
  channel: 0,
  closeChanRes: {
    id: <chan id>,
    status: DISCONNECT | CLOSE | NOTHING,
  },
  ref: "[ref]",
}

once closing completes. Closing will never error.

writing a good client

A client must always be ready for any global notification on channel 0, these could be:

Any unexpected messages should be ignored.

writing a service

Create a new package in pkg/pid1 with the name of your service, e.g. exec. You should start by defining a struct to implement the pid1.Service interface, e.g.

type MyService struct {
    // Any state that your service needs to keep track of, e.g.

    // If you want to store the broadcast writer for use later
    // when sending a message to all connected clients.
    api.Writer

    // whether or not a process is currently running
    state api.State

    // language configuration loaded from the JSON file
    config *types.LanguageConfig
}

Then you need to write a New() function, e.g.

func New(s pid1.Self, w api.Writer) (pid1.Service, error) {
    e := &MyService{
        Writer: w,
        state: api.State_Stopped,
        config: s.Config,
    }
    return e, nil
}

This function is called any time a channel to this service is opened and meant to instantiate a fresh instance of the service. The passed in pid1.Self provides access to the global config and an internal client. The passed in api.Writer is what we call the service's "broadcast" writer.

The broadcast writer allows a service to send messages to all connected users at any point. These messages are not allowed to have the ref set, meaning they should not be responses. The call to New is the only chance to get the broadcast writer so if you want to use it later you should store it in your service's struct. Messages send on the broadcast writer can also be routed be setting the Session field. The values are:

< 0 : all clients *except* the client with abs(id)
  0 : all clients connected to this channel, this is the default
> 0 : only this the client with this id

The Handle method (see below) is also passed an api.Writer, but that writer is used for returning a response to a single client (the one that send the message being handled). We call this writer the response writer. If you try to use it to send more than one response, pid1 will yell at you.

Next, you need to implement the Handle() method for your service. This is the service's only entry point. You will get a pid1.Request object, whose format is defined using Protobuf in api/types.proto.

You can read more about Protobuf with Go in the official docs, but here are the important parts. types.proto defines a message Command, which has a body field that can be any different type of message. For example, if body happens to contain an OpenChannel message (which uses the openChan field in Protobuf), then it will be of type Command_OpenChan, where the type name is auto generated from the struct by Protobuf. Within a Go type switch on the body, you can then access the OpenChannel message by means of the .OpenChan field on the body.

Let's say you want to implement a simple service that implements the following interface:

First you would want to visit types.proto and add new message types:

message ItsPingPongTime {}
message Ping {
  string payload = 1;
}
message Pong {
  string payload = 1;
}

Then you would want to register those message types as allowable choices for body (where the ID numbers are not already taken, and follow the existing conventions):

oneof body {
  ...

  ItsPingPongTime itsPingPongTime = n;
  Ping ping = n;
  Pong pong = n;
}

Then your Handle() implementation might look like this:

func (s *MyService) Handle(r pid1.Request, w api.Writer) {
    switch b := r.Body.(type) {
    case *api.Command_OpenChan:
        if checkNothingCurrentlyOnFire {
            w.Write(&api.Command{
                Session: r.Session, Body: &api.Command_ItsPingPongTime{}
            })
        } else {
            w.Write(&api.Command{
                Session: r.Session, Body: &api.Command_Error{
                    Error: "lp0 on fire",
                },
            })
        }

    case *api.Command_Ping:
        payload := b.Ping.payload
        payload, err := doComplicatedPingPongCalculation(payload)
        if err != nil {
            s.Write(&api.Command{Body: &api.Command_Pong{
                Payload: payload
            }})
        }
        api.Respond(w).Status(err)
    }
}

Note that we ignore all unknown messages. Note also that we write to w when we want to return a response to the client who invoked the currently request, and we write to s (i.e. to ourselves) when we want to broadcast to all clients. The latter works because of the anonymous struct field trick with api.Writer. Finally, note the use of api.Respond from api/main.go. Its .Status() method sends either Command_Ok or Command_Error based on an error value.

After you create your service, please document it for the sake of our sanity!