protocol

overview

The goval protocol is built around isolated synchronous channels for all communication. The channels act as a namespacing mechanism and have a 1 to 1 mapping to service instances inside pid1. Creating a channel will always construct a new instance of a service. A global channel (channel with id 0) is also present and meant for managing channels and signaling state which is isolated per user. Each service decides the semantics around its channel (e.g. events, request/response, etc.). All messages within a single channel are guaranteed to arrive in order but across channels there is no guarantee.

Channels are assigned int IDs given by pid1 upon opening a channel. A channel can optionally be named which allows other clients to share the channel by way of attaching. An unnamed channel is called an anonymous channel and can never be attached to, it is bound the creating client.

There is also a built in request/response style system to make life easier for clients. A service can decide to "respond" to a incoming message through its passed in response writer instead of the broadcast writer. A response will automatically have the corresponding request message's ref (if it has one) burned into it. This way a client can make a request without needing to keep track of order.

Goval is setup like this:

+------+       +------+       +----+
|client| <---> |conman| <---> |pid1|
+------+       |      |       +----+
+------+       |      |
|client| <---> |      |
+------+       +------+

channel 0, a channel for channels

It all starts with channel 0. Every client automatically "has" channel 0 and can use it to open more channels.

All messages on channel 0 are synchronous and globally locking across all channels. This means time spent opening/closing a channel should be minimal as it blocks messages to all other channels. The advantage of this is guaranteeing all messages after a chan 0 message will be dispatched after that chan 0 message has been processed and global state updated. This makes it possible to open a channel and sent initial messages before receiving a chanOpenRes.

opening channels

To open a channel the client sends one of:

// Create a new named channel of the given service and with the given name. An
// error is returned if a service with that name already exists.
{
  channel: 0,
  openChan: {
    service: "<service name>",
    name: "<channel name>",
    action: CREATE,
  },
  ref: "[ref]",
} |

// Create a new anonymous channel of the given service. This service is bound
// to the creating client and cannot be shared.
{
  channel: 0,
  openChan: {
    service: "<service name>",
    name: "",
    action: CREATE,
  },
  ref: "[ref]",
} |

// Attach to an existing channel with the provided name. If it does not exist
// an error is returned.
//
// TODO: there should be a variant of this in which you can specify the service
//       resulting in an error if the service names mismatch
{
  channel: 0,
  openChan: {
    name: "<channel name>",
    action: ATTACH,
  },
  ref: "[ref]",
} |

// Attach to the privided channel if an existing channel by the same name
// exists. Otherwise create a new named channel with the given service.
//
// Note: this will attach you to an existing channel even if the service names
//       differ
{
  channel: 0,
  openChan: {
    service: "<service name>",
    name: "<channel name>",
    action: ATTACH_OR_CREATE,
  },
  ref: "[ref]",
}

Which can include an optional ref.

And should can expect the next openChanRes to correspond to this open request. It's worth noting that: although openChanRes will correspond to the order in which openChans are sent, other messages (i.e. state notifications) can occur in-between on the global channel. If successful the openChanRes will include the channel's numeric id. All messages destined for this channel/service should have channel set to this id and all messages from this channel will have the same id.

Opening a channel will launch the service. Some services might start sending back responses immediately (i.e. interp will send back the prompt as soon as it starts).

The openChan command has an optional (as in the default value indicates to ignore) id allowing the client to choose the channel's id ahead of time. This makes it possible to request an open channel and send messages before receiving a openChanRes. Channel IDs are namespaced to the client (session) preventing collisions across sessions. A collision within a session (trying to assign a channel an existing id) is an error and will prevent the channel from opening.

There is a weird case where an error opening a channel will cause any inflight messages destined for that channel to become invalid. It's unclear what to do here, so we'll panic :/.

closing channels

Channel closing provides a flexible way for a client to stop talking to a service. This service can either be destroy from the close or kept alive in the background. It's important to note that a channel will never be closed out from under a client who holds a reference to that channel.

A service instance will receive a closeChan message when its channel is closed. Any services which require internal cleanup should perform that here. As with openChan, the passed in writer will be the broadcast writer. The broadcast writer will be held open until the handle function returns, after that it is an error to write to the broadcast writer. For most simple services it is safe to ignore this message as the service instance will dropped and garbage collected.

Anonymous channels will also be automatically closed when the corresponding client is disconnected. This sends a closeChan message to the service the same way a explicitly closing the channel would. In this case the action of the closeChan sent to the service is DISCONNECT.

A closeChan command takes a few different forms depending on the provided action:

A channel close message should look like:

{
  channel: 0,
  closeChan: {
    id: <chan id>,
    action: DISCONNECT | TRY_CLOSE | CLOSE,
  },
  ref: "[ref]"
}

which will send back:

{
  channel: 0,
  closeChanRes: {
    id: <chan id>,
    status: DISCONNECT | CLOSE | NOTHING,
  },
  ref: "[ref]",
}

once closing completes. Closing will never error.

writing a good client

A client must always be ready for any global notification on channel 0, these could be:

Any unexpected messages should be ignored.

writing a service

Create a new package in pkg/pid1 with the name of your service, e.g. exec. You should start by defining a struct to implement the pid1.Service interface, e.g.

type MyService struct {
    api.Writer

    // Any state that your service needs to keep track of, e.g.

    // whether or not a process is currently running
    state api.State

    // language configuration loaded from the JSON file
    config *types.LanguageConfig
}

Note the use of an anonymous struct field for api.Writer. This is a common pattern which allows you to call .Write() directly on your service to send messages to all clients. Then you need to write a New() function, e.g.

func New(s pid1.Self, w api.Writer) (pid1.Service, error) {
    e := &MyService{
        Writer: w,
        state: api.State_Stopped,
        config: s.Config,
    }
    return e, nil
}

Make sure to save w into your service struct. You will need it to broadcast messages to all clients. The Handle method (see below) is also passed an api.Writer, but that writer is used for returning a response to a single client (the one that made the request being handled). If you try to use it to send more than one response, pid1 will panic.

Next, you need to implement the Handle() method for your service. This is the service's only entry point. You will get a pid1.Request object, whose format is defined using Protobuf in api/types.proto.

You can read more about Protobuf with Go in the official docs, but here are the important parts. types.proto defines a message Command, which has a body field that can be any different type of message. For example, if body happens to contain an OpenChannel message (which uses the openChan field in Protobuf), then it will be of type Command_OpenChan, where the type name is auto generated from the struct by Protobuf. Within a Go type switch on the body, you can then access the OpenChannel message by means of the .OpenChan field on the body.

Let's say you want to implement a simple service that implements the following interface:

First you would want to visit types.proto and add new message types:

message ItsPingPongTime {}
message Ping {
  string payload = 1;
}
message Pong {
  string payload = 1;
}

Then you would want to register those message types as allowable choices for body (where the ID numbers are not already taken, and follow the existing conventions):

oneof body {
  ...

  ItsPingPongTime itsPingPongTime = 280;
  Ping ping = 281;
  Pong pong = 282;
}

Then your Handle() implementation might look like this:

func (s *MyService) Handle(r pid1.Request, w api.Writer) {
    switch b := r.Body.(type) {
    case *api.Command_OpenChan:
        if checkNothingCurrentlyOnFire {
            w.Write(&api.Command{
                Session: r.Session, Body: &api.Command_ItsPingPongTime{}
            })
        } else {
            w.Write(&api.Command{
                Session: r.Session, Body: &api.Command_Error{
                    Error: "lp0 on fire",
                },
            })
        }

    case *api.Command_Ping:
        payload := b.Ping.payload
        payload, err := doComplicatedPingPongCalculation(payload)
        if err != nil {
            s.Write(&api.Command{Body: &api.Command_Pong{
                Payload: payload
            }})
        }
        api.Respond(w).Status(err)
    }
}

Note that we ignore all unknown messages. Note also that we write to w when we want to return a response to the client who invoked the currently request, and we write to s (i.e. to ourselves) when we want to broadcast to all clients. The latter works because of the anonymous struct field trick with api.Writer. Finally, note the use of api.Respond from api/main.go. Its .Status() method sends either Command_Ok or Command_Error based on an error value.

After you create your service, please document it for the sake of our sanity!