glyde/testing/adapter

The conformance suite an adapter has to pass before anyone should trust it.

A Scenario scripts what the socket and the clock do, and the expectation is the exact list of acts the adapter should have produced, in order.

import glyde/testing/adapter

let failures =
  adapter.run_all(my_adapter)
  |> list.filter(adapter.failed)

list.each(failures, fn(report) { io.println(adapter.describe(report)) })

What an adapter has to provide

Three functions, in an Adapter: build a bot from a Setup, feed it one Input, and hand back the World its transport wrote into.

The scripted world never opens anything. Setup.transport is already wired to record, and an adapter under test passes it through untouched.

Why the expectations are opcodes and not payloads

IDENTIFY carries the bot token, and a suite whose expected values contain a token is a suite nobody can paste into an issue.

Types

One thing the adapter was asked to do. Coarser than gateway.Output: what has to match between adapters is the sequence and the numbers, not bytes.

pub type Act {
  Dialled(host: String, path: String)
  Wrote(op: Sent)
  Shut(code: Int)
  Dropped
  Armed(timer: gateway.Timer, in_ms: Int)
  Disarmed(timer: gateway.Timer)
  Emitted(event: String)
  Reconnecting(in_ms: Int, resuming: Bool)
  Halted(reason: gateway.Halt)
}

Constructors

  • Dialled(host: String, path: String)

    A socket was opened to wss://<host><path>.

  • Wrote(op: Sent)

    A text frame went out, named by the opcode it was built with. The body is never recorded: only IDENTIFY’s would be worth reading, and it holds the token.

  • Shut(code: Int)

    A close frame with this code, socket still to report back.

  • Dropped

    The socket was abandoned with no close frame.

  • Armed(timer: gateway.Timer, in_ms: Int)
  • Disarmed(timer: gateway.Timer)
  • Emitted(event: String)

    A dispatch, or an event with nothing worth carrying, reached the host.

  • Reconnecting(in_ms: Int, resuming: Bool)

    Its own act because the delay is the interesting part of it.

  • Halted(reason: gateway.Halt)

    Its own act because “the host asked to stop” and “Discord refused with 4004 and no reconnect is coming” are different endings.

The adapter under test, as the three things the harness needs of it. A loop that cannot take a given transport one input at a time cannot be tested.

pub type Adapter(bot) {
  Adapter(
    start: fn(Setup) -> bot,
    feed: fn(bot, gateway.Input) -> bot,
    world: fn(bot) -> World,
  )
}

Constructors

  • Adapter(
      start: fn(Setup) -> bot,
      feed: fn(bot, gateway.Input) -> bot,
      world: fn(bot) -> World,
    )

    Arguments

    start

    Build a bot on this shard, wired to this transport. Do not start it: the harness feeds gateway.Start so its acts are part of the trace.

    feed

    Deliver one input and perform everything it produces.

    world

    The state the transport has been writing into.

One ArmTimer as the scripted world sees it. A delay and not an instant: the world has no clock, and Waits is what turns one into the other.

pub type Arming {
  Arming(timer: gateway.Timer, in_ms: Int, stamp: gateway.Stamp)
}

Constructors

One thing the world does to the adapter. Every scenario begins started, and a beat naming a socket means the one the adapter last opened.

pub type Beat {
  Connects
  Refuses(failure: gateway.DialFailure)
  Delivers(payload: String)
  PeerShuts(code: option.Option(Int))
  ShutConfirmed
  Waits(ms: Int)
  StaleFire(timer: gateway.Timer)
  HostSends(command: command.Command)
  HostStops
}

Constructors

  • Connects

    The socket finished its upgrade.

  • Refuses(failure: gateway.DialFailure)

    The socket never came up. gateway.Refused carries a status the shard reads; gateway.Unreachable is a dial that never got an answer.

  • Delivers(payload: String)

    A text frame arrived. glyde/testing/frames builds them.

  • PeerShuts(code: option.Option(Int))

    The peer closed. None is a transport that died with no close frame.

  • ShutConfirmed

    The close the adapter asked for, reported back on the socket that was closed. Answering with the wrong connection disables the staleness guard.

  • Waits(ms: Int)

    Time passes. Every timer due inside the window fires, soonest first, at the instant it was due.

  • StaleFire(timer: gateway.Timer)

    A firing that lost the race with its own cancellation: the right timer, a stamp from an older arming. It must change nothing.

  • HostSends(command: command.Command)

    The host asks to send a gateway command from outside a handler.

  • HostStops

    The host asks the bot to shut down.

A command the host asks for from inside its event handler, the moment the triggering event is delivered.

Queued, its Wrote lands after the batch; run inline, inside it.

pub type Reentry {
  Reentry(on: Trigger, send: command.Command)
}

Constructors

How a scenario went.

pub type Report {
  Report(name: String, why: String, verdict: Verdict)
}

Constructors

  • Report(name: String, why: String, verdict: Verdict)

A script, and what the adapter should have done by the end of it.

pub type Scenario {
  Scenario(
    name: String,
    why: String,
    beats: List(Beat),
    expect: List(Act),
    reentrant: option.Option(Reentry),
  )
}

Constructors

  • Scenario(
      name: String,
      why: String,
      beats: List(Beat),
      expect: List(Act),
      reentrant: option.Option(Reentry),
    )

    Arguments

    why

    What breaks if this one fails, in one sentence. Printed by describe.

    expect

    Every act, in order, including the ones the implicit start produces.

The opcode of a frame the adapter sent. Named rather than numbered, and prefixed so the constructors do not collide with gateway.Heartbeat.

pub type Sent {
  SentHeartbeat
  SentIdentify
  SentPresence
  SentResume
  SentOther(op: Int)
}

Constructors

  • SentHeartbeat

    Op 1.

  • SentIdentify

    Op 2.

  • SentPresence

    Op 3.

  • SentResume

    Op 6.

  • SentOther(op: Int)

    An opcode no scenario expects by name, op 4 and op 8 among them.

Everything an adapter needs to build the bot under test.

pub type Setup {
  Setup(
    shard: gateway.Shard,
    world: World,
    transport: client.Transport(World),
    reentrant: option.Option(Reentry),
  )
}

Constructors

Which event a Reentry waits for. A constructor and not a name, so a hook cannot quietly never fire because the scenario wrote “READY” where the core says Ready.

pub type Trigger {
  OnReady
  OnResumed
  OnDispatch(name: String)
  OnReconnecting
  OnHalted
}

Constructors

  • OnReady
  • OnResumed
  • OnDispatch(name: String)

    By the Discord event name gateway.Dispatch carries.

  • OnReconnecting
  • OnHalted
pub type Verdict {
  Passed
  Diverged(
    at: Int,
    expected: option.Option(Act),
    got: option.Option(Act),
    trace: List(Act),
    wanted: List(Act),
  )
  Stalled(trace: List(Act))
  Unplayable(
    at: Int,
    beat: Beat,
    trace: List(Act),
    wanted: List(Act),
  )
}

Constructors

  • Passed
  • Diverged(
      at: Int,
      expected: option.Option(Act),
      got: option.Option(Act),
      trace: List(Act),
      wanted: List(Act),
    )

    The traces agree up to at and disagree there. expected and got are None when one trace ran out.

  • Stalled(trace: List(Act))

    A wait fired testing.max_firings timers and time had not reached the end of the beat. The run stopped there, so the trace is the one the spin left.

  • Unplayable(
      at: Int,
      beat: Beat,
      trace: List(Act),
      wanted: List(Act),
    )

    Beat at asked for something the world could not do: a socket that was never opened, or a close nobody was waiting on. The trace was still the start of wanted when it happened, so either the beat is out of order or the adapter never produced the act it needed. Which of the two it is cannot be read off the trace, so neither is asserted. The run stopped there rather than scoring the beats that did play.

What the scripted socket and clock have seen the adapter do. The adapter threads it as its own state, where a real transport writes to a socket.

pub type World {
  World(
    log: List(Act),
    socket: option.Option(gateway.Conn),
    closing: option.Option(#(gateway.Conn, Int)),
    pending: List(Arming),
  )
}

Constructors

  • World(
      log: List(Act),
      socket: option.Option(gateway.Conn),
      closing: option.Option(#(gateway.Conn, Int)),
      pending: List(Arming),
    )

    Arguments

    log

    Newest first while running. acts reverses it.

    socket

    The connection the adapter was handed by the last open.

    closing

    The connection whose close the adapter asked for and which has not reported back yet. ShutConfirmed is the report.

    pending

    Every timer the adapter has armed and not yet fired.

Values

pub fn acts(world: World) -> List(Act)

Everything the adapter did, oldest first.

pub fn blank() -> World

An empty world: nothing dialled, nothing armed, nothing recorded.

pub fn config() -> gateway.Config

The config every scenario runs against. Defaults throughout, so a delay in an expectation traces back to gateway.config.

pub fn describe(report: Report) -> String

A one-line summary, and the two traces side by side when they disagree.

pub fn event_name(event: gateway.Event) -> String

The name an Emitted act carries. A dispatch is named by its Discord event name, so a scenario’s expectation reads Emitted("MESSAGE_CREATE"). Reconnecting and Halted never reach it: they have acts of their own, because what they carry is the point of them.

pub fn failed(report: Report) -> Bool
pub fn over_client() -> Adapter(client.Bot(World))

The Adapter for a bot driven by glyde/client, and the shape to copy.

pub fn passed(report: Report) -> Bool
pub fn recorder() -> client.Transport(World)

The transport a scenario runs the adapter against. Opens nothing, writes nothing, waits for nothing.

pub fn recorder_in(
  read read: fn(carrier) -> World,
  write write: fn(carrier, World) -> carrier,
) -> client.Transport(carrier)

A recorder for an adapter whose transport threads more than the world. Say how to read the world out of the carrier and how to put it back.

adapter.recorder_in(
  read: fn(app: App) { app.user },
  write: fn(app, world) { App(..app, user: world) },
)
pub fn run(scenario: Scenario, adapter: Adapter(bot)) -> Report

Run one.

pub fn run_all(adapter: Adapter(bot)) -> List(Report)

Run every scenario. The order is scenarios(), so a run is comparable between adapters line by line.

pub fn saw(world: World, event: gateway.Event) -> World

Record that an event reached the host: the one part of the trace a transport cannot see. over_client shows the one line it takes.

pub fn scenarios() -> List(Scenario)

Every scenario, in the order run_all runs them.

pub const seed: Int

The seed every scenario runs with. The expected Armed values below are jittered from it, so an adapter does not get to choose.

pub fn triggered(trigger: Trigger, event: gateway.Event) -> Bool

Whether this event is the one the trigger waits for. An adapter that builds its own bot matches its re-entrancy hook with this, as over_client does.

Search Document