<!-- AUTO-GENERATED by gen-vdsl-docs.py — do not edit -->

# The `--where` language

`--where` takes a program of VDSL, i9's small query language. A program is a
predicate over one span: the span's own fields, its attributes, its events and
links, and the trust and completeness the registry gives it. The same program
filters `i9 trace search`, `i9 trace tail` and `i9 trace stats` over an archive,
and the live span stream (`where=` on `/api/agent/spans`). Repeat `--where` and
every program must hold.

```
i9 trace search --where 'process.exit.code != 0 and has(process.command_args)' run.i9io
i9 trace tail --where 'status=error and duration > 100ms'
i9 trace stats --by slowest --where 'channel(blk.requests) and trust >= vmm'
```

A program has no loop, no recursion and no user function, so it always ends, and
what it costs is known before it runs.

## What a key names

A key is a span field (`name`, `duration`, `status`, `kind`, `start`, `end`,
`trace_id`, `span_id`, `parent_id`, `source`) or an attribute name (semantic
conventions, or `i9.*`). A field is always the field; reach an attribute spelled
like one by its full name (`i9.span.source`). `duration`, `start` and `end` are
nanoseconds, written as a number or as a duration such as `100ms`. The ids
compare by hex prefix under `=`, `!=` and `in`.

`kind` is the span's kind in both its facets: i9's kind for the span when it
has one (the kind a row prints, such as `i9.exec.window`) and its OTLP kind word
(`internal`, `client`, ...). The two never overlap, since i9's kinds are dotted,
so `=`, `in` and `=~` hold when either facet does, and `!=` and `!~` when
neither does.

A comparison on a key the span does not carry is false, `!=` included: not equal
is not absent. Absence is `not has(key)`.

When both sides are numbers they compare as numbers (an attribute that is a
number, or text that reads as one, against an integer, a float, a duration or a
size such as `4KiB`). Otherwise `=`, `!=` and `in` compare text, and `<`, `<=`,
`>` and `>=` are false. A value written without quotes is text when it is not a
number, `true`, `false`, a duration or a size, so `status=error` and
`i9.span.source!=coalescer` are programs.

`=~` and `!~` match a regex anywhere in the text (`^` and `$` anchor it). Write it
as a literal, `name =~ /^i9\.restore\./`: between the slashes the regex is as
written, and `\/` is a slash. A quoted pattern, `name =~ "^exec"`, is the same
regex, but a quoted string escapes only `\"` and `\\`, so its backslashes double.
`^=` tests a prefix as text, `name ^= "i9.restore."`, a dot being a dot; it costs
one, never a regex's states. There is no `!^=`; write `not name ^= "..."`.
A pattern has literals, `.`, classes such as `[a-z]` and `[^0-9]`, `\d \w \s`,
groups, `|`, and the repeats `* + ?` and `{m}`, `{m,}`, `{m,n}` up to 32. It has no
backreference and no lookaround: it compiles to one automaton that reads each byte
once.

## Channels and trust

`channel(name)` holds for a span of that registry channel; an unknown name is
refused. Examples are `blk.requests`, `blk.completions` and `egress`; a
refusal for an unknown name is a mistyped name.

`trust >= label` holds for a span whose source has that trust label or one above
it. The order is `bottom` < `guest` < `agent` and `kernel` < `i9guest` < `proxy`,
`vmm` and `net`; `agent` and `kernel` are not comparable with each other.
`complete`, `incomplete` and `unknown` test what is known about a span: `incomplete`
is a span still open, `unknown` one whose source has no trust label, and
`complete` neither.

## Cookbook

Each entry is a question, a program that answers it, why the names in it keep
meaning what they mean, and the spellings that look right and are refused. Every
program below is run by the project's tests, so what is written here holds.

### Process exits and signals

**Which commands exited with a non-zero code?**

```
i9 trace search --where 'process.exit.code != 0' --session <PID>
```

process.exit.code is a semantic-convention name of the proc channel, and a comparison on a span that lacks it is false, so only spans that exited are read.

Checked against a recorded session: 6 rows.

| Refused | Hint | Write |
|---|---|---|
| `process.exit.code ! 0` | `trace.where-syntax` | `process.exit.code != 0` |

**Which commands exited with code 1?**

```
i9 trace tail --where 'process.exit.code = 1' --session <PID>
```

process.exit.code is a number, so it compares as one against an integer written without quotes.

Checked against a recorded session: 2 rows.

| Refused | Hint | Write |
|---|---|---|
| `process.exit.code == 1` | `trace.where-syntax` | `process.exit.code = 1` |

**Which commands exited with one of several known codes?**

```
i9 trace search --where 'process.exit.code in [1, 3, 127, 139]' --session <PID>
```

process.exit.code is a proc channel name; `in` lists the literals it may equal and costs one per literal.

Checked against a recorded session: 5 rows.

| Refused | Hint | Write |
|---|---|---|
| `process.exit.code in 1, 3` | `trace.where-syntax` | `process.exit.code in [1, 3, 127, 139]` |

**Which process was killed by a signal?**

```
i9 trace search --where 'has(i9.process.signal)' --session <PID>
```

i9.process.signal is an i9 name of the proc channel, carried only by a process that a signal ended; has(key) is the one spelling of presence.

Checked against a recorded session: 1 row.

| Refused | Hint | Write |
|---|---|---|
| `i9.process.signal?` | `trace.where-legacy-form` | `has(i9.process.signal)` |
| `!i9.process.signal` | `trace.where-legacy-form` | `not has(i9.process.signal)` |

### Exec arguments and missing commands

**What did each failed command run? Which failures recorded their arguments?**

```
i9 trace search --where 'process.exit.code != 0 and has(process.command_args)' --session <PID>
```

process.command_args is a semantic-convention name of the proc channel and is an array, so a program tests that it is present and the row shows its value.

Checked against a recorded session: 5 rows.

| Refused | Hint | Write |
|---|---|---|
| `process.command_args?` | `trace.where-legacy-form` | `has(process.command_args)` |

**Which command could not be found to run?**

```
i9 trace search --where 'process.exit.code = 127 and has(process.executable.name)' --session <PID>
```

A shell reports a missing command as exit code 127; process.exit.code and process.executable.name are both names of the proc channel.

Checked against a recorded session: 1 row.

| Refused | Hint | Write |
|---|---|---|
| `exists(process.executable.name)` | `trace.where-syntax` | `has(process.executable.name)` |

**Which exec failed with ENOENT, and which path did it try?**

```
i9 trace search --where 'i9.process.exec_errno = 2 and has(process.executable.path)' --session <PID>
```

The kernel records every execve that fails before it runs, with the errno and the path it tried: i9.process.exec_errno and process.executable.path are names of the proc channel, stamped by the host decoder from the kernel-exec-fail record.

Checked against a recorded session: 2 rows.

| Refused | Hint | Write |
|---|---|---|
| `exists(process.executable.path)` | `trace.where-syntax` | `has(process.executable.path)` |

### Durations

**Which commands that ran to an exit took longer than a second?**

```
i9 trace search --where 'duration > 1s and has(process.exit.code)' --session <PID>
```

duration is a span field, always the field, in nanoseconds; a duration literal such as 1s or 100ms is a number of nanoseconds.

Checked against a recorded session: 3 rows.

| Refused | Hint | Write |
|---|---|---|
| `duration_ms>1000` | `trace.where-legacy-form` | `duration > 1s` |
| `duration > soon` | `trace.where-literal` | `duration > 1s` |
| `duration > 1 second` | `trace.where-syntax` | `duration > 1s` |

### Provenance, trust and completeness

**Which spans did the session server record?**

```
i9 trace search --where 'i9.span.source = vm.server' --session <PID>
```

i9.span.source is an i9 name with a closed value set in the registry; vm.server is a member, and a name outside the set matches nothing. A positive comparison on a provenance key names the spans it selects, so the default view shows them without --internal; a negative one, such as i9.span.source != vm.server, leaves internal spans folded and counted.

Checked against a recorded session: 100 rows.

| Refused | Hint | Write |
|---|---|---|
| `!i9.span.source` | `trace.where-legacy-form` | `not has(i9.span.source)` |

**Which failures did the kernel itself witness, not a guest report?**

```
i9 trace search --where 'trust >= kernel and process.exit.code != 0' --session <PID>
```

trust is the label of the span's source in the trust order (bottom, guest, agent, kernel, i9guest, proxy, vmm, net); `trust >= kernel` holds for the kernel and everything above it.

Checked against a recorded session: 6 rows.

| Refused | Hint | Write |
|---|---|---|
| `trust >= root` | `trace.where-trust-label` | `trust >= kernel` |
| `trust >=` | `trace.where-trust-label` | `trust >= kernel` |

**Which spans come from a source that has no trust label?**

```
i9 trace search --where 'unknown' --session <PID>
```

unknown is a completeness label, not a third truth value: it holds for a span whose source has no row in the trust order, so nothing says how far to believe it.

Checked against a recorded session: 34 rows.

### Open spans and the --since window

**Which spans have not ended yet?**

```
i9 trace search --where 'incomplete' --session <PID>
```

incomplete is a completeness label, not a field: it holds for a span that has not ended and never for one that has, so no end time or clock is written in the program.

Checked against a recorded session: no row.

### Gaps and how a span joined its trace

**Which exits were recorded with no trace they could be attached to?**

```
i9 trace search --where 'i9.trace.join = blind-root and has(process.exit.code)' --session <PID>
```

i9.trace.join states how a span joined its trace, from a closed value set of the registry (carrier, ancestry, root, blind-root); blind-root is a span nothing placed.

Checked against a recorded session: no row, which is the answer for a session that lost nothing.

**Did the recorder lose anything, anywhere in this session?**

```
i9 trace search --where 'event(i9.gap) or has(i9.gap.kind)' --session <PID>
```

i9.gap is a registry channel and i9.gap.kind one of its names; a gap is carried as an i9.gap event, so a clean session returns no row and the answer is the empty result.

Checked against a recorded session: no row, which is the answer for a session that lost nothing.

**Which spans record spans dropped or held back, rather than not tapped?**

```
i9 trace search --where 'i9.gap.kind in [dropped, backpressure]' --session <PID>
```

i9.gap.kind has a closed value set in the registry (dropped, not_tapped, interval_not_recorded, unknown_exit, backpressure); a name outside the set matches nothing.

Checked against a recorded session: no row, which is the answer for a session that lost nothing.

| Refused | Hint | Write |
|---|---|---|
| `i9.gap.kind in dropped` | `trace.where-syntax` | `i9.gap.kind in [dropped, backpressure]` |

### Egress and the network

**What did the guest send out through the egress proxy?**

```
i9 trace search --where 'channel(egress)' --session <PID>
```

egress is a row of the channel registry (`i9 schema io-channels`); channel() takes a registry name and refuses a name that is not one.

Checked against a recorded session: no row.

| Refused | Hint | Write |
|---|---|---|
| `channel(egres)` | `trace.where-unknown-channel` | `channel(egress)` |

### Rollups and counters with stats --by

**How many commands ended with each exit code?**

```
i9 trace stats --by by-attr --attr process.exit.code --where 'has(process.exit.code)' --session <PID>
```

stats --by by-attr groups the spans a --where program keeps by the value of one attribute; process.exit.code is a registry name, and the program keeps only spans that exited.

Checked against a recorded session: 6 groups.

**How long was each of the workload's commands running, counting copies that ran at once only once?**

```
i9 trace stats --by busy --where '@workload' --session <PID>
```

stats --by busy is the wall time the spans a --where program keeps covered, the union of their intervals, so parallel copies of a command count once; --group LABEL=PROGRAM gives a row per program instead of per name, and an open span is clipped at the window's end.

Checked against a recorded session: 8 groups.

**What did the counters read at the last cut of the session?**

```
i9 trace stats --by counters --session <PID>
```

stats --by counters reads the counter nodes the drain cut, one value per registry counter; it takes no --where, since a cut is a whole-session reading rather than a set of spans.

Checked against a recorded session: the cut is printed.

### The live span stream

**Stream only the failed commands as they happen.**

```
i9 trace tail --otlp --where 'status = error and has(process.exit.code)' --session <PID>
```

The span stream's where= and --where of i9 trace are judged by one evaluator, so a program that works over an archive works unchanged on GET /api/agent/spans?where=; status is a span field and process.exit.code a registry name.

Checked against a recorded session: 17 spans.

| Refused | Hint | Write |
|---|---|---|
| `status = error && has(process.exit.code)` | `trace.where-syntax` | `status = error and has(process.exit.code)` |

### Process trees and ancestry

**Which processes ran as part of the build, at any depth below the build script?**

```
i9 trace search --where 'under(name = "build.sh")' --session <PID>
```

under() holds for a span when the span or any span on its parent chain satisfies the program inside it, each judged on its own fields, and name is the program a process span ran; a script started as bash build.sh is named bash, and its body is an argv match anchored to the script, process.command_args =~ /^bash build[.]sh/. The chain is span_id and parent_id over the spans the verb read, so a phase is its script and everything the script started, whatever the toolchain names its tools. A span whose chain leaves the spans read is not selected, and the verb says how many were.

| Refused | Hint | Write |
|---|---|---|
| `under()` | `trace.where-syntax` | `under(name = "build.sh")` |
| `under(under(name = "build.sh"))` | `trace.where-syntax` | `under(name = "build.sh")` |

### Patterns and ids

**Which spans are the shell or which?**

```
i9 trace search --where 'name =~ /^(sh|which)$/' --session <PID>
```

name is a span field; =~ matches a regex literal, written between slashes with its backslashes as the regex has them, that compiles to a bounded automaton anchored with ^ and $, with no backreference or lookaround. A plain prefix is ^=, such as name ^= "i9.restore.", where a dot is a dot.

Checked against a recorded session: 8 rows.

| Refused | Hint | Write |
|---|---|---|
| `name =~ /(sh/` | `trace.where-regex` | `name =~ /^sh$/` |
| `name =~ "^exec\.run$"` | `trace.where-syntax` | `name =~ /^exec\.run$/` |

**Which spans hang under the span whose id starts with these digits?**

```
i9 trace search --where 'parent_id = aaaaaaaa' --session <PID>
```

trace_id, span_id and parent_id are span fields that compare by hex prefix under =, != and in, so a prefix copied from `i9 trace show` is enough and no full id is needed; the same form selects a whole trace through trace_id.

| Refused | Hint | Write |
|---|---|---|
| `parent_id = xyz` | `trace.where-literal` | `parent_id = aaaaaaaa` |

### Named predicates

A program can name a predicate with `@name`. Each name stands for the program beside it, and is replaced by that program before the program is costed, so you can always write it out by hand instead.

| Name | Stands for | What it selects |
|---|---|---|
| `@workload` | `i9.proc.class = "user"` | The guest's own workload processes: what the user ran, not the guest agent's or the kernel's. |
| `@failed` | `@workload and process.exit.code != 0` | A workload process that exited with a non-zero code. |
| `@capture` | `name = "i9.snapshot.pause"` | The VMM pausing the guest to capture a snapshot: the moment a saved state is taken. |
| `@restore` | `name = "i9.restore.stage"` | One stage of a VM restore, from registering guest memory to resuming its vCPUs. |
| `@vmm` | `i9.span.source in [vmm, restore]` | Every span the VMM or its restore path emitted, folded or not: vCPU exits, memory, capture, the trace ring and restore stages. |

**Which processes did the guest's own workload run, leaving out the guest agent's and the kernel's?**

```
i9 trace search --where '@workload' --session <PID>
```

@workload is a row of i9's named-predicate table and stands for i9.proc.class = "user"; i9 docs vdsl prints every name with the program it expands to, and a name not in the table is refused.

Checked against a recorded session: 15 rows.

| Refused | Hint | Write |
|---|---|---|
| `@workloads` | `trace.where-unknown-predicate` | `@workload` |
| `status = error and @nosuch` | `trace.where-unknown-predicate` | `status = error and @failed` |

**Show every vCPU exit the VMM recorded.**

```
i9 trace search --where 'name = "i9.vcpu.exit"' --session <PID>
```

Naming a span by its name is a positive comparison on an identity key, so the internal spans it selects are shown as a --target's would be, with no --internal.

Checked against a recorded session: 200 rows.

## Grammar

```ebnf
program      = ws , or-expr , ws ;
or-expr      = and-expr , { ws , "or" , ws , and-expr } ;
and-expr     = not-expr , { ws , "and" , ws , not-expr } ;
not-expr     = "not" , ws , not-expr | primary ;
primary      = "(" , ws , or-expr , ws , ")" | under | predicate-ref | predicate ;
under        = "under" , "(" , ws , or-expr , ws , ")" ;
predicate-ref= "@" , predicate-name ;
predicate-name= letter , { letter | digit | "-" } ;
predicate    = "has" , "(" , key , ")" | "channel" , "(" , channel-name , ")" | "event" , "(" , event-name , ")" | "link" , "(" , ")" | "complete" | "incomplete" | "unknown" | "trust" , ws , ">=" , ws , trust-label | key , ws , "in" , ws , list | key , ws , ( "=~" | "!~" ) , ws , regex-literal | key , ws , compare-op , ws , literal ;
compare-op   = "=" | "!=" | "<" | "<=" | ">" | ">=" | "=~" | "!~" | "^=" ;
regex-literal= "/" , { char - "/" - "\" | "\" , char } , "/" ;
list         = "[" , ws , literal , { ws , "," , ws , literal } , ws , "]" ;
literal      = string | float | int | bool | duration | bytes | bare-word ;
string       = '"' , { char - '"' - "\" | "\" , ( '"' | "\" ) } , '"' ;
int          = [ "-" ] , digit , { digit } ;
float        = [ "-" ] , digit , { digit } , "." , digit , { digit } ;
bool         = "true" | "false" ;
duration     = int , ( "ns" | "us" | "ms" | "s" | "m" | "h" ) ;
bytes        = int , ( "B" | "KiB" | "MiB" | "GiB" | "TiB" ) ;
bare-word    = word-char , { word-char } ;
key          = ( letter | "_" ) , { letter | digit | "." | "_" | ":" | "/" | "-" } | string ;
channel-name = letter , { letter | digit | "." | "_" } ;
event-name   = key ;
trust-label  = letter , { letter | digit } ;
ws           = { " " | tab | newline } ;
```

## Cost

The cost of a program is the number of its nodes: each `and`, `or` and `not`
costs 1, each predicate costs 1, `in` adds 1 per listed value, and a pattern adds
the number of states of its automaton. The cost is fixed when the program is
compiled and the evaluator charges every node it visits against it, so a program
never spends more than it was admitted with. A program above the cost bound of
512, a pattern above 128 states, or program text above 4096 bytes is refused
before it runs.

| Production | Cost |
|---|---|
| `program` | 0: the root adds nothing; its expression carries the cost |
| `or-expr` | 1 per `or`, plus its operands |
| `and-expr` | 1 per `and`, plus its operands |
| `not-expr` | 1 per `not`, plus its operand |
| `primary` | 0: parentheses group and are free |
| `under` | 1, plus its body (VDSL15). `under(P)` holds when the span or a span on its parent chain satisfies P, each judged with its own fields. The chain is `span_id` and `parent_id` over the set the verb read; a chain that leaves the set, or loops, is unknown: the span is not selected and the verb states how many were. A body holds no `under`: an ancestor's ancestor is already its ancestor. |
| `predicate-ref` | 0: a named predicate costs what its body costs. The name is replaced by its body before the program is costed (VDSL11), so the bounds are those of the program written out. The names and bodies are the named-predicate table this i9 is built with; a name not in it is refused. |
| `predicate-name` | 0: a lowercase name of the predicate table |
| `predicate` | 1 per predicate; `in` adds one per listed literal; `=~` and `!~` add the states of the regex, written as a regex literal or a quoted pattern; `^=` is a prefix test and costs 1; `link()` and the label tests cost 1 |
| `compare-op` | 0: `^=` holds when the key's text starts with the literal; there is no `!^=`, negation is `not` |
| `regex-literal` | 0: the regex as written; `\/` is the only escape the language reads, every other backslash is the regex's. The same automaton and state cap as the quoted pattern with the same text (VDSL13) |
| `list` | 0: the cost of a list is its length, charged by the `in` above |
| `literal` | 0 |
| `string` | 0: a quoted string; `\"` and `\\` are its only escapes |
| `int` | 0 |
| `float` | 0 |
| `bool` | 0 |
| `duration` | 0: a number followed by ns, us, ms, s, m or h; its value is nanoseconds |
| `bytes` | 0: a number followed by B, KiB, MiB, GiB or TiB; its value is bytes |
| `bare-word` | 0: letters, digits and . _ : / @ + - % # ~, not starting as a number |
| `key` | 0: letters, digits, `.`, `_`, `:`, `/`, `-`; a quoted string also names a key |
| `channel-name` | 0: a registry channel name, `i9 schema io-channels` prints them; an unknown name is refused when the program compiles (VDSL02) |
| `event-name` | 0 |
| `trust-label` | 0: one of the eight labels of the trust order (TSUB04): bottom, guest, agent, kernel, i9guest, proxy, vmm, net. `trust >= L` holds when the span's trust is L or above it; agent and kernel are incomparable |
| `ws` | 0 |

## Fixture spans

The examples below run against these spans.

| Span | Fields and attributes |
|---|---|
| `s1` | `name=exec.run status=error kind=tool start=1000000000 end=1250000000 source=kernel-exit trace_id=000000000000000000000000000000aa span_id=00000000000000000000000000000001 process.exit.code=1 process.pid=42 process.command_args="ls -l"` |
| `s2` | `name=i9.vmm.virtio.blk.req status=ok kind=internal start=2000000000 end=2000004000 source=vmm trace_id=000000000000000000000000000000aa span_id=00000000000000000000000000000002 parent_id=aaaaaaaa00000000000000000000aaaa blk.bytes=4096 events=retry|done links=1` |
| `s3` | `name=i9.egress status=ok kind=client start=3000000000 end=0 source=proxy span_id=00000000000000000000000000000003 http.request.method=GET http.response.status_code=200 url.full="https://example.com/a?b=1"` |
| `s4` | `name=i9.guest.rpc status=unset kind=internal start=4000000000 end=4000500000 source=guest-agent span_id=00000000000000000000000000000004 process.pid=7 gen_ai.usage.input_tokens=1500 latency.ratio=0.75 cached=true` |
| `s5` | `name=mystery status=ok kind=internal start=5000000000 end=5000000100 source=zzz span_id=00000000000000000000000000000005 process.pid=1` |
| `s6` | `name=i9.vm.admission status=ok kind=internal start=6000000000 end=6000100000 source=vm.server span_id=00000000000000000000000000000006` |
| `s7` | `name=i9.trace.ring status=ok kind=internal start=7000000000 end=7000000500 source=vmm span_id=00000000000000000000000000000007 i9.gap.kind=dropped i9.gap.count=12 events=i9.gap` |
| `s8` | `name=sh status=error kind=internal start=8000000000 end=8001000000 source=kernel-exit span_id=00000000000000000000000000000008 process.exit.code=139 i9.process.signal=11 process.command_args="sh -c kill"` |
| `s9` | `name=/no/such/binary status=error kind=internal start=9000000000 end=9000100000 source=kernel-exit span_id=00000000000000000000000000000009 process.exit.code=127 process.executable.name=/no/such/binary process.command_args="/no/such/binary"` |
| `s10` | `name=sh status=ok kind=internal start=10000000000 end=10000200000 source=kernel-exit span_id=0000000000000000000000000000000a process.exit.code=0 i9.trace.join=blind-root` |
| `s11` | `name=curl status=error kind=internal start=11000000000 end=13000000000 source=kernel-exit span_id=0000000000000000000000000000000b process.exit.code=7` |
| `s14` | `name=python3 status=error kind=internal start=12000000000 end=12400000000 source=kernel-exit span_id=0000000000000000000000000000000c process.exit.code=1 i9.proc.class=user` |
| `s15` | `name=init status=ok kind=internal start=13000000000 end=13000100000 source=kernel-exit span_id=0000000000000000000000000000000d process.exit.code=0 i9.proc.class=agent` |
| `s16` | `name=i9.vcpu.exit status=ok kind=internal start=14000000000 end=14000002000 source=vmm span_id=0000000000000000000000000000000e i9.vcpu.exit.reason=mmio` |
| `s12` | `name=/no/such/binary status=error kind=internal start=12000000000 end=12000300000 source=kernel-exec-fail span_id=0000000000000000000000000000000c i9.process.exec_errno=2 process.executable.path=/no/such/binary process.args_count=1 process.command_args="/no/such/binary"` |
| `s13` | `name=window status=ok kind=internal start=13000000000 end=13001000000 source=vm.server span_id=0000000000000000000000000000000d row_kind=i9.exec.window` |
| `t1` | `name=build.sh status=ok kind=internal start=20000000000 end=29000000000 source=kernel-exit span_id=00000000000000000000000000000101 process.exit.code=0` |
| `t2` | `name=make status=ok kind=internal start=20100000000 end=28000000000 source=kernel-exit span_id=00000000000000000000000000000102 parent_id=00000000000000000000000000000101 process.exit.code=0` |
| `t3` | `name=cc1 status=error kind=internal start=20200000000 end=20900000000 source=kernel-exit span_id=00000000000000000000000000000103 parent_id=00000000000000000000000000000102 process.exit.code=1` |
| `t4` | `name=orphan status=ok kind=internal start=21000000000 end=21100000000 source=kernel-exit span_id=00000000000000000000000000000104 parent_id=000000000000000000000000000001ff process.exit.code=0` |
| `t5` | `name=loop-a status=ok kind=internal start=22000000000 end=22100000000 source=kernel-exit span_id=00000000000000000000000000000105 parent_id=00000000000000000000000000000106` |
| `t6` | `name=loop-b status=ok kind=internal start=22000000000 end=22100000000 source=kernel-exit span_id=00000000000000000000000000000106 parent_id=00000000000000000000000000000105` |

## Examples

| Program | Span | Holds |
|---|---|---|
| `@workload` | `s14` | true |
| `@workload` | `s15` | false |
| `@failed` | `s14` | true |
| `@failed` | `s15` | false |
| `@workload and not @capture` | `s14` | true |
| `status = error` | `s1` | true |
| `status = ok` | `s1` | false |
| `status=error` | `s1` | true |
| `i9.span.source=kernel-exit` | `s1` | true |
| `i9.span.source!=coalescer` | `s1` | true |
| `i9.span.source!=vmm` | `s2` | false |
| `kind = tool` | `s1` | true |
| `kind = i9.exec.window` | `s13` | true |
| `kind = internal` | `s13` | true |
| `kind != i9.exec.window` | `s13` | false |
| `kind != internal` | `s13` | false |
| `kind != tool` | `s13` | true |
| `kind in ["i9.exec.window", "i9.exec.run"]` | `s13` | true |
| `kind =~ "^i9\\.exec\\."` | `s13` | true |
| `kind !~ "^i9\\.exec\\."` | `s13` | false |
| `kind =~ /^i9\.exec\./` | `s13` | true |
| `kind !~ /^i9\.exec\./` | `s13` | false |
| `kind ^= "i9.exec."` | `s13` | true |
| `kind ^= i9.run` | `s13` | false |
| `kind = i9.exec.window` | `s2` | false |
| `kind != i9.exec.window` | `s2` | true |
| `source = kernel-exit` | `s1` | true |
| `duration > 100ms and duration <= 250ms` | `s1` | true |
| `duration < 10ms` | `s1` | false |
| `duration >= 250000000` | `s1` | true |
| `start >= 1000000000 and end <= 2000000000` | `s1` | true |
| `blk.bytes >= 4KiB` | `s2` | true |
| `blk.bytes > 4KiB` | `s2` | false |
| `process.exit.code != 0` | `s1` | true |
| `process.exit.code != 0` | `s2` | false |
| `process.exit.code = 0` | `s2` | false |
| `has(process.exit.code)` | `s2` | false |
| `not has(process.exit.code)` | `s2` | true |
| `process.exit.code in [1, 2, 3]` | `s1` | true |
| `process.exit.code in [2, 3]` | `s1` | false |
| `name in ["exec.run", "exec.spawn"]` | `s1` | true |
| `name =~ "^exec\\.(run\|spawn)$"` | `s1` | true |
| `name =~ "^exec\\.(run\|spawn)$"` | `s2` | false |
| `name !~ "^vmm"` | `s1` | true |
| `name =~ /^exec\.(run\|spawn)$/` | `s1` | true |
| `name =~ /^exec\.(run\|spawn)$/` | `s2` | false |
| `url.full =~ /example\.com\/a\?b=1$/` | `s3` | true |
| `process.executable.name =~ /^\/no\/such\//` | `s9` | true |
| `name ^= "exec."` | `s1` | true |
| `name ^= "exec."` | `s2` | false |
| `name ^= i9.vmm.virtio.` | `s2` | true |
| `process.command_args ^= "ls "` | `s1` | true |
| `i9.span.source ^= vm` | `s2` | true |
| `has(process.command_args) and process.command_args ^= ""` | `s2` | false |
| `not name ^= "exec."` | `s1` | false |
| `url.full =~ "example\\.com"` | `s3` | true |
| `url.full = "https://example.com/a?b=1"` | `s3` | true |
| `process.command_args =~ "^ls( -[a-z])*$"` | `s1` | true |
| `process.pid > 1 and has(process.command_args)` | `s1` | true |
| `process.pid > 1 and has(process.command_args)` | `s5` | false |
| `channel(blk.requests)` | `s2` | true |
| `channel(blk.completions)` | `s2` | false |
| `channel(blk.requests)` | `s1` | false |
| `trust >= vmm` | `s2` | true |
| `trust >= vmm` | `s1` | false |
| `trust >= kernel` | `s1` | true |
| `trust >= guest` | `s1` | true |
| `trust >= guest` | `s4` | true |
| `trust >= kernel` | `s4` | false |
| `trust >= guest` | `s5` | false |
| `incomplete` | `s3` | true |
| `incomplete` | `s1` | false |
| `complete` | `s1` | true |
| `unknown` | `s5` | true |
| `unknown` | `s1` | false |
| `event(retry)` | `s2` | true |
| `event(missing)` | `s2` | false |
| `event(retry)` | `s1` | false |
| `link()` | `s2` | true |
| `link()` | `s1` | false |
| `gen_ai.usage.input_tokens > 1000 and cached = true` | `s4` | true |
| `latency.ratio >= 0.5` | `s4` | true |
| `latency.ratio >= 0.8` | `s4` | false |
| `cached = false` | `s4` | false |
| `not cached = false` | `s4` | true |
| `(status = error or status = unset) and not has(parent_id)` | `s1` | true |
| `parent_id = aaaaaaaa` | `s2` | true |
| `parent_id != aaaaaaaa` | `s2` | false |
| `parent_id != aaaaaaaa` | `s1` | false |
| `trace_id = 00000000000000000000000000000aa` | `s1` | false |
| `trace_id = 000000000000000000000000000000aa` | `s1` | true |
| `span_id = 0000` | `s1` | true |
| `status = unset` | `s4` | true |
| `under(name = build.sh)` | `t1` | true |
| `under(name = build.sh)` | `t3` | true |
| `under(name = make) and status = error` | `t3` | true |
| `under(name = make)` | `t1` | false |
| `under(name = cc1)` | `t2` | false |
| `under(process.exit.code != 0)` | `t3` | true |
| `under(name = orphan)` | `t4` | true |
| `under(name = build.sh)` | `t4` | false |
| `not under(name = build.sh)` | `t4` | false |
| `under(name = loop-b)` | `t5` | true |
| `under(name = build.sh)` | `t5` | false |
| `process.exit.code != 0` | `s1` | true |
| `process.exit.code != 0` | `s2` | false |
| `process.exit.code = 1` | `s1` | true |
| `process.exit.code = 1` | `s2` | false |
| `process.exit.code in [1, 3, 127, 139]` | `s1` | true |
| `process.exit.code in [1, 3, 127, 139]` | `s2` | false |
| `has(i9.process.signal)` | `s8` | true |
| `has(i9.process.signal)` | `s1` | false |
| `process.exit.code != 0 and has(process.command_args)` | `s1` | true |
| `process.exit.code != 0 and has(process.command_args)` | `s2` | false |
| `process.exit.code = 127 and has(process.executable.name)` | `s9` | true |
| `process.exit.code = 127 and has(process.executable.name)` | `s1` | false |
| `i9.process.exec_errno = 2 and has(process.executable.path)` | `s12` | true |
| `i9.process.exec_errno = 2 and has(process.executable.path)` | `s9` | false |
| `duration > 1s and has(process.exit.code)` | `s11` | true |
| `duration > 1s and has(process.exit.code)` | `s1` | false |
| `i9.span.source = vm.server` | `s6` | true |
| `i9.span.source = vm.server` | `s1` | false |
| `trust >= kernel and process.exit.code != 0` | `s1` | true |
| `trust >= kernel and process.exit.code != 0` | `s2` | false |
| `trust >= kernel and process.exit.code != 0` | `s4` | false |
| `unknown` | `s5` | true |
| `unknown` | `s1` | false |
| `incomplete` | `s3` | true |
| `incomplete` | `s1` | false |
| `i9.trace.join = blind-root and has(process.exit.code)` | `s10` | true |
| `i9.trace.join = blind-root and has(process.exit.code)` | `s1` | false |
| `event(i9.gap) or has(i9.gap.kind)` | `s7` | true |
| `event(i9.gap) or has(i9.gap.kind)` | `s1` | false |
| `i9.gap.kind in [dropped, backpressure]` | `s7` | true |
| `i9.gap.kind in [dropped, backpressure]` | `s1` | false |
| `channel(egress)` | `s3` | true |
| `channel(egress)` | `s1` | false |
| `has(process.exit.code)` | `s1` | true |
| `has(process.exit.code)` | `s2` | false |
| `@workload` | `s14` | true |
| `@workload` | `s15` | false |
| `status = error and has(process.exit.code)` | `s1` | true |
| `status = error and has(process.exit.code)` | `s2` | false |
| `name =~ /^(sh\|which)$/` | `s8` | true |
| `name =~ /^(sh\|which)$/` | `s1` | false |
| `under(name = "build.sh")` | `t3` | true |
| `under(name = "build.sh")` | `t4` | false |
| `parent_id = aaaaaaaa` | `s2` | true |
| `parent_id = aaaaaaaa` | `s1` | false |
| `name = "i9.vcpu.exit"` | `s16` | true |
| `name = "i9.vcpu.exit"` | `s2` | false |

## Refusals

A program the grammar does not take is refused with a hint that names the fix.

| Program | Hint | Fix |
|---|---|---|
| `under()` | `trace.where-syntax` | Write `process.exit.code = 1`, not `process.exit.code == 1`: a --where is comparisons joined by and, or, not; i9 docs vdsl has the grammar. |
| `under(under(name = make))` | `trace.where-syntax` | Write `process.exit.code = 1`, not `process.exit.code == 1`: a --where is comparisons joined by and, or, not; i9 docs vdsl has the grammar. |
| `under(name = make` | `trace.where-syntax` | Write `process.exit.code = 1`, not `process.exit.code == 1`: a --where is comparisons joined by and, or, not; i9 docs vdsl has the grammar. |
| `(empty)` | `trace.where-syntax` | Write `process.exit.code = 1`, not `process.exit.code == 1`: a --where is comparisons joined by and, or, not; i9 docs vdsl has the grammar. |
| `status =` | `trace.where-syntax` | Write `process.exit.code = 1`, not `process.exit.code == 1`: a --where is comparisons joined by and, or, not; i9 docs vdsl has the grammar. |
| `(status = ok` | `trace.where-syntax` | Write `process.exit.code = 1`, not `process.exit.code == 1`: a --where is comparisons joined by and, or, not; i9 docs vdsl has the grammar. |
| `status = ok)` | `trace.where-syntax` | Write `process.exit.code = 1`, not `process.exit.code == 1`: a --where is comparisons joined by and, or, not; i9 docs vdsl has the grammar. |
| `and` | `trace.where-syntax` | Write `process.exit.code = 1`, not `process.exit.code == 1`: a --where is comparisons joined by and, or, not; i9 docs vdsl has the grammar. |
| `status = ok or` | `trace.where-syntax` | Write `process.exit.code = 1`, not `process.exit.code == 1`: a --where is comparisons joined by and, or, not; i9 docs vdsl has the grammar. |
| `status = ok garbage` | `trace.where-syntax` | Write `process.exit.code = 1`, not `process.exit.code == 1`: a --where is comparisons joined by and, or, not; i9 docs vdsl has the grammar. |
| `process.exit.code ! 0` | `trace.where-syntax` | Write `process.exit.code = 1`, not `process.exit.code == 1`: a --where is comparisons joined by and, or, not; i9 docs vdsl has the grammar. |
| `status = [ok` | `trace.where-syntax` | Write `process.exit.code = 1`, not `process.exit.code == 1`: a --where is comparisons joined by and, or, not; i9 docs vdsl has the grammar. |
| `status in []` | `trace.where-syntax` | Write `process.exit.code = 1`, not `process.exit.code == 1`: a --where is comparisons joined by and, or, not; i9 docs vdsl has the grammar. |
| `has()` | `trace.where-syntax` | Write `process.exit.code = 1`, not `process.exit.code == 1`: a --where is comparisons joined by and, or, not; i9 docs vdsl has the grammar. |
| `status = "unterminated` | `trace.where-syntax` | Write `process.exit.code = 1`, not `process.exit.code == 1`: a --where is comparisons joined by and, or, not; i9 docs vdsl has the grammar. |
| `not` | `trace.where-syntax` | Write `process.exit.code = 1`, not `process.exit.code == 1`: a --where is comparisons joined by and, or, not; i9 docs vdsl has the grammar. |
| `process.exit.code?` | `trace.where-legacy-form` | Write `has(i9.process.signal)`, not `i9.process.signal?`: presence is has(key), absence is not has(key), times are duration, start and end. |
| `!i9.span.source` | `trace.where-legacy-form` | Write `has(i9.process.signal)`, not `i9.process.signal?`: presence is has(key), absence is not has(key), times are duration, start and end. |
| `duration_ms>100` | `trace.where-legacy-form` | Write `has(i9.process.signal)`, not `i9.process.signal?`: presence is has(key), absence is not has(key), times are duration, start and end. |
| `start_ns>=1000` | `trace.where-legacy-form` | Write `has(i9.process.signal)`, not `i9.process.signal?`: presence is has(key), absence is not has(key), times are duration, start and end. |
| `trust >= root` | `trace.where-trust-label` | Write `trust >= kernel`, not `trust >= root`: trust >= takes bottom, guest, agent, kernel, i9guest, proxy, vmm or net. |
| `trust >=` | `trace.where-trust-label` | Write `trust >= kernel`, not `trust >= root`: trust >= takes bottom, guest, agent, kernel, i9guest, proxy, vmm or net. |
| `channel(nosuch.channel)` | `trace.where-unknown-channel` | Write `channel(egress)`, not `channel(egres)`: channel() takes a registry name such as blk.requests or egress; i9 docs vdsl lists them. |
| `duration > soon` | `trace.where-literal` | Write `parent_id = aaaaaaaa`, not `parent_id = xyz`: ids take a hex prefix, times a duration such as 100ms, =~ a /regex/, ^= quoted text, < and > a number. |
| `name =~ 5` | `trace.where-literal` | Write `parent_id = aaaaaaaa`, not `parent_id = xyz`: ids take a hex prefix, times a duration such as 100ms, =~ a /regex/, ^= quoted text, < and > a number. |
| `duration > 10lightyears` | `trace.where-literal` | Write `parent_id = aaaaaaaa`, not `parent_id = xyz`: ids take a hex prefix, times a duration such as 100ms, =~ a /regex/, ^= quoted text, < and > a number. |
| `status > "a"` | `trace.where-literal` | Write `parent_id = aaaaaaaa`, not `parent_id = xyz`: ids take a hex prefix, times a duration such as 100ms, =~ a /regex/, ^= quoted text, < and > a number. |
| `name =~ "("` | `trace.where-regex` | A =~ pattern has no backreference or lookaround and a state cap; write a simpler pattern or split it into clauses. |
| `name =~ /(/` | `trace.where-regex` | A =~ pattern has no backreference or lookaround and a state cap; write a simpler pattern or split it into clauses. |
| `name =~ /^exec` | `trace.where-syntax` | Write `process.exit.code = 1`, not `process.exit.code == 1`: a --where is comparisons joined by and, or, not; i9 docs vdsl has the grammar. |
| `name !^= "exec."` | `trace.where-syntax` | Write `process.exit.code = 1`, not `process.exit.code == 1`: a --where is comparisons joined by and, or, not; i9 docs vdsl has the grammar. |
| `name ^= 5` | `trace.where-literal` | Write `parent_id = aaaaaaaa`, not `parent_id = xyz`: ids take a hex prefix, times a duration such as 100ms, =~ a /regex/, ^= quoted text, < and > a number. |
| `duration ^= "1"` | `trace.where-literal` | Write `parent_id = aaaaaaaa`, not `parent_id = xyz`: ids take a hex prefix, times a duration such as 100ms, =~ a /regex/, ^= quoted text, < and > a number. |
| `process.exit.code ^= 1` | `trace.where-literal` | Write `parent_id = aaaaaaaa`, not `parent_id = xyz`: ids take a hex prefix, times a duration such as 100ms, =~ a /regex/, ^= quoted text, < and > a number. |
| `name =~ "a**"` | `trace.where-regex` | A =~ pattern has no backreference or lookaround and a state cap; write a simpler pattern or split it into clauses. |
| `name =~ "[z-a]"` | `trace.where-regex` | A =~ pattern has no backreference or lookaround and a state cap; write a simpler pattern or split it into clauses. |
| `name =~ "(a\|b)*c(d\|e)*f"` (bound 2) | `trace.where-regex` | A =~ pattern has no backreference or lookaround and a state cap; write a simpler pattern or split it into clauses. |
| `a=1 and b=2 and c=3` (bound 4) | `trace.where-cost` | A --where costs its node count, and a regex its states, up to a bound; split it into narrower clauses. |
| `a in [1, 2, 3]` (bound 3) | `trace.where-cost` | A --where costs its node count, and a regex its states, up to a bound; split it into narrower clauses. |
| `process.exit.code ! 0` | `trace.where-syntax` | Write `process.exit.code = 1`, not `process.exit.code == 1`: a --where is comparisons joined by and, or, not; i9 docs vdsl has the grammar. |
| `process.exit.code == 1` | `trace.where-syntax` | Write `process.exit.code = 1`, not `process.exit.code == 1`: a --where is comparisons joined by and, or, not; i9 docs vdsl has the grammar. |
| `process.exit.code in 1, 3` | `trace.where-syntax` | Write `process.exit.code = 1`, not `process.exit.code == 1`: a --where is comparisons joined by and, or, not; i9 docs vdsl has the grammar. |
| `i9.process.signal?` | `trace.where-legacy-form` | Write `has(i9.process.signal)`, not `i9.process.signal?`: presence is has(key), absence is not has(key), times are duration, start and end. |
| `!i9.process.signal` | `trace.where-legacy-form` | Write `has(i9.process.signal)`, not `i9.process.signal?`: presence is has(key), absence is not has(key), times are duration, start and end. |
| `process.command_args?` | `trace.where-legacy-form` | Write `has(i9.process.signal)`, not `i9.process.signal?`: presence is has(key), absence is not has(key), times are duration, start and end. |
| `exists(process.executable.name)` | `trace.where-syntax` | Write `process.exit.code = 1`, not `process.exit.code == 1`: a --where is comparisons joined by and, or, not; i9 docs vdsl has the grammar. |
| `exists(process.executable.path)` | `trace.where-syntax` | Write `process.exit.code = 1`, not `process.exit.code == 1`: a --where is comparisons joined by and, or, not; i9 docs vdsl has the grammar. |
| `duration_ms>1000` | `trace.where-legacy-form` | Write `has(i9.process.signal)`, not `i9.process.signal?`: presence is has(key), absence is not has(key), times are duration, start and end. |
| `duration > soon` | `trace.where-literal` | Write `parent_id = aaaaaaaa`, not `parent_id = xyz`: ids take a hex prefix, times a duration such as 100ms, =~ a /regex/, ^= quoted text, < and > a number. |
| `duration > 1 second` | `trace.where-syntax` | Write `process.exit.code = 1`, not `process.exit.code == 1`: a --where is comparisons joined by and, or, not; i9 docs vdsl has the grammar. |
| `!i9.span.source` | `trace.where-legacy-form` | Write `has(i9.process.signal)`, not `i9.process.signal?`: presence is has(key), absence is not has(key), times are duration, start and end. |
| `trust >= root` | `trace.where-trust-label` | Write `trust >= kernel`, not `trust >= root`: trust >= takes bottom, guest, agent, kernel, i9guest, proxy, vmm or net. |
| `trust >=` | `trace.where-trust-label` | Write `trust >= kernel`, not `trust >= root`: trust >= takes bottom, guest, agent, kernel, i9guest, proxy, vmm or net. |
| `i9.gap.kind in dropped` | `trace.where-syntax` | Write `process.exit.code = 1`, not `process.exit.code == 1`: a --where is comparisons joined by and, or, not; i9 docs vdsl has the grammar. |
| `channel(egres)` | `trace.where-unknown-channel` | Write `channel(egress)`, not `channel(egres)`: channel() takes a registry name such as blk.requests or egress; i9 docs vdsl lists them. |
| `status = error && has(process.exit.code)` | `trace.where-syntax` | Write `process.exit.code = 1`, not `process.exit.code == 1`: a --where is comparisons joined by and, or, not; i9 docs vdsl has the grammar. |
| `name =~ /(sh/` | `trace.where-regex` | A =~ pattern has no backreference or lookaround and a state cap; write a simpler pattern or split it into clauses. |
| `name =~ "^exec\.run$"` | `trace.where-syntax` | Write `process.exit.code = 1`, not `process.exit.code == 1`: a --where is comparisons joined by and, or, not; i9 docs vdsl has the grammar. |
| `under()` | `trace.where-syntax` | Write `process.exit.code = 1`, not `process.exit.code == 1`: a --where is comparisons joined by and, or, not; i9 docs vdsl has the grammar. |
| `under(under(name = "build.sh"))` | `trace.where-syntax` | Write `process.exit.code = 1`, not `process.exit.code == 1`: a --where is comparisons joined by and, or, not; i9 docs vdsl has the grammar. |
| `parent_id = xyz` | `trace.where-literal` | Write `parent_id = aaaaaaaa`, not `parent_id = xyz`: ids take a hex prefix, times a duration such as 100ms, =~ a /regex/, ^= quoted text, < and > a number. |
| `@workloads` | `trace.where-unknown-predicate` | Write `@workload`, not `@workloads`: a named predicate is a name from the table the refusal lists; i9 docs vdsl prints each with its program. |
| `status = error and @nosuch` | `trace.where-unknown-predicate` | Write `@workload`, not `@workloads`: a named predicate is a name from the table the refusal lists; i9 docs vdsl prints each with its program. |
