# How to measure startup on your host

Measure the install and startup figures that
[Getting started](getting-started.md#how-long-each-step-takes) quotes, on your
own machine, with the `i9` you installed.

## Prerequisites

- i9 installed in `~/.i9/bin/i9`. See [How to start a VM](how-to-start-a-vm.md).
- Linux with a readable `/dev/kvm`, or macOS on Apple silicon.
- Network access to `i9.io` and `a.i9.io`, and `sh`, `curl`, `awk` and `sort`.
- No other VM running for your user: each measurement starts its own VM, and
  the free tier counts running VMs against a 4-vCPU cap.

## Step 1: Save the script

Save this script as `measure_startup.sh`. It creates scratch homes in a
temporary directory and removes them, and the VMs it started, when it exits:

```sh
#!/bin/sh
# Measure the startup figures the public docs quote, with a public i9 binary.
#
#   measure_startup.sh PATH-TO-I9
#
# Prints the median of each figure and the platform, in the shape the docs
# quote ("measured on <platform>, median of n runs"):
#   install cold    the installer into a fresh home (binary download, no VM)
#   install warm    the installer over a home that already has the binary
#   first start     `i9 run <manifest> --session --detach` in a fresh home
#   first exec      the first `i9 vm exec -- echo` after that start
#   warm exec       `i9 vm exec -- echo` on the running VM
#
# Environment: COLD_RUNS (3), WARM_RUNS (5), START_RUNS (5), EXEC_RUNS (5).
# Host caches are not flushed and the host is not idle-checked, so state the
# host and its load next to the numbers when you publish them. Needs /dev/kvm
# (Linux) or Apple silicon (macOS) for the start figures, network access to
# i9.io and a.i9.io, and sh, curl, awk, sort. Nothing is installed outside the
# scratch homes this script creates and removes.
set -eu

I9=${1:?usage: measure_startup.sh PATH-TO-I9}
COLD_RUNS=${COLD_RUNS:-3}
WARM_RUNS=${WARM_RUNS:-5}
START_RUNS=${START_RUNS:-5}
EXEC_RUNS=${EXEC_RUNS:-5}
IMAGE=https://a.i9.io/vm/manifest.json
WORK=$(mktemp -d "${TMPDIR:-/tmp}/i9-measure.XXXXXX")
NAME=measure$$

cleanup() {
  for h in "$WORK"/home-*; do
    [ -d "$h" ] && "$I9" --home "$h" vm stop "$NAME" --discard >/dev/null 2>&1 || true
  done
  rm -rf "$WORK"
}
trap cleanup EXIT INT TERM

now_ms() {
  # GNU date has %N; macOS date prints a literal N, so fall back to whole
  # seconds there.
  ns=$(date +%s%N)
  case $ns in
    *N) echo $(( $(date +%s) * 1000 )) ;;
    *) echo $(( ns / 1000000 )) ;;
  esac
}

# elapsed_ms CMD...  runs the command, stdout and stderr to the work dir.
elapsed_ms() {
  t0=$(now_ms)
  "$@" >>"$WORK/log" 2>&1
  t1=$(now_ms)
  echo $(( t1 - t0 ))
}

# median of the numbers on stdin, one per line.
median() {
  sort -n | awk '{ v[NR] = $1 } END { if (NR == 0) { print "n/a"; exit } print (NR % 2) ? v[(NR + 1) / 2] : (v[NR / 2] + v[NR / 2 + 1]) / 2 }'
}

install_script="$WORK/install.sh"
curl -fsSL https://i9.io/vm -o "$install_script"

cold=""
i=0
while [ "$i" -lt "$COLD_RUNS" ]; do
  i=$((i + 1))
  home="$WORK/home-cold-$i"
  cold="$cold $(elapsed_ms env I9_HOME="$home" I9_NO_BOOT=1 sh "$install_script")"
done

warm=""
i=0
while [ "$i" -lt "$WARM_RUNS" ]; do
  i=$((i + 1))
  warm="$warm $(elapsed_ms env I9_HOME="$WORK/home-cold-1" I9_NO_BOOT=1 sh "$install_script")"
done

start=""
first_exec=""
warm_exec=""
i=0
while [ "$i" -lt "$START_RUNS" ]; do
  i=$((i + 1))
  home="$WORK/home-start-$i"
  start="$start $(elapsed_ms "$I9" --home "$home" run "$IMAGE" --session --detach --name "$NAME")"
  first_exec="$first_exec $(elapsed_ms "$I9" --home "$home" vm exec "$NAME" -- echo ok)"
  if [ "$i" -eq "$START_RUNS" ]; then
    warm_exec=""
    j=0
    while [ "$j" -lt "$EXEC_RUNS" ]; do
      j=$((j + 1))
      warm_exec="$warm_exec $(elapsed_ms "$I9" --home "$home" vm exec "$NAME" -- echo ok)"
    done
  fi
  "$I9" --home "$home" vm stop "$NAME" --discard >/dev/null 2>&1
done

med() { printf '%s\n' $1 | median; }

echo "measured on $(uname -s) $(uname -m), $("$I9" --version 2>&1 | head -n 1), host caches not flushed"
echo "install cold:  median $(med "$cold") ms (n=$COLD_RUNS)"
echo "install warm:  median $(med "$warm") ms (n=$WARM_RUNS)"
echo "first start:   median $(med "$start") ms (n=$START_RUNS)"
echo "first exec:    median $(med "$first_exec") ms (n=$START_RUNS)"
echo "warm exec:     median $(med "$warm_exec") ms (n=$EXEC_RUNS)"
```

## Step 2: Run it

```sh
sh measure_startup.sh ~/.i9/bin/i9
```

`COLD_RUNS`, `WARM_RUNS`, `START_RUNS` and `EXEC_RUNS` set how many times each
step runs (3, 5, 5 and 5 by default).

## Verify

The script prints the platform, the `i9` version and one median per step:

```text
measured on Linux aarch64, i9 vX.Y.Z, host caches not flushed
install cold:  median <ms> ms (n=3)
install warm:  median <ms> ms (n=5)
first start:   median <ms> ms (n=5)
first exec:    median <ms> ms (n=5)
warm exec:     median <ms> ms (n=5)
```

Host caches are not flushed and the host's load is not checked, so state the
host and its load next to any figure you share.
