> ## Documentation Index
> Fetch the complete documentation index at: https://arkor-92aeef0e-eng-615.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Lifecycle callbacks

> The five callbacks Arkor fires during a training run, and what each one is good for.

Arkor fires five callbacks as a training run progresses. They are all optional, and each one is a plain TypeScript function that runs in your process. This is what makes the loop feel like the rest of your application: no notebook, no out-of-band dashboard, no separate config language.

```ts theme={null}
createTrainer({
  name: "support-bot-v1",
  model: "unsloth/gemma-4-E4B-it",
  dataset: { type: "huggingface", name: "arkorlab/triage-demo" },
  callbacks: {
    onStarted, onLog, onCheckpoint, onCompleted, onFailed,
  },
});
```

## When each callback fires

```
.start()                 ── submits the job, returns { jobId }. No callbacks here.
   │
   ▼
.wait()                  ── opens the SSE event stream. Callbacks dispatch from here.
   │
   ▼
onStarted ─── once, when the stream reports `training.started`
   │
   ▼
onLog     ─── many times, once per training-step batch of metrics
   │
   ▼
onCheckpoint ── several times, when an adapter checkpoint is saved
   │
   ▼
onCompleted  ── once, on successful finish
or
onFailed     ── once, on backend-reported failure
```

All five callbacks are dispatched from inside `wait()`. If you call `start()` without later calling `wait()`, **no callbacks fire**, even though the run is still happening on the backend. `arkor start` calls `wait()` for you; if you wire training into your own code outside the CLI, make sure you do too.

`onCompleted` and `onFailed` are mutually exclusive: at most one of them fires per run. If `wait()` throws before a terminal event arrives (for example when `abortSignal` is aborted, or reconnect attempts are exhausted), it is possible that neither one fires.

Returning a promise from any callback is fine. Arkor awaits it before moving on, so you can do async work (writing to a database, posting to Slack, calling `infer`) without races.

## `onStarted({ job })`

Fires once, when the SSE stream opened by `wait()` reports a `training.started` event. Note that this is not the same moment `start()` resolves: `start()` only submits the job and returns its `jobId`.

```ts theme={null}
onStarted: ({ job }) => {
  console.log(`Run ${job.id} accepted`);
},
```

Use it for log lines, metric counters, or sending a "training started" notification.

## `onLog({ step, loss, evalLoss, learningRate, epoch, samplesPerSecond, job })`

Fires repeatedly as training progresses. Each numeric field can be `null` when the backend has not produced that metric yet (for example `evalLoss` only fires on eval steps).

```ts theme={null}
onLog: ({ step, loss, evalLoss }) => {
  if (loss !== null) {
    console.log(`step=${step} loss=${loss.toFixed(4)} evalLoss=${evalLoss ?? "-"}`);
  }
},
```

Common uses:

* Forward to your own metrics pipeline (e.g. PostHog, Datadog).
* Detect divergence early: if `loss` is climbing, abort `wait()` via `abortSignal` and call `trainer.cancel()` to stop the GPU on the backend.
* Implement custom early stopping (abort a run automatically when metrics regress; see the [Early stopping recipe](/cookbook/early-stopping)).

## `onCheckpoint({ step, adapter, job, infer, artifacts })`

Fires when a checkpoint is saved on the backend, while the run is still going.

```ts theme={null}
onCheckpoint: async ({ step, infer }) => {
  const res = await infer({
    messages: [{ role: "user", content: "Can't log in" }],
  });
  console.log(`step=${step} sample=`, await res.text());
},
```

`adapter` is a small object identifying the checkpoint (`{ kind: "checkpoint", jobId, step }`). `infer` is a function: it takes a chat-style request and returns a raw `Response`. You call `await res.text()` (or `res.json()`, or stream the body) to read it.

This is the most useful callback in practice. It lets you sanity-check the model mid-run rather than waiting until the end. If the checkpoint is already worse than the base model, you know to stop.

## `onCompleted({ job, artifacts })`

Fires once, on success. `artifacts` lists what the backend produced for this run. Use it to:

* Persist the final adapter ID where the rest of your app can find it.
* Run a final smoke test before promoting the model.
* Send a "training done" notification.

```ts theme={null}
onCompleted: ({ job, artifacts }) => {
  console.log(`Run ${job.id} done, ${artifacts.length} artifacts`);
},
```

## `onFailed({ job, error })`

Fires once if the backend reports a failure. Note that `error` is a `string` (the message the backend sent), not an `Error` instance:

```ts theme={null}
onFailed: ({ job, error }) => {
  console.error(`Run ${job.id} failed: ${error}`);
},
```

`onFailed` is for backend-reported failures only. A thrown exception inside one of your own callbacks does **not** route through `onFailed`; it rejects `wait()` immediately with your error. A thrown callback is **not** treated as an SSE transport failure, so it is not routed through the reconnect loop or retried: the reconnect loop handles only transient stream failures (a dropped connection, a retryable HTTP error such as `404`/`408`/`429`/`5xx` on open; permanent statuses like `401`/`403`/`410`/`426` reject immediately, see [Trainer control § Reconnects](/sdk/trainer-control#reconnects)). This is deliberate, because the failing event's `Last-Event-ID` has already advanced, so retrying would skip the event, swallow your error, and (for a terminal `training.completed` event) resolve `wait()` with empty artifacts. If you need non-fatal handling, catch errors inside the callback and decide what to do (log, abort, persist) before they escape.

## A complete example

```ts theme={null}
import { createTrainer } from "arkor";

export const trainer = createTrainer({
  name: "support-bot-v1",
  model: "unsloth/gemma-4-E4B-it",
  dataset: { type: "huggingface", name: "arkorlab/triage-demo" },
  lora: { r: 16, alpha: 16 },
  maxSteps: 100,
  callbacks: {
    onStarted: ({ job }) => console.log(`started ${job.id}`),
    onLog: ({ step, loss }) => {
      if (loss !== null) console.log(`step=${step} loss=${loss.toFixed(4)}`);
    },
    onCheckpoint: async ({ step, infer }) => {
      const res = await infer({
        messages: [{ role: "user", content: "Hello!" }],
      });
      console.log(`ckpt @ ${step}:`, await res.text());
    },
    onCompleted: ({ job }) => console.log(`done ${job.id}`),
    onFailed: ({ error }) => console.error(`failed: ${error}`),
  },
});
```

Once you understand callbacks, almost everything else in Arkor is just configuration on top.