# Keeping data in sync

Pull every page of a snapshot, check it belongs to the right company, and replace your copy only after a complete pull.

A pull reads one register from its first page to its last. Every page of a pull comes from the same pinned snapshot of the company's records, so the pages agree with each other even while people keep completing training.

## Pull a register

1. Start a pull without a cursor, with a `limit` of up to 200. Keep the `snapshot` with the result.
2. While `next_cursor` is not null, call the same endpoint with that cursor and your page size. The cursor carries the company, deployment, dates and filters of the first call. Send it unchanged to the same endpoint. Cursors are sealed: an edited cursor, or one from another company or endpoint, is refused with [`invalid_cursor`](https://docs.kapiworks.com/errors/invalid_cursor). Start the pull again when that happens.
3. Collect every page into a temporary result, removing duplicates by `record_id`, `certificate_id`, `person_id` or a course's `row_id`. Check that `snapshot.sequence` and `snapshot.capture_sequence` stay the same on every page.
4. Replace your previous copy only after the whole pull succeeds.

A page can hold fewer rows than you asked for, to stay within 3 MiB, so only a null `next_cursor` means the pull is finished. You may change the page size between pages; changing a filter means starting a new pull.

**curl**

```bash
# Supply the key from your server's secret manager. Do not log it.
# curl reads the header from stdin, keeping the key out of its arguments.
printf 'header = "Authorization: Bearer %s"\n' "$KAPI_WORKS_API_KEY" |
  curl --config - --silent --show-error --fail-with-body \
  'https://api.kapiworks.com/v1/training-records?limit=200'
```

**Node**

```js
// Node 24, as an ES module (first-call.mjs). KAPI_WORKS_API_KEY comes from your secret manager.
const response = await fetch("https://api.kapiworks.com/v1/training-records?limit=200", {
  headers: { Authorization: `Bearer ${process.env.KAPI_WORKS_API_KEY}`, Accept: "application/json" },
  redirect: "error",
  signal: AbortSignal.timeout(30_000),
});
const requestId = response.headers.get("x-request-id");
const body = await response.json();
if (!response.ok) throw new Error(`HTTP ${response.status} ${body.code}; request ID ${requestId}`);
console.log(JSON.stringify(body, null, 2));
```

**Python**

```python
# Python 3, standard library only. KAPI_WORKS_API_KEY comes from your secret manager.
import json, os, urllib.error, urllib.request

class NoRedirect(urllib.request.HTTPRedirectHandler):
    def redirect_request(self, *args):  # do not resend the key to another address
        return None

request = urllib.request.Request(
    "https://api.kapiworks.com/v1/training-records?limit=200",
    headers={"Authorization": "Bearer " + os.environ["KAPI_WORKS_API_KEY"], "Accept": "application/json"},
)
try:
    with urllib.request.build_opener(NoRedirect).open(request, timeout=30) as response:
        print(json.dumps(json.load(response), indent=2))
except urllib.error.HTTPError as error:
    problem = json.load(error)
    raise SystemExit(f"HTTP {error.code} {problem['code']}; request ID {error.headers['X-Request-Id']}")
```

## Check the snapshot

Match `snapshot.organization_id` and `snapshot.deployment_id` to the company connection you saved before you import any rows. Each register is pinned separately, so two registers can come from different snapshots. When you need several registers from one snapshot, use a records pack from Works Studio.

`snapshot.known_at` is when Kapi's history writer captured the facts; compare it with `snapshot.as_at`. A later pull can include changes captured since. If relevant changes have waited more than five minutes for capture, a new pull answers [`writer_unavailable`](https://docs.kapiworks.com/errors/writer_unavailable) until they are captured; a pull that already has its cursor can still finish.

Names, work emails and course titles are current display values, not historical identity records.

## Schedule pulls

Run a full pull of each register on a schedule. One hourly job per company is a reasonable start; the schedule is your choice, not a delivery promise. Avoid overlapping runs, and stagger companies.

Between full pulls, [`/v1/snapshot`](https://docs.kapiworks.com/reference/get-snapshot) answers "has anything changed?" in one call, with any scope. If its `sequence` and `capture_sequence` match your last complete pull, no recorded facts have changed. Outcomes that depend on the clock can still change, for example `not_due_yet` becoming `overdue`, so recompute them from `due_at` or pull again at least daily.

The history sequence is not a change feed. There is no changed-since endpoint and no webhooks in v1.

## Retry what can be retried

Retry connection failures and statuses 429, 502, 503 and 504, at most five times, with delays such as 1, 2, 4, 8 and 16 seconds plus a little random jitter. When a response has `Retry-After`, wait at least that many seconds; it takes precedence over your own delay. Use a request timeout on every call.

Stop at once on 400, 401, 403, 404, 405 and 422: retrying the same request gives the same answer. The [errors](https://docs.kapiworks.com/errors) list what to do for each code. Retrying a `GET` is safe, because reading creates no training or completion records.

## Show what happened

Keep the time of the last complete pull and show it to your users, with any connection error. A failed pull is not the same as "no training missing": keep the previous copy and say the sync failed.