# Updates and maintenance

We deploy updates during market hours without stopping the service. Your clients may see a reconnection or a pause of a few seconds; they never lose data, as long as they follow the rules on this page.

> The [TypeScript SDK](https://fathomcharts.com/docs/sdk.md) applies every rule of this page by itself. Read on if you write your own client, or to know what your application will observe.

## What happens during an update

| What is updated              | Real time (WebSocket)                                                                                                                                                                              | REST                                                                                                                  |
| ---------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- |
| The stream servers           | A `notice` (`kind: "reconnect"`), then the connection is closed with code `1012` within 20 seconds. Reconnect and resume from your cursors: nothing is lost, nothing is sent twice.                | —                                                                                                                     |
| The computation servers      | The connection stays open. Each subscription receives a `status` `source_delayed`, then `live` once the computation has caught up, usually within 30 seconds. Late messages arrive then, in order. | —                                                                                                                     |
| How an indicator is computed | Each subscription receives a `reset` (`engine_change`), a new `subscribed`, then a new snapshot.                                                                                                   | A paginated query or an export started before the change cannot be continued: `410` `CURSOR_EXPIRED`. Start it again. |
| The API servers              | —                                                                                                                                                                                                  | Requests in progress complete. Nothing to do.                                                                         |

A computation change is rare and announced in advance with a [maintenance notice](#maintenance-notices). It is the only case where you must rebuild a state: every other update is a pause followed by the missed messages.

## Maintenance notices

When we plan a maintenance with a visible effect (a pause longer than usual, a computation change), every connection receives a notice, as soon as it is announced:

```json
{"t":"notice","kind":"maintenance","id":"6f1c…","effectiveAt":"1791295200000000000","endsAt":"1791295500000000000","message":"Computation update: each subscription restarts from a new snapshot."}
```

- `effectiveAt` and `endsAt`: the window during which the service may pause, in nanoseconds since January 1, 1970 UTC, as strings. Show the delay to your users (`effectiveAt` minus now), or schedule your own jobs around it;
- every new connection receives the notices of the windows not over yet, right after it opens: a client that connects after the announcement is warned too;
- a window can be moved: you then receive a notice with the same `id` and new times. A cancellation is announced with `{"t":"notice","kind":"maintenance-cancelled","id":"6f1c…"}`;
- the windows announced and not over are also listed by `GET /v1/status`, in `maintenance`.

A notice is information: you have nothing to send back, and your subscriptions continue until the maintenance actually happens.

## Rules for your WebSocket client

1. **Keep, for each subscription, the last cursor and the `engine` of its latest `subscribed`.** The cursor tells the server where you stopped; `engine` tells it which computation produced your state.
2. **Reconnect** on `1012`, on any close not listed as final on the [WebSocket page](https://fathomcharts.com/docs/websocket.md#closing-and-reconnecting), and after 15 seconds without any message, with a delay that doubles after each failure (up to 30 seconds) plus random jitter.
3. **Resume each subscription** with both values:

```json
{"t":"subscribe","sub":"bt","instrument":"NQ.front","indicator":"big-trades","params":{"minimum":30},"mode":"confirmed","from":{"cursor":"20732.44294.1","engine":"3ee45f60a463ab09"}}
```

If the computation changed while you were away, you receive `subscribed`, a `reset` (`engine_change`) and a snapshot, instead of messages computed differently from your state. Without `engine`, the server cannot tell, and you could keep objects that the new computation no longer produces.
4\. **On a `reset`, discard the state of the subscription** and start again from the snapshot that follows.
5\. **Treat `source_delayed` as a pause, not an error.** Keep the connection: the stream resumes by itself and sends what was late.
6\. **Ignore any `upsert` or `remove` whose cursor is not after the last one you processed**: after a reconnection, a message can arrive twice.

## Rules for your REST client

- **Retry a `503` that carries a `Retry-After` header** after the delay it gives: the request was not processed.
- **Retry a `GET` that failed without an answer** (network error, connection closed) or with a `502`, after a short delay that grows with each attempt: a read has no effect other than its answer. A query page or an export that failed without an answer may have been computed and charged: retrying it is safe, but may be charged again.
- **Paginated queries**: a `410` `CURSOR_EXPIRED` on a next page means the computation changed between two pages. Start the query again from its first page, and drop the pages already read.
- **Exports**: after a cut, resume with the `X-Export-Token` of the first response (see [Exports](https://fathomcharts.com/docs/exports.md#resume-after-a-disconnect)). A `410` `CURSOR_EXPIRED` on the resume means the computation changed since the export started: start the export again, and drop the lines already received.

## With the SDK

The SDK reconnects, resumes with the cursor and the `engine` of each subscription, drops duplicates, retries the REST requests the rules above allow, and resumes cut exports. Your code only sees events:

```ts
import { FathomChartsStream } from '@fathom-charts/sdk';

const stream = new FathomChartsStream({
  url: 'wss://stream.fathomcharts.com/v1',
  apiKey: process.env.FATHOM_CHARTS_API_KEY!,
  onNotice: (notice) => {
    if (notice.kind === 'maintenance') {
      const inMinutes = Math.round((Number(BigInt(notice.effectiveAt!) / 1_000_000n) - Date.now()) / 60_000);
      console.log(`Maintenance in ${inMinutes} min: ${notice.message}`);
    }
  },
});

for await (const event of stream.subscribe({ instrument: 'NQ.front', indicator: 'big-trades', params: { minimum: 30 } })) {
  switch (event.type) {
    case 'reset': /* discard the local state: a snapshot follows */ break;
    case 'snapshot': /* the new starting state */ break;
    case 'status': /* source_delayed: a pause, then live */ break;
    case 'notice': /* the same notices, in each subscription */ break;
  }
}
```

- `stream.maintenance` lists the windows announced and not over;
- to resume after your own process restarts, save `sub.cursor` **and** `sub.engine`, then subscribe again with `from: { cursor, engine }` (see [SDK](https://fathomcharts.com/docs/sdk.md#resume-after-a-restart));
- `exportNdjson()` resumes a cut export by itself; it throws `CURSOR_EXPIRED` when the computation changed since the export started.