Reference

Updates and maintenance

We deploy updates during market hours without stopping the service. Your clients may see a reconnection or a pause of a few seconds; they never lose data, as long as they follow the rules on this page.

The SDKs apply every rule on this page by themselves. Read on if you write your own client, or to know what your application will observe.

What happens during an update

What is updatedReal time (WebSocket)REST
The stream serversA notice (kind: "reconnect"), then the connection is closed with code 1012 within 20 seconds. Reconnect and resume from your cursors: nothing is lost, nothing is sent twice.—
The computation serversThe connection stays open. Each subscription receives a status source_delayed, then live once the computation has caught up, usually within 30 seconds. The delayed messages then arrive, in order.—
How an indicator is computed, or its default valuesEach subscription receives a reset (engine_change), a new subscribed, then a new snapshot.A paginated query or an export started before the change cannot be continued: 410 CURSOR_EXPIRED. Start it again.
The API servers—Requests in progress complete. Nothing to do.

A computation change is announced in advance with a maintenance notice. Among updates, it is the only one that requires rebuilding state: every other update is a pause followed by the missed messages.

Default values are set per ticker and regularly updated. An update of the defaults is an engine update: the engine of subscribed changes, every subscription receives a reset with engine_change, and a history query that omits those parameters uses the new defaults from then on, over past ranges too. A subscription that passes all its parameters explicitly is reset too, but its results do not change: to keep exactly the same computation over time, pass the parameters explicitly.

Maintenance notices

When we plan maintenance with a visible effect (a pause longer than usual, a computation change), every connection receives a notice, as soon as it is announced:

JSON
{"t":"notice","kind":"maintenance","id":"6f1c2a7e-0b5d-4c1e-9a3f-2d8e4b7c1a90","effectiveAt":"1791295200000000000","endsAt":"1791295500000000000","message":"Computation update: each subscription restarts from a new snapshot."}
  • effectiveAt and endsAt: the window during which the service may pause, in nanoseconds since January 1, 1970 UTC, as strings. Show the delay to your users (effectiveAt minus now), or schedule your own jobs around it;
  • every new connection receives the notices of the windows not over yet, right after it opens: a client that connects after the announcement is warned too;
  • a window can be moved: you then receive a notice with the same id and new times. A cancellation is announced with {"t":"notice","kind":"maintenance-cancelled","id":"6f1c2a7e-0b5d-4c1e-9a3f-2d8e4b7c1a90"};
  • the windows announced and not over are also listed by GET /v1/status, in maintenance, ordered by start time.

In GET /v1/status, each window has these fields. The start of the window is called startsAt there: it is the same instant as the effectiveAt of the notice.

FieldContent
idThe id of the window, the same as in the notices.
startsAtStart of the window, in nanoseconds since January 1, 1970 UTC, as a string.
endsAtEnd of the window, in the same format.
messageThe text of the announcement.
JSON
{
  "maintenance": [
    { "id": "6f1c2a7e-0b5d-4c1e-9a3f-2d8e4b7c1a90", "startsAt": "1791295200000000000", "endsAt": "1791295500000000000", "message": "Computation update: each subscription restarts from a new snapshot." }
  ]
}

A notice is information: you have nothing to send back, and your subscriptions continue until the maintenance actually happens.

Rules for your WebSocket client

  1. Keep, for each subscription, the last cursor and the engine of its latest subscribed. The cursor tells the server where you stopped; engine tells it which computation produced your state.

  2. Reconnect on 1012, on any close not listed as final on the WebSocket page, and after 15 seconds without any message, with a delay that doubles after each failure (up to 30 seconds) plus random jitter.

  3. Resume each subscription with both values:

    JSON
    {"t":"subscribe","sub":"bt","instrument":"NQ.front","indicator":"big-trades","params":{"minimum":30},"mode":"confirmed","from":{"cursor":"20732.44294.1","engine":"82cbdecf93748dd5"}}

    If the computation changed while you were away, you receive subscribed, a reset (engine_change) and a snapshot, instead of messages computed differently from your state. Without engine, the server cannot tell, and you could keep objects that the new computation no longer produces.

  4. On a reset, discard the state of the subscription and start again from the snapshot that follows.

  5. Treat source_delayed as a pause, not an error. Keep the connection: the stream resumes by itself and sends what was late.

  6. Ignore any upsert or remove whose cursor is not after the last one you processed: after a reconnection, a message can arrive twice.

Rules for your REST client

  • Retry a 503 that carries a Retry-After header after the delay it gives: the request was not processed.
  • Retry a GET that failed without a response (network error, connection closed) or with a 502, after a short delay that grows with each attempt: a read has no effect other than its response. A query page or an export that failed without a response may have been computed and charged. Retrying it is harmless for your data, but the retry may be charged again.
  • Paginated queries: a 410 CURSOR_EXPIRED on a next page means the computation changed between two pages. Start the query again from its first page, and drop the pages already read.
  • Exports: after an interruption, resume with the X-Export-Token of the first response (see Exports). A 410 CURSOR_EXPIRED on the resume means the computation changed since the export started: start the export again, and drop the lines already received.

With the SDK

The SDK reconnects, resumes with the cursor and the engine of each subscription, drops duplicates, retries the REST requests the rules above allow, and resumes interrupted exports. Your code only sees events:

src/maintenance.ts
import { FathomChartsStream } from '@fathom-charts/sdk';

const stream = new FathomChartsStream({
  url: 'wss://stream.fathomcharts.com/v1',
  apiKey: process.env.FATHOM_CHARTS_API_KEY!,
  onNotice: (notice) => {
    if (notice.kind === 'maintenance') {
      const inMinutes = Math.round((Number(BigInt(notice.effectiveAt!) / 1_000_000n) - Date.now()) / 60_000);
      console.log(`Maintenance in ${inMinutes} min: ${notice.message}`);
    }
  },
});

for await (const event of stream.subscribe({ instrument: 'NQ.front', indicator: 'big-trades', params: { minimum: 30 } })) {
  switch (event.type) {
    case 'reset': /* discard the local state: a snapshot follows */ break;
    case 'snapshot': /* the new starting state */ break;
    case 'status': console.log('status', event.state); break; // source_delayed: a pause, then live
    case 'notice': /* the same notices, in each subscription */ break;
  }
}
maintenance.py
import asyncio
import os
import time

from fathom_charts import FathomChartsStream, Notice, Reset, Snapshot, Status


def on_notice(notice: Notice) -> None:
    if notice.kind == "maintenance" and notice.effective_at is not None:
        in_minutes = round((int(notice.effective_at) / 1e9 - time.time()) / 60)
        print(f"Maintenance in {in_minutes} min: {notice.message}")


async def main() -> None:
    async with FathomChartsStream(
        url="wss://stream.fathomcharts.com/v1",
        api_key=os.environ["FATHOM_CHARTS_API_KEY"],
        on_notice=on_notice,
    ) as stream:
        async with stream.subscribe(instrument="NQ.front", indicator="big-trades", params={"minimum": 30}) as sub:
            async for event in sub:
                match event:
                    case Reset():
                        pass  # discard the local state: a snapshot follows
                    case Snapshot():
                        pass  # the new starting state
                    case Status():
                        print("status", event.state)  # source_delayed: a pause, then live
                    case Notice():
                        pass  # the same notices, in each subscription
                    case _:
                        pass


asyncio.run(main())
Program.cs
using FathomCharts;

await using var stream = new FathomChartsStream(new StreamOptions
{
    Url = "wss://stream.fathomcharts.com/v1",
    ApiKey = Environment.GetEnvironmentVariable("FATHOM_CHARTS_API_KEY"),
    OnNotice = notice =>
    {
        if (notice.Kind == NoticeKind.Maintenance && notice.EffectiveAt is { } at)
        {
            var inMinutes = Math.Round((long.Parse(at) / 1_000_000 - DateTimeOffset.UtcNow.ToUnixTimeMilliseconds()) / 60_000.0);
            Console.WriteLine($"Maintenance in {inMinutes} min: {notice.Message}");
        }
    },
});

await foreach (var e in stream.Subscribe(new SubscribeOptions { Instrument = "NQ.front", Indicator = "big-trades", Params = new { minimum = 30 } }))
{
    switch (e)
    {
        case Reset: /* discard the local state: a snapshot follows */ break;
        case Snapshot<JsonValue>: /* the new starting state */ break;
        case Status status: Console.WriteLine($"status {status.State}"); break; // SourceDelayed: a pause, then Live
        case Notice: /* the same notices, in each subscription */ break;
    }
}
src/main.rs
use std::time::{SystemTime, UNIX_EPOCH};

use fathom_charts::{FathomChartsStream, NoticeKind, StreamEvent, SubscribeOptions};
use futures::StreamExt;
use serde_json::json;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let stream = FathomChartsStream::builder("wss://stream.fathomcharts.com/v1")
        .api_key(std::env::var("FATHOM_CHARTS_API_KEY").map_err(|_| "FATHOM_CHARTS_API_KEY is not set")?)
        .on_notice(|notice| {
            // Timestamps are nanoseconds: parse them as integers, never as f64 (above 2^53).
            if let (NoticeKind::Maintenance, Some(Ok(at))) = (&notice.kind, notice.effective_at.as_deref().map(str::parse::<u64>)) {
                let now = SystemTime::now().duration_since(UNIX_EPOCH).unwrap_or_default().as_secs();
                let in_minutes = ((at / 1_000_000_000) as i64 - now as i64) / 60;
                println!("Maintenance in {in_minutes} min: {}", notice.message.as_deref().unwrap_or(""));
            }
        })
        .build()?;

    let mut sub = stream.subscribe::<serde_json::Value>(
        SubscribeOptions::new("NQ.front", "big-trades").params(json!({ "minimum": 30 })),
    )?;
    while let Some(event) = sub.next().await {
        match event? {
            StreamEvent::Reset(_) => { /* discard the local state: a snapshot follows */ }
            StreamEvent::Snapshot(_) => { /* the new starting state */ }
            StreamEvent::Status(status) => println!("status {}", status.state), // source_delayed: a pause, then live
            StreamEvent::Notice(_) => { /* the same notices, in each subscription */ }
            _ => {}
        }
    }
    Ok(())
}
  • stream.maintenancestream.maintenancestream.Maintenancestream.maintenance() lists the windows announced and not over;
  • to resume after your own process restarts, save the cursor and the engine of the subscription, then subscribe again from them (see Resume after a restart);
  • rest.exportNdjson()rest.export_ndjson()rest.ExportNdjsonAsync()rest.export_ndjson() resumes an interrupted export by itself; it throws CURSOR_EXPIRED when the computation changed since the export started.