Custom DataFeed

A DataFeed is the source of truth for all price bars, fundamentals, and corporate events consumed by a strategy. The SDK ships without a built-in live data adapter by design — you plug in the data source that suits your deployment, whether that is Yahoo Finance, a broker API, a CSV file, or an in-memory fixture. This page explains the contract, common implementation patterns, and how to wire a custom feed into a real backtest.

Composing multiple feeds

You don't always have one vendor for everything. A single tactical strategy might pull equity bars from Yahoo, macro time series from FRED, and (eventually) options chains from a third source. The SDK ships a reference RoutingDataFeed that dispatches each bars() call to the right inner feed based on asset.kind:

import { RoutingDataFeed } from '@livefolio/sdk';

const feed = new RoutingDataFeed({
  equity: new YfinanceDataFeed(),
  macro:  new FredDataFeed({ apiKey: process.env.FRED_API_KEY! }),
});

RoutingDataFeed itself implements DataFeed, so the rest of the runtime sees it as a regular feed. See the Composing data feeds recipe for an end-to-end example with a tactical/v1 spec.

Contract

The DataFeed interface has one required method and two optional ones.

interface DataFeed {
  bars(asset: Asset, range: DateRange, freq: Frequency): AsyncIterable<Bar>;
  fundamentals?(asset: Asset, t: Date): Promise<Fundamentals>;
  events?(range: DateRange, kinds: ReadonlyArray<EventKind>): AsyncIterable<DataEvent>;
}

bars — required

• Returns an AsyncIterable<Bar>. Use an async function* generator to stream bars one at a time; this keeps memory constant for large ranges.
• Bars must be yielded in ascending t order. The backtest engine relies on this ordering for indicator calculations.
• Respect the half-open interval: yield bars where bar.t >= range.from and bar.t < range.to.
• Omit non-trading periods. Do not emit synthetic zero-volume bars for weekends or holidays. Gaps are expected and normal.
• The freq parameter describes bar width ('1d' for daily, etc.). Implement only the frequencies your data provider supports, and throw if an unsupported frequency is requested.

fundamentals — optional

Returns a point-in-time snapshot of fundamental data (P/E ratio, sector, etc.) for an asset as of date t. Return undefined when no data is available. Omit the method entirely when your provider does not carry fundamentals — consumers feature-detect via 'fundamentals' in feed.

events — optional

Streams corporate events (earnings, dividends, splits, other actions) in ascending t order, filtered to the requested kinds. Omit when not supported.

Real-world example: YfinanceDataFeed

The sibling package @livefolio/yfinance (in yfinance/src/yfinance-data-feed.ts) is a production-grade reference. Key patterns it uses:

• Symbol translation — assetToYahooSymbol(asset) maps the SDK's asset.id format to Yahoo Finance ticker strings.
• In-process bar cache (BarCache) — A Map<symbol, Map<freq, Bar[]>> deduplicates fetches within a single backtest run. The same asset/frequency pair is only fetched once no matter how many features reference it.
• In-flight deduplication — A Map<string, Promise<Bar[]>> prevents concurrent requests for the same (symbol, freq) key from issuing duplicate HTTP calls. The second caller awaits the first caller's promise instead of starting a new request.

class YfinanceDataFeed implements DataFeed {
  private readonly cache = new BarCache();
  private readonly inflight = new Map<string, Promise<Bar[]>>();

  bars(asset: Asset, range: DateRange, freq: Frequency): AsyncIterable<Bar> {
    return this.iterate(asset, range, freq);
  }

  private async *iterate(asset: Asset, range: DateRange, freq: Frequency) {
    const symbol = assetToYahooSymbol(asset);
    const cached = this.cache.get(symbol, range, freq);
    if (cached) { for (const b of cached) yield b; return; }

    const key = `${symbol}:${freq}`;
    let pending = this.inflight.get(key);
    if (!pending) {
      pending = fetchYahooBars(symbol, range, freq, { includeIncompleteToday: false })
        .then(bars => { this.cache.set(symbol, freq, range, bars); return bars; })
        .finally(() => this.inflight.delete(key));
      this.inflight.set(key, pending);
    }
    const bars = await pending;
    for (const b of bars) {
      if (b.t >= range.from && b.t < range.to) yield b;
    }
  }
}

Sample: MockDataFeed

The sample at scripts/docs/guides-runtime/custom-datafeed.ts is self-contained and runnable:

npx tsx scripts/docs/guides-runtime/custom-datafeed.ts

// Custom DataFeed — guide sample
// Demonstrates a MockDataFeed that returns deterministic synthetic bars,
// then drives a real runBacktest call with it.
//
//   npx tsx scripts/docs/guides-runtime/custom-datafeed.ts

import {
  fromSpec,
  runBacktest,
  FeatureRuntime,
  NYSEExchangeCalendar,
  MemoryFeatureCache,
  BacktestExecutor,
} from '@livefolio/sdk';
import type { DataFeed, Asset, Bar, DateRange, Frequency, TacticalSpec } from '@livefolio/sdk';

// ─── 1. Implement DataFeed ────────────────────────────────────────────────────
//
// Contract checklist:
//   - bars() is an async generator — AsyncIterable<Bar>
//   - Bars MUST be yielded in ascending `t` order
//   - Only yield bars whose `t` satisfies: range.from <= t < range.to
//   - Omit non-trading periods (gaps are normal and expected)
//   - fundamentals / events are optional; omit them when not supported

const MS_DAY = 86_400_000;

/** Generate deterministic synthetic daily bars for a single asset. */
function generateBars(startIso: string, count: number, seed: number): Bar[] {
  const bars: Bar[] = [];
  let price = seed;
  let t = new Date(`${startIso}T00:00:00Z`);

  for (let i = 0; i < count; i++) {
    // Skip weekends — a real feed would skip exchange holidays too.
    const dow = t.getUTCDay();
    if (dow !== 0 && dow !== 6) {
      price = price * (1 + Math.sin(i / 12) * 0.008 + 0.0003);
      bars.push({
        t: new Date(t),
        open: price * 0.999,
        high: price * 1.006,
        low: price * 0.994,
        close: price,
        volume: 1_000_000 + i * 1_000,
      });
    }
    t = new Date(t.getTime() + MS_DAY);
  }
  return bars;
}

const FIXTURE_BARS: Record<string, Bar[]> = {
  'us:SPY': generateBars('2023-01-02', 600, 380),
  'us:IEF': generateBars('2023-01-02', 600, 95),
};

/**
 * MockDataFeed — fulfils the DataFeed contract using pre-built in-memory bars.
 *
 * Real adapters follow the same shape. The only difference is that `bars()`
 * would issue an HTTP request (or read a file) rather than filtering a local
 * array. The half-open range filter and ascending-order guarantee are
 * identical in both cases.
 */
class MockDataFeed implements DataFeed {
  bars(asset: Asset, range: DateRange, _freq: Frequency): AsyncIterable<Bar> {
    return this.iterate(asset, range);
  }

  private async *iterate(asset: Asset, range: DateRange): AsyncIterable<Bar> {
    const all = FIXTURE_BARS[asset.id];
    if (all === undefined) {
      throw new Error(`MockDataFeed: no fixture for asset "${asset.id}"`);
    }

    const fromMs = range.from.getTime();
    const toMs = range.to.getTime();

    for (const bar of all) {
      const tMs = bar.t.getTime();
      // Half-open interval: [range.from, range.to)
      if (tMs >= fromMs && tMs < toMs) {
        yield bar;
      }
    }
  }
}

// ─── 2. Wire into a backtest ──────────────────────────────────────────────────

const SPY = { id: 'us:SPY', symbol: 'SPY' };
const IEF = { id: 'us:IEF', symbol: 'IEF' };

const spec: TacticalSpec = {
  kind: 'tactical/v1',
  universe: [SPY, IEF],
  rebalance: { frequency: 'Monthly' },
  features: [
    { id: 'spy_price', kind: 'price', asset: SPY },
    { id: 'spy_sma50', kind: 'sma', asset: SPY, period: 50 },
  ],
  rules: {
    op: 'if',
    cond: { op: 'gt', left: { ref: 'spy_price' }, right: { ref: 'spy_sma50' } },
    then: { op: 'allocate', weights: { 'us:SPY': 1.0 } },
    else: { op: 'allocate', weights: { 'us:IEF': 1.0 } },
  },
};

const dataFeed = new MockDataFeed();
const calendar = new NYSEExchangeCalendar();
const featureCache = new MemoryFeatureCache();
const range: DateRange = { from: new Date('2023-03-01T00:00:00Z'), to: new Date('2024-06-01T00:00:00Z') };
const runtime = new FeatureRuntime({ dataFeed, featureCache, range, freq: '1d' });

const executor = new BacktestExecutor({
  calendar,
  nextOpen: async (asset, t) => {
    const bars = FIXTURE_BARS[asset.id];
    if (bars === undefined) throw new Error(`no fixture for ${asset.id}`);
    const next = bars.find((b) => b.t.getTime() > t.getTime());
    if (!next) throw new Error(`no bar after ${t.toISOString()} for ${asset.id}`);
    return { t: next.t, price: next.open };
  },
});

const strategy = fromSpec(spec, { runtime, calendar });

const result = await runBacktest({
  strategy,
  range,
  initialPortfolio: { cash: 100_000, positions: [], t: range.from },
  dataFeed,
  executor,
  calendar,
});

const final = result.snapshots.at(-1);
console.log(`sessions    : ${result.snapshots.length}`);
console.log(`rebalances  : ${result.snapshots.filter((s) => s.orders.length > 0).length}`);
console.log(`final cash  : $${final?.portfolio.cash.toFixed(2)}`);
for (const p of final?.portfolio.positions ?? []) {
  console.log(`  ${p.asset.symbol} qty=${p.quantity} basis=$${p.basis.toFixed(2)}`);
}

Things to verify

• [ ] Bars are in ascending t order. Sort the source array if your provider doesn't guarantee it.
• [ ] The half-open range filter is correct: bar.t >= range.from && bar.t < range.to.
• [ ] No synthetic bars emitted for weekends or holidays — only real sessions.
• [ ] bars() throws (or yields nothing) for asset IDs you don't support.
• [ ] Your implementation compiles: npm run docs:check.
• [ ] Integration: pass your feed to runBacktest and inspect result.snapshots to confirm expected session counts.

What's next

• Feature cache — indicator results are memoized on top of your DataFeed. See Custom FeatureCache for how caching is keyed and when to replace MemoryFeatureCache.
• Calendar — the backtest engine uses a Calendar to determine which days are sessions. Make sure your DataFeed only emits bars on days your calendar considers open. See Custom Calendar.
• API reference — DataFeed · Bar · DateRange.