Real-time client-side trading data

Problem

Quantreex is an automated stock-trading platform. Its web UI had to let a trader search and sort more than 10,000 live position records in the browser while new data kept arriving, without the interface stalling under the recomputation.

Constraints

• A 2016 stack: BackboneJS and RequireJS, no virtual DOM, and a main-thread budget shared with rendering and input.
• Position data updated continuously, so filtering and sorting could not be a one-off computation.
• The production bundle had to ship obfuscated through a RequireJS and Grunt build, which made production diagnosis harder.

My responsibility

Owned the UI: the data-processing architecture, the Backbone application structure, third-party component integration, and the production build.

What moved off the main thread, and what stayed

• Into a Web Worker: parsing the incoming feed, maintaining the in-memory index of positions, and running filter, search and sort over the full set. The worker returns only the slice the current view needs.
• On the UI thread: rendering that slice, handling input, and posting the user’s filter and sort state to the worker. Rendering was kept to the visible window so a full recomputation never meant a full re-render.
• Cost accepted: each update crosses a message boundary with serialised data, so results arrive a beat later than a synchronous call would return them. That delay was preferable to blocking input during recomputation.

How responsiveness was judged

By hand and in the browser’s own tools, not against a benchmark. The test that mattered was typing in the search box while the feed was still streaming: before the change the field stopped taking keystrokes during a recomputation, and afterwards it kept accepting them. A profiler trace showed why, with the long tasks that had blocked the main thread no longer on it, and frame timing held steady through a recomputation instead of dropping out. No measurement from that work survives, so there is no figure to quote.

Trade-offs

• Web Worker vs. main-thread optimisation. Message-passing overhead and serialisation in exchange for input that stays responsive during recomputation. At 10,000+ records the trade was clear.
• Obfuscated production builds vs. debuggability. Met the platform’s source-protection requirement at the cost of harder production diagnosis.

Results

• Search and sort over 10,000+ position records in the browser with the UI staying responsive while data streamed in.

Stack

JavaScript · BackboneJS · RequireJS · Web Workers · Bootstrap · LESS · Grunt · Nightwatch

What I’d change today

I did push for the server side at the time, and the backend could not answer that kind of query: the position feed was passed through, not held anywhere a search could run against it. Today I would build that missing middle first, a queryable store fed by the stream - Elasticsearch then, a columnar store now - so the client asks for one window at a time and gets back a hundred rows rather than ten thousand. Ten thousand records was never the hard part; having nowhere to ask the question was, and the Web Worker was the right answer to that.