All Systems Operational
Backend service for Lighter.xyz that ingests perpetuals liquidation events over WebSockets, deduplicates noisy replay batches, aggregates notional USD into per-minute buckets in Upstash Redis, and serves a rolling 24 hour liquidation total over HTTP.
Lighter Liquidations is a real-time data pipeline that tracks liquidation activity across Lighter.xyz markets and maintains a rolling 24-hour liquidation metric. The system is designed to handle noisy WebSocket streams, eliminate duplicate events, and aggregate high-frequency data into a query-efficient time series. It follows a split architecture: a persistent worker ingests and processes streaming data, while a stateless HTTP API serves pre-aggregated results. Redis acts as the aggregation layer using minute-level buckets with TTL-based retention.
The system is split into two independent processes: a long-running worker and a stateless HTTP server. The worker maintains WebSocket connections, processes incoming liquidation events, and updates Redis in real time. The API server reads aggregated data from Redis and exposes it via REST endpoints.
WebSocket ingestion is handled using Node.js with persistent connections and reconnection logic. Redis (Upstash REST) is used as a time-series store where each minute acts as a bucket. Deduplication logic ensures idempotent aggregation by filtering repeated events within batch windows. The rolling 24-hour metric is computed dynamically by summing active buckets.
Lighter streams often send repeated or batched liquidation events, which can lead to overcounting. A deduplication strategy was implemented by comparing key fields within short time windows, ensuring idempotent aggregation without dropping legitimate events.
Instead of storing a single aggregate value, liquidation data is stored in minute-level buckets with a 25-hour TTL. This allows efficient computation of a rolling 24-hour metric while keeping memory usage bounded and predictable.
Running WebSocket ingestion and API serving in the same process created stability issues. Splitting them into worker and server processes ensured better fault isolation and allowed independent scaling and restarts.
Frequent disconnects and network instability required implementing reconnection logic and ensuring no data corruption during reconnect cycles. The system prioritizes consistency over completeness, avoiding incorrect spikes in metrics.
Built for analytics platforms, trading dashboards, and developers who need accurate real-time liquidation data from Lighter.xyz without handling raw WebSocket streams or dealing with duplicate event noise.
Analytics and exploration platform for the Lighter.xyz ecosystem, covering dashboards, a block explorer, exchange statistics, multi-venue funding comparison, and protocol announcements. Redis-backed caching and a split between live and cached data paths keep the UI responsive.
Full-stack FinTech-style wallet demo: P2P transfers, balances, history, and dashboard analytics. MongoDB multi-document transactions enforce ACID semantics so debits and credits stay consistent under concurrency (simulated funds, not real money).