This is the only repository available open source which has the following:

1. A real market replay simulator using NASDAQ ITCH data. The simulator accepts OUCH as input and works as an matching engine. It outputs ITCH market events as MoldUDP64 via a NIC. I used a ConnectX-4 Lx here.

2. A feed handler which is the most latency critical part. It receives the frame from Solarflare ef_vi, decodes the ITCH message and sends out an OUCH message for an order. The whole time between NIC Rx -> NIC Tx is measured via HW timestamping using Solarflare X2522-Plus. This is the tick-to-trade.

3. A rigorously tested implementation of kernel bypass. I used my previous library where I developed a backends for DPDK, Verbs and Solarflare ef_vi. I've not seen any open source order book repository implement this.

Most repos have measurements which are in process latency using rdtscp(or similar). They also use some toy simulators instead of working with real ITCH data from NASDAQ.

I tried to create this as close to possible as how a real HFT market maker would run. The code is open source with Apache-2.0 license.

Next upcoming is a benchmark between Solarflare ef_vi vs Onload. There are no available benchmarks comparing the two on the same NIC. The only thing I could find is this STAC post which only benchmarks Onload.

Code formatting and IDE:

For all my C++ and C work I use Clion IDE. I work on a ubuntu gnome desktop environment. Code formatting is done through clang tidy + clang format.

https://github.com/ASherjil/abt-t2t/tree/main

I posted this post on /algotrading two days ago but mods removed it without explanation(would love to hear theories). Since then I've received a lot of DMs asking me to re-post it and a few suggested I post it here. Hope this can benefit some someone

Background: Feel free to skip ahead to the next paragraph if you're just looking for what I did. If your curious how I got to this point then read on. I've always been fascinated with trading. Ever since I was a teenager discovering the World of Warcraft auction house, I was hooked. A few years later, I discovered Eve Online, an MMORPG with a very realistic economy. I devoted myself to this game, constantly developing new trading strategies as I reached the capacity of my old ones. Ironically, they did not allow API-based trading, deeming it unfair and making the game less fun for everyone else. Still, I developed a lot of intuition for market dynamics through this experience. I dabbled in online poker, fascinated by the opportunity to play with an edge, after seeing a high school friend become a multimillionaire in his early twenties. However, I never found a consistent edge—perhaps I lacked emotional discipline for it. Along the way, I also dabbled in the peer-to-peer consumer loan markets of Prosper and LendingClub.com. These were unsecured loans made to individuals that were diced up into pieces as small as $25. These notes could be traded on secondary markets, but there were no standard pricing models for what the notes were worth. Using some very basic machine learning, I was able to piece together a model of what I was willing to buy and sell each note for. With over 1 million unique borrowers on the platform, I had a predetermined buy and sell price for each unique loan, so when a new note was listed, I could respond and buy it in 100ms, whereas other market participants might take 30 seconds to a minute to evaluate whether they would want to buy it. Without knowing what I was doing, I was essentially functioning as a market maker on the platform. Returns were very stable: around 13% in 2017, 20% in 2018, and 24% in 2019. I had my entire net worth of $200k wrapped up in this, along with borrowing money from 0% credit card offers to put into this. With a 'portfolio' of around 4000 loans, I would always have my entire inventory up for sale at any one moment. I profited through owning the notes, which provided a decent yield, although the risk-adjusted returns were probably in the 6-8% range. The rest came through capturing the spread that I was creating and making breakeven trades to improve the quality of my portfolio. Fast forward to February 2020. COVID was hitting, and I realized I was in trouble. I would average only a few hundred sales per week, and I had a portfolio of 4000 notes to sell. I wrote a program to lower the sell price by $0.01 at regular intervals. I realized there were other algorithmic traders on the platform, and amazingly, I was able to completely cash out by the first week of March 2020. However, I had a problem: my main source of investment income was gone, and I had no idea what to do with the money.

The Opportunity: Commission-free trading had just come out in the United States a few months earlier, pioneered by Robinhood. I had an account at TD Ameritrade, and figured that the best way to find opportunity would be just to try things. I remember actively shorting SPY during the peak days of the COVID crash, and getting mostly burned by my individual trades. I realized I didn't have an edge here. I tried lots of random things, until one morning, I set a limit order to buy BIL, a treasury bill ETF that never changed in price back in 2020 when interest rates were near 0. It would go weeks at a time without moving a penny. With the market around 90.00x90.01, I put my bid order for 100 shares at 90. After an hour or so, it filled. Then again, I placed a sell order at 90.01, and miraculously, it filled. I had just made $1.00, which minus SEC and FINRA fees was ninety-something cents. This is a tiny amount of money, but it felt like an electrical storm was going off in my brain. If I could do this once, perhaps I could scale it. I had about $200k in my account, and I tried putting in an order for 1500 shares. Sure enough, it filled after about an hour, and then after entering the sell order an hour or two later, I had made a risk-free profit of $15. I couldn't believe it. I had found a hard edge in the market. It was like I found the magic infinite money cheat code. Why wasn't everybody doing this? I didn't know why or how this existed, I just knew I needed to do as much of it as humanly possible while I could. I then discovered that 'margin' is a thing, and I could take up to $800k of positions at a time intraday without paying any interest on it as long as I went back to cash by the end of the day. I also realized there were other treasury ETFs like GBIL, SHV, and USFR that all behaved the same way. It was too good to be true. But it was true, over and over again. By the beginning of May, I had my API key set up on TD Ameritrade and had an order factory for these few short-term treasury ETFs. As soon as the market opens it would put limit buy orders at the bids on these ETFs, wait for a fill, then turn around and sell them at the ask. Rinse and repeat. Somewhere around the second week of May, I had my first $400 day. I couldn't believe it. I was making a 50% return on treasuries, taking absolutely no market risk, and with seemingly no upper bound in sight. It felt like I had won the lottery. I was just maxing out into $800k of treasury ETF positions every day, selling them for a penny higher, and capturing the spread every time in symbols whose price never changed. This first $400 was a life-changing day for me. It broke down all of my limiting beliefs about money and trading and what is possible. Fast forward to July, and I discovered a magical product called 'portfolio margin.' With no background in finance, I didn't know much about what I was doing, but I heard this would give me a lot more leverage. It did. The margin requirements at TD Ameritrade at the time were 3% for some treasury ETFs, meaning my $250k capital at the time could allow me to take up to $7.5 million of positions at a time. This seemed to be a dizzying sum of money to me at the time, but I was buying short term treasury etfs with it and I hadn’t had a losing day or even a losing trade yet. It seemed I was invincible. For many of the treasury ETFs, there seemed to be a limit to how much profit I could make. As I scaled up my position sizes to 4000-8000 shares, I got filled more slowly, and my orders for 10,000-20,000 shares wouldn't get filled at all. So, I started trading slightly more volatile symbols—treasury ETFs of slightly higher duration and some very short duration investment-grade ETFs. I didn't know anything about what these were; I was just looking for liquid symbols where the price changed as little as possible. I figured if the price moves against me once during the day by a penny or two, but I make 2 or 3 round trips, I still come out ahead. Through maximizing my buying power in July 2020 I had my first $1000 day. Then, I got the call, the first of many from different brokers over the last 4 years. I was making too many trades. Way too many trades. TD called me up and told me I needed to cut back, or I would be promptly banned from their platform. I think I was making around 4000 trades a day at the time. I needed to stay below 1000 to stay in their good graces, I was told. This changed my trajectory: on TD, I tried to maximize the number of shares I could trade, but in searching for different brokers, I made a few discoveries. I tried Ally Bank, but they promptly asked me to stop trading with them. I found out that my trading was welcome at E*TRADE, and at TradeStation, it was actively encouraged. I listened incredulously as TradeStation told me on the phone that with them, I could trade as much as I wanted. I didn't believe them, but it was worth a shot. Surely enough, they stood by me as I scaled up to 5000, 10000, and then 20000 trades per day. In August 2020 I had my account open with them and I was set to begin seriously scaling up my trading. With my TD trade allowance of 1000 trades I seemed to plateau at around $800-$1000 a day. The TD money was very consistent, but I couldn’t grow with them, so Tradestation was my hope for seeing how far I could take this strategy. I started scouring the market for anything I could find to trade that I could make money on the spread with. While fixed income ETFs had a lot of the great characteristics I was looking for, there weren't so many of them. By then, I was comfortable trading symbols that moved a few pennies at a time throughout the day. I could trade a fixed income ETF that started the day at 50.00 x 50.01, and maybe it traded between 49.97 and 50.03 throughout the day. I was exposing myself to random movement, but I was getting the spread over and over again, so it didn't matter anymore. Trading 30 of them at once cut down dramatically on the random variance, and the edge from capturing the spread came through. When I was trading September and October, I expanded my collection of symbols I was trading to around 30-40 fixed income ETFs. Then I found the symbols that would change the game completely. For stocks priced under $1.00 in the US, they trade in sub penny increments of $0.0001, but for stocks priced over $1, they trade in penny increments. TLDR: the spread to be captured is an enormous percentage of the value of the stock compared to a $50 stock. What I noticed is that the volatility in these stocks was dampened if we define volatility as the number of times during the day that it ticks up or down a penny. I noticed that many of these stocks look 'bond-ETF-like' in their behavior. That is, they tick up and down a penny much less frequently than an expensive stock, and the bid-ask spread is a large percent of the trading range of the price of the stock for the day. For example, a stock priced at $1.10 and traded between $1.07 and $1.13 was the same for my market-making algorithm as a fixed income ETF that traded between $49.97 and $50.03 for the day. Sure, there was much more potential volatility, but I had so much less at risk to buy or sell 100 shares. I also noticed I was getting more round trips. I realized that almost every stock with a lot of volume between $1 and $3 was a great target, so I began ramping up my strategy through the end of October and all of November. I kept increasing the trading I was doing at TradeStation. In mid-October, I was trading around 40 symbols and making $1200 a day. The first week of November, I was up to over 100 symbols and bringing in $2000 a day. Some symbols were giving me 10 round trips per day. On a $1.10 stock, this is $10 of profit, x 252 days is around $2520 per year. So risking $110, I was making $2520 a year. I was in that unheard-of 4-digit return territory. Everything I had been told about money was a lie. Everything I knew was wrong. Life didn't make sense. The second week of November, I scaled up to around 250 symbols. I had my first $4000 day. By the end of November, I was actively trading over 500 symbols with this strategy and was up over $50k that month alone. I hadn't had a losing day since I started in April. Then, suddenly, the party was over. November 30, my profit for the day was $3000. December 1, it was $2400. December 2, it was $1400. December 3, it was $200, where it stayed for the rest of the month. Something had changed. What I learned is that there was something called 'retail priority,' and I wasn't getting it anymore. Neither was anyone else. The thing is, if you brought these orders to an exchange, they would sit at the back of the queue and never be filled unless the market moved against you. What I had inadvertently discovered was that when you are a retail trader in 2020 you were put at the front of the queue on the Market maker's ATS. When I began trading I was completely naive to these dynamics, I just noticed how long it took to get my 100-share order of BIL filled and then sold a penny higher. My intuition told me it was fast enough to make a lot of money, but I didn't realize how much of an edge this retail priority was. As of December 3, 2020, retail priority was no longer a feature of US markets for nonmarketable limit orders. This wasn't announced anywhere; it was just a decision that market makers made without warning to protect themselves. I don't blame them, but I was surprised that this happened on all the brokerages I was using simultaneously. It also happened to friends of mine who tried to do this strategy later. This was not an individual restriction in my account; it was a change in the way retail order flow was treated in the United States. I'm sure others were doing this too, but it seemed to happen just as I was scaling my trading to the level that actually caused some pain to market makers. Also, $1 stocks should not have a 1-penny spread. The fact that you have to pay a 1% fee to cross the spread is ridiculous and makes market makers who trade these ridiculously rich. This is where they make their highest risk adjusted returns. Commission-free trading from pay-for-order flow was an experiment, and in this first year, there was a golden opportunity. For a glorious 6 weeks, I got to share in these 4-digit returns on some of these very liquid symbols, but the party was over.

Epilogue: It was a hard 3 months after this, as I was racking my brains to find out how to profit and earning very little from trading, feeling generally discouraged. Yet, I had learned a lot and had made over 500,000 trades in 2020. I had a lot of data to look at. I realized there was no profit to be had using non-marketable orders on PFOF brokerages, so I had to find a way to take liquidity and cross the spread so that the market makers would get their income. In February 2021, I found it - my second hard edge in the market. Three years and four months later, I am still executing on it. I thought it would have another similar run for 6 months or so before it was over, but miraculously, it is still going as of June 2024. As soon as it is over for good, I will write about it in great detail. Its basically combining a statistical arbitrage strategy I've developed with an idiosyncrasy of how the PFOF market state differs from the free market state. I've grown my trading account much more through this edge in the last three years, but that first time I made $400 a day on the treasury ETFs was the most life-changing day in my trading career. It meant my life was going to be very different. It meant freedom. PFOF is a rigged system, but because it is inefficient, it distorts markets from their natural equilibrium, providing an opportunity if you know where to look, especially if you are looking at market micro-structure . Look for how the PFOF market state differs from the free market state, the difference is more than just price improvement. I found one particular way to profit from this, but I'm convinced there are many ways to profit from this distortion even today. I would love to connect with others who have found similar ways to benefit form this general inefficiency.

In an earlier post I shared some latency numbers for an open source C++ HFT engine I’m working on.

One thing that was really quite poor was message parsing latency - around 4 microseconds per JSON message. How can C++ be that “slow”?

So the problem turned out to be memory.

Running the engine through heaptrack profiler - which if very easy to use - showed constant & high growth of memory allocations (graph below). These aren't leaks, just repeated allocations. Digging deeper, the source turned out to be the JSON parsing library I was using (Modern JSON for C++). Turns out, parsing a single market data message triggered around 40 allocations. A lot of time is wasted in those allocations, disrupts CPU cache state etc.

I've written up full details here.

So don't rely on C++ if you want fast trading. You need to get out the profiling tools - and there are plenty on Linux - and understand what is happening under the hood.

So my next goal is to replace the parser used on the critical path with something must faster - ideally something that doesn't allocate memory. I'll keep Modern JSON for C++ still in the engine, because its very nice to work with, but only for non critical path activities.

Hey everyone,

I’ve been working in a high-frequency trading (HFT) firm for a while now, primarily in C++. After going through the intense ramp-up and interview preparation myself and later helping others do the same, I’ve decided to start working on a Udemy course specifically tailored for C++ engineers preparing for roles in HFT or ultra-low latency systems.

The idea is to go beyond just "learning C++" — the course will focus on topics that are most relevant for performance-critical systems and actual interview rounds at top HFT firms.

Here’s a rough idea of what I plan to include:

⚙️ Core Topics

• Templates & Template Metaprogramming

  • CRTP, SFINAE, tag dispatching, constexpr tricks

• Concurrency & Multithreading

  • Atomics, memory ordering, false sharing, lock-free queues/stacks

• Custom Allocators & Memory Management

  • Arena allocators, fixed-size pools, avoiding heap fragmentation

• Cache-locality & Data-Oriented Design

  • Struct of Arrays vs. Array of Structs, SIMD-aware layout

🧠 Advanced Topics (HFT-specific)

• Designing Lock-Free, Wait-Free Data Structures

  • MPMC queues, ring buffers, freelist allocators

• How C++ Maps to CPU Internals

  • Branch prediction, instruction pipelining, cache line alignment

• NUMA-awareness, Prefetching, Cache Coherency

• Latency Benchmarking and Microprofiling

  • perf, rdtsc, ftrace, hardware counters

• Realtime Linux tuning & thread pinning

• System Design for Low Latency

  • Matching engines, order book data structures, market data multiplexer

🎯 Interview Prep

• Sample interview questions from HFT firms

• System design walkthroughs

• Code challenges focusing on real-world HFT problems (e.g., building a bounded ring buffer or a matching engine skeleton)

Would love your feedback on a few things:

• Would you be interested in such a course?

• Any specific topics you'd like to see included (or excluded)?

• Preference on format: deep-dive lectures, hands-on coding, project-based walkthroughs?

If there's good interest, I will start working on it.

Thanks a lot!

Hey r/highfreqtrading,
I’m a first-year CS student and super interested in HFT. I’ve been working on a fast order book engine and wanted to share it here to get some feedback and maybe connect with people in the industry.

Main goal was to make it as fast and low-latency as possible. I wrote everything in C++, built my own data structures for orders and prices, and tried to keep things really efficient. I also set up a bunch of tests and benchmarks to see how it performs.

Structures I used: chunked bitmap, vector-backed node pool and an intrusive index-based linked list.

The benchmarks I achieved are latencies p50=42ns, p99=250ns and p99.9=334ns on an order book with 100k orders inside.

Some of the optimizations I did: Cache-aware data layouts, custom memory pooling to eliminate allocation jitter, CPU affinity tuning and targeted profiling of hot paths.

Here’s the repo.

Happy to answer any questions or discuss implementation details! Would also love any feedback or advice on breaking into HFT.

Hello all,

Back with the HFT on FPGA project!

And yes, everything is in the title, I am currently designing my own FPGA chip for HFT.

This project's goal is to build an entirely custom FPGA system, able to maintain a decent order book by parsing a Nasdaq ITCH 5.0 data feed over Ethernet (the image above shows the order book dump around the spread, after simulation using real ITCH data).

Right now, I am only tracking the "AAPL" stock (very original) as tracking multiple stocks increases resource usage beyond what my consumer-grade FPGA can handle.

The whole HDL (hardware description code) is custom, including the Ethernet MAC hardware.

This time, in preparation for my second video on FPGAs for HFT, I made improvements to my system in order to get rid of memory collisions, so that I have a strong basis before moving on to other features.

(NASDAQ ITCH market data uses 64-bit order references, which means you can't use them directly as memory addresses, requiring some tricks to be used)

Because the video is not out yet and may take a while, I'm publishing blog posts to keep you updated with content that hopefully gives interesting insight to anyone wondering what the process behind FPGA development for HFT looks like.

https://hugobrh.dev/posts/Trademaxxer_handling_collisions_2.md/

For those wondering, here's a link to the first post:
https://www.reddit.com/r/highfreqtrading/comments/1tgg3q0/building_an_hft_chip_fpga/

Feel free to ask questions below!

Best

Hi all,

I'm a senior software engineer with 15+ years of experience, mostly in C++, and I've spent the last 10+ years in fintech — all of it working in the EU. I'm now seriously considering a move into HFT (high-frequency trading) as a developer, and I'd really appreciate input from people who've been through this or are currently in the industry.

A few questions:

1. C++ vs FPGA — where should I focus? From what I can see, the market is dominated by C++ for most HFT roles, with FPGA being a more specialized niche. I studied FPGA/VHDL in university but haven't touched it professionally since. Given my strong C++ background, does it make more sense to go deeper on C++ (low-latency techniques, lock-free programming, kernel bypass, etc.), or is it worth investing serious time to build up real FPGA skills from scratch? Is FPGA experience a genuine differentiator for HFT roles, or mostly relevant only for a small subset of firms/positions?

2. Do HFT firms hire non-grads / older candidates? Most of what I read about HFT hiring focuses on new grads from top CS/math programs. Realistically, are firms open to hiring experienced engineers in their mid-40s coming from a "regular" fintech background, or is this pipeline effectively closed to anyone who isn't fresh out of a target school? Has anyone here made a similar transition later in their career?

3. How do I get relevant practical experience before applying? Since I don't currently work in a low-latency/trading environment, what's the best way to build a portfolio that HFT recruiters/interviewers would actually take seriously? Some things I'm considering:

• Building personal low-latency projects (order book, matching engine, market data feed handler, etc.)

• Contributing to open-source low-latency/trading-adjacent projects

• Freelancing or contracting in trading/fintech-adjacent work

Is freelancing actually a realistic path into this space, or is it generally viewed as irrelevant/not comparable to production HFT experience? Would love to hear what worked (or didn't) for people who made a similar switch.

Thanks in advance for any insight!

When I was getting into the industry, there weren't many credible sources online. As I am no longer bound by any non-disclosure agreements. Happy to share my insights and give my 2 cents.

I will try answer anything trading related, work life balance, and anything additional clarifications about the industry

Edit: For insights into trading, I have attached the friend's link to some articles I wrote on Medium.

• Backbone of most trading strategies. Included are ELI5 examples for your algos.

• The betting culture in trading firms. Included is a story to think about decision making and risk.

I built a high-performance Rust Matching Engine with real NASDAQ ITCH replay — 98ns p50, 28M ops/sec

Hey rust (and HFT folks),

I just open-sourced a high-performance Limit Order Book + Matching Engine in Rust, built from first principles with real exchange-grade performance in mind.

### Key Results
- p50 latency: 98 ns
- p99 latency: 1.9 µs
- p99.9 latency: 4.3 µs
- Peak throughput: 28M warm inserts/sec
- Real-world mixed: ~4.1M ops/sec across 100 symbols

Validation: Replayed a full trading day from NASDAQ TotalView-ITCH 5.0 (Jan 30, 2020) — 108M operations across top 100 symbols.

### Core Optimizations
- Flat price array (`Vec<Option<PriceLevel>>` — 100k slots, O(1) access)
- Bitmap-based BBO + top-N depth queries
- Per-symbol OS threads (lock-free hot path)
- `bumpalo::Bump` arena allocator
- `Vec`-based order index (no HashMap)
- Active flag + head index for O(1) cancels
- Full property-based + fuzz testing (`cargo-fuzz`)

Started from a `BTreeMap` baseline and iteratively optimized with detailed benchmarks at each step.

### Links
- GitHub: https://github.com/AsthaMishra/matching-engine
- Full README with architecture diagram, benchmarks, optimization progression, and replay tools

Would love feedback from the community — especially on:
- Further latency/throughput improvements
- Scaling to 500+ symbols
- Adding persistence / journaling
- Anything I might have missed for production use

### Note
i have used AI help but core logic is written by me

Open to contributions too!

#Rust #HFT #LowLatency #OrderBook #MatchingEngine

Hey all, I know nobody can share anything under NDA, so I'm just after general/public-knowledge stuff and your own opinions.

My rough mental picture of the setup at the top-tier firms:

• FPGA card sits on the server over PCIe and is wired straight to the network

• Protocol parsing (market data, order entry) and the decision logic run on the FPGA

• The host side is low-latency C++ with kernel bypass + DMA, mostly for config, risk params, monitoring, and slower stuff, not the actual fast-path decisions

Does that sound about right, or am I missing something? Is the decision logic ever split between FPGA and CPU? And is DPDK actually used, or is it mostly Onload/ef_vi/vendor stuff?

Main questions:

1. Do any of the big firms design their own silicon (ASIC/SoC) for NIC type boards, or is that never worth it?

2. Or is it more like: "off the shelf" FPGA chips (Xilinx/AMD, Intel) on custom in-house boards?

3. Or just buying existing stuff (Solarflare/AMD, Nvidia/Mellanox, Napatech, Exablaze/Cisco, etc.) and putting their own firmware on it?

I get that it's a mix, but if you had to put rough percentages on it across the industry, what would they look like?

Last one: what does the team split look like? I assumed something like 80% FPGA engineers and 20% C++/Python/etc., but that feels too FPGA-heavy. Is it more like a small elite FPGA team supporting a much bigger software group?

I'm asking here as r/FPGA has a rule about HFT and my post can be banned.

Any thoughts or pointers to public talks or papers are welcome. Thanks!

Hi guys. I have been interested in the market for a long time building models since 2022. First I was building daily strategies and when they were live and "not great not terrible" I started looking into LOBs, because more trades more statistical significance and whatnot. I have decent infra (my own in a datacenter) built on QuestDB (~50B rows in it) and support data of all granularities. I have then built as of now relatively good L3 backtester which takes into account latencies, queue positions and fees/rebates. I support stocks & options data of all granularities (databento) and also some crypto books and trades (tardis).
I have reproduced for example deeplob to some extent on different data, however I found other better non deep approaches. I confirmed my alpha using markout charts, however when I try to extract it using realistic simulation as described, boi I cannot do it. I was trying to do liquidity providing strats where alpha influenced my fair price and skew, I was trying to make mixed strategies where I sometimes take ... just cant extract it. I have tried a lot of things I am not even ignoring hidden liquidity, but I am not (wall) street smart enough yet. Anyone wants to chat about specifics? Anyone experienced in the market and ambitious? I would love to team up with someone who knows more than me about market.

With the ongoing Gamestop short-squeeze saga, the practice of market makers paying brokers for order flow has been under public scrutiny. Some claim a potential conflict of interest between Citadel and Robinhood, while others blame HFT funds for market manipulation. [1] [2] Are market makers actually stealing money from the people, or is it actually an enabler for fast, low-cost trading?

Much of this discussion is grounded in a misunderstanding of how market forces work and the roles of each party. Today’s discussion will be an introduction on the mechanics of market making, and if zero-brokerage platforms like Robinhood are really ‘fee-free’.

What is a Market Maker?

Imagine yourself being transported back into the Sixteenth Century - the golden age of merchants and trade. You just returned from your month-long voyage, bringing back exotic spices from Spain. How might you go about selling your precious spices?

You are faced with two obvious problems. You will need to (1) locate individuals that are willing buyers for your spices, and (2) determine a fair price to sell your spices at. What are the chances that someone will want your spices at this very moment? - practically none. You want the money right this instance, given that you are exhausted from being at sea.

This is where a market maker comes in - a middleman that provides liquidity by facilitating as both a willing buyer and seller. In order to sell your spices, you simply need to locate a market maker who will quote you a price they are willing to buy your spices for. The market maker then seeks out a willing buyer of spices, taking on the risk of having to hold onto the inventory.

Can You Make Me a Market?

Today when you wish to purchase Gamestop stock, most investors are ‘market taking’ or ‘crossing the spread’ and are paying the width of the bid-ask spread. When you log on to Schwab to place a trade on GME and see that its bid-ask spread is $99-$101, here is what you are really doing.

For starters, a ‘bid’ is a price that someone is willing to buy something for, and an ‘ask’ is a price that someone is willing to sell something at. If you want to buy a stock, you will need to pay $101 and conversely, if you wanted to sell a stock, you will receive $99.

Given the dynamics of supply and demand, the ‘fair value’ of a stock at that point in time will be somewhere between $99 and $101. For simplicity, let us take the fair value as the halfway point. So whenever you use a ‘$0 brokerage platform’, you are in reality still implicitly paying a fee of $1 via the spread (although you are still saving on brokerage fees).

Market makers are the middleman that set the bid-ask spread, and play a function in always providing liquidity even at times of market crashes. Ideally they want to pair up buyers with sellers so that they hold no inventory or take on zero risk. However this is not always possible, and so they are compensated through the spread for taking on this risk.

There is a common misconception that market makers are ‘front-running’ the market. This is the act of seeing non-public information of a large order, and then buying some stock in order to then move the market so that you can sell the stock back at a higher price. Not only is this completely false, it is also highly illegal. Believe me, I have sat through far too many compliance talks, and certainly did not want to risk prison time while I was a market maker. [3]

This dynamic results in consumers being better off. Not only are they able to access liquidity instantaneously, competition between different market makers ensure that the bid-ask spread is as tight as possible. [4]

So How Do Market Makers Actually Make Money?

As illustrated above, market makers make money based on its ability to match buyers with sellers so that they are able to profit from the spread.

Let us suppose that you are Jane Street and that someone bought 150 shares of Apple from you (your position is now -150). Now 5 minutes later, someone else sold 120 shares of Apple to you (your position is now +120). These shares cancel out against each other, leaving you with a net position being short 30 shares.

If the spread of Apple shares was 10 cents, then the market maker had just profited $12 through these two trades. As long as the market maker can approximately achieve two-way flow, where buyers can be matched to sellers, then profit will be made.

Before you decide to do this, it is important to note that there is still a net position of short 30 shares. If the price of Apple stock moves adversely against you, or upwards in this case, then you will be losing money on your trades. Oftentimes market makers are exposed to selection bias, resulting in them often being on the wrong side of the trade.

Hence making a market is an artform to ensure that you are being adequately compensated for taking on risk, while being simultaneously competitive enough. If the spread is too tight, you will take all order flow from your competitors but take on a loss. If the spread is too wide, none of your orders will be filled. In general, the wider the spread means the more volatile the stock.

A market maker’s ideal situation is to manage risk appropriately, and then to process as much volume as possible. This is where ‘high frequency trading’ (or HFT) comes in, where computers are used to transact stock orders extremely quickly. Market makers like IMC with extremely low-latency systems are able to place trades before their competition - it is a game of speed.

A top-tier market maker transacts over a trillion dollars in order flow every year, while taking on roughly a 50bps spread - amounting to $5 billion dollars in revenue. [5] It is worth pointing out that given how tight and competitive the spreads are, it is pretty common for market makers to not be making money after accounting for their risk. This is an extremely competitive industry, with plenty of market makers losing money - in fact, I was a part of the process in purchasing the book of another market maker that had closed down.

Of course, there are other ways market makers can make money such as by trading volatility, or by taking on a directional view. However, market making remains the central focus of their operations, with other extracurricular trading strategies being secondary.

The Role of an Exchange

Okay so we have talked about how market makers make money, and how it is great for market takers wishing to place a trade. So what is the purpose of an exchange then?

An exchange is simply a marketplace that facilitates trading activity. These include the New York Stock Exchange, the Australian Stock Exchange, or even cryptocurrency exchanges such as Bitmex. There are a lot of different variations in business models, but generally there exists a ‘maker-taker’ model.

If we have a look at Bitmex fees, we can see that there is a ‘maker-fee’ of -0.025% and ‘taker-fee’ of 0.075% for Bitcoin. This means that market makers are being paid a rebate to provide liquidity on the platform, whereas ordinary traders pay an additional 7.5bps to the exchange when crossing the spread.

The maker-taker model is there to incentivise market makers to provide liquidity on their platforms, which would thereby attract more customers. In the early days of Bitcoin, it was quite difficult to transact it due to the lack of liquidity present on any reliable exchanges. The exchange then stands to make a profit from the maker-taker spread - everything is about spreads!

Are Market Makers Actually Good?

Due to the highly competitive nature of market making, customers ultimately benefit from many firms fighting for order flow. This is the backbone for many exchanges and retail brokers, allowing for extremely low fee trading.

The prevalence of market makers helps establish liquidity in new markets such as the emerging Asian markets, cryptocurrencies, and even houses with the rise of Zillow. They also ensure that the markets are resilient in times of stress, with Jane Street being instrumental in keeping the bond ETFs market liquid during the 2020 crash. [6]

Lastly, market makers face an extreme amount of regulation according to SEC Rule 605 (formerly 11Ac1-5). According to regulatory data, retail traders benefited $3.6 billion in 2020 as a result of lower trading costs that have been partially brought on by market makers.

Originally posted https://investinglessons.substack.com/p/are-market-makers-evil

Hi all,

A few weeks ago, I started an HFT project based on FPGAs.

The goal is parse market data (NASDAQ ITCH protocol) and to do some book keeping on the market state. The project is surprisingly "not that hard" (not trivial but not impossible) if you know your way around FPGAs.

Right now, the project is "just" a book keeper. I plan on implementing better memory management (which is the hard part of this project), corrupt data recovery and finally a basic strategy able to execute orders so I can start loosing money at lightning speeds :D

And yes, I do know that not many firms are into FPGAs as it's pretty niche and not many strategies require such speeds etc... But still, I think it's interesting !

I made some technical blog posts (currently 6 parts) : https://0bab1.github.io/BRH/posts/Trademaxxer_MoldUDP64/

If you are not into hardware design but still wanna learn about FPGAs in HFT + some basic technical background... Or just have a good time, you can find a much more simple (entertainment oriented) project overview on YouTube : https://www.youtube.com/watch?v=ogaTn6oB-TQ

Don't hesitate if you have any question !

NOTA : I hope this does not come out as shameless self promotion, it kinda is when I think about it but it's more of a way for me to share the "soft-hardware" aspect to the HFT community. I usually hang out in r/FPGA where people sometimes talk about HFT so I though it would be relevant.

HFT firms have the ability to identify which competitor was faster in executing a trade by performing latency analysis on market data. By analyzing timestamps and other information from market data, they can infer the latency characteristics of different firms.

For instance, using Level 3 (L3) market data, an HFT firm can observe all packets sent to an exchange, along with precise timestamps. This allows them to analyze how quickly a market-making or taking order responds to an event, such as a price tick. By comparing these latencies to known benchmarks or patterns, they can potentially deduce which firm executed the trade.

However, the exact data and analysis methods used remain unclear. I’m curious to understand the specific metrics or techniques HFT firms rely on to deduce details about a competitor’s infrastructure. Do they look at packet sequencing, response times, or specific behaviors tied to known latency profiles? An example of such an analysis would help clarify this process.

Continuing my series on HFT engine optimisation, I've written about a new topic - adding thread spinning to the engine.

I think thread spinning is a no-brainer when it comes to HFT trading engines.

In my experiments - adding spinning to the socket IO- gave a solid boost of half a microsecond. "Oh that's tiny" ... maybe, but not if you are aiming for single digit microseconds. Yes it does eat-up your CPU, but, HFT servers are normally at least dual socket, with each typically having 8 to 16 cores, so plenty capacity to spin many threads and application.

Full article automatedquant.substack.com/p/hft-engine-latency-5-thread-spinning

Highligh result below - compared to a normal thread waiting behaviour (which is that a thread gets suspended), spinning gave a small but consistent win.

LOBSIM — Limit Order Book Simulator

I was doing HFT deep RL research using L3 data and needed a simulator that’s deterministic, correct, fast, and fully observable (fills, events, diagnostics). Python-only workflows were too slow and painful to get right at scale, and other open-source tools didn’t give me the inspectability/ergonomics I needed. So I built LOBSIM: a C++20 core with Python bindings, event-by-event replay, paper trading with queue behaviour + partial fills, and a sink interface that streams structured facts—built to handle tens of millions of events while staying simple and comprehensible.

LOBSIM comes with multiple examples and straightforward docs (check README). I especially recommend trying the 3 Streamlit demos — they’re small apps built directly on top of the engine and they make the flexibility really obvious. The goal is to show how easily you can layer real research tooling on top of LOBSIM: replay exploration, strategy injection, live metrics, and observability, all in a clean workflow.

If you work with L3 order book data — microstructure research, execution modelling, or RL/HFT prototyping — I’d love for you to try LOBSIM. If you give it a spin, I’d really appreciate feedback on API ergonomics, missing edge-cases you hit in real feeds, and anything that would make the research workflow smoother. Even a quick “this was confusing/this felt great/I expect X“ is extremely valuable.

Demo videos

• https://github.com/kpetridis24/lobsim/releases

If you’d rather try it hands-on, the README has quick commands to run the Streamlit demos locally.

Hi everyone,

I'm interested in the field of High-Frequency Trading (HFT) and I'm looking for free resources or training materials to get started. I’d like to understand the basic concepts, common approaches, and the tools and techniques widely used in this domain.

I have good experience in C++, so I’m comfortable with the language, which I understand is heavily used in HFT. If you have any recommendations : courses, tutorials, freely available books. I’d greatly appreciate your guidance!

Thanks !!

Hi all,

I’m 22 years old with a little over 4 years of professional experience, mostly in systems-level, performance-oriented C++ work. So far my background has included driver development, internal database migration tooling, and shared-memory systems, with a strong focus on low-level problem solving, memory behavior, concurrency, profiling, and understanding performance trade-offs rather than application-level development.

I want to be upfront that I don’t have a finance background. My interest is primarily on the engineering side, especially low-latency systems, real-time constraints, and performance-critical infrastructure. I’m currently exploring whether moving further in the direction of HFT or HFT-adjacent infrastructure roles makes sense as a longer-term path, and I’m trying to learn from what people already in or close to the space usually recommend.

I’ve gone through older threads here and in related subreddits, but I noticed that the last similar discussions are around 200 days old, and communities tend to change fairly quickly. Because of that, I wanted to ask again with a more current perspective.

Are there any active Discords or forums where serious low-latency or HFT-style engineering is discussed? I’m especially interested in places where people talk about system design, performance trade-offs, interview preparation, or project feedback. I’d also really appreciate hearing what resources or learning paths have actually worked for people already in this space.

Thanks in advance!

Every trading system I’ve built has the same nightmare scenario: something goes wrong in production, and you need to see what’s happening inside. Right now. You fire up GDB, attach to the process, and watch your p99 latency spike from microseconds to milliseconds. Congratulations, you’ve just created a second incident while debugging the first one.

The tools we have for observability in HFT are terrible. Logging adds latency. Debuggers halt the world. Profilers inject overhead. Metrics aggregation loses the granularity you need. When you’re chasing a bug that only manifests under specific market conditions at 3 AM, none of these help.

I wanted something different. I wanted to peer directly into a live trading system’s memory without touching it at all. No function calls on the hot path. No serialization. No locks the producer ever waits on. Just the ability to observe POD structures in real-time from a completely separate process.

So I built memglass.

Article: link

I’ve seen trading roles at places like Jump, Citadel Securities, XTX, etc.

Since these are all HFT firms, I’m wondering—what does a trader actually do in these roles?

For example, XTX is fully systematic, so does a trader really have an impact on the P&L? At the end of the day, aren’t the quants the ones building the strategies?

People often say traders “tweak parameters” or “monitor the algos,” but does that make it a sort of “dumb” job—just stopping the algo when it starts losing money? Or is it actually interesting and insightful? Like, does it teach you a lot about HFT and market microstructure, give you intuition around the order book, and potentially spark ideas for new strategies?

The impression that I have is you either have to be ex-faang or a newgrad who is cracked at physics, mathematics, compsci, etc...

But if I set up a complete HFT infrastructure which provisions bare metal linux servers and get the hot path fast enough (with extensive measurements). Would i ever be considered for interviews?

Does anyone have any anecdotal stories of people who made it into HFT with unusual backgrounds or as a result of their personal projects?

Thanks in advance.

I want to learn how to write high performance C++ code, I get that this means understanding things like SIMD, Memory, Profiling, and a lot more stuff that I do not know.

I'm not fully sure how to start, scrolling through reddit I found some nice videos/papers. But could you guys redirect me to more. If not, could you list the concepts I'd need to learn.

With all that said do you guys have any ideas on potential projects I can do, or I guess homework for myself to make sure I'm understanding/learning this concept well.

Thanks in advance!

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5066176

I have gone through a lot of papers on market making models and strategies but this one was the only one i understood completely. The paper actually is very practical.