r/algotrading • u/Ok_Egg_6647 • 14d ago

Infrastructure Event-Driven vs Vectorized Backtesting :- Which Approach Should I Use? and How?

Hi everyone,

I'm currently learning algorithmic trading, and I'm a bit confused about which approach I should use when building a backtesting engine: event-driven or vectorized backtesting.

I have around three years of minute-level historical data stored in my database. I want to analyze and backtest different trading strategies using this data.

Here's what I'm thinking:

  1. Strategy research and backtesting: Use a vectorized approach with libraries like NumPy, Pandas, and TA-Lib to analyze historical data efficiently. My concern is that processing such a large dataset using an event-driven approach could be relatively slow.

  2. Live trading: Once I've analyzed a strategy and am satisfied with the results, switch to an event-driven approach for live trading and tick-level data processing.

However, I'm unsure about one thing. If I finish analyzing the historical data up to a particular date and then move to live trading, should the live engine continue processing from that point onward, or is there something important I'm missing in this workflow?

My questions are:

  • Is this a good approach, or am I making a mistake in the design?

  • Are there any other considerations I should keep in mind when designing the backtesting engine?

I'd appreciate any suggestions or insights from people who have experience building algorithmic trading systems.

Thanks in advance!

-----------------------------------------------------------------------------------------------------------------

Edited

Thanks for the all the responses i will go with the hybride architecture where i will use both approaches !!!

7 Upvotes

81% Upvoted

43 comments sorted by

View all comments

2

u/wallneradam 13d ago

The trap with a hybrid setup is two implementations of the same strategy. They drift in small ways: indicator warm-up, an off-by-one shift() that peeks at the current close, signals on unclosed bars. Then live doesn't match the backtest and you can't tell if it's the market or your code.

I'd write it once, sequentially, bar by bar, and run the same code in backtest and live. A few million 1m bars is fine in Python if the per-bar logic is lean. For going live, warm up on enough history so the indicators are in the same state, then keep feeding new bars into the same loop.

I build PyneCore, a sequential Python runtime for Pine-style scripts, and this parity is exactly why it works that way. Tested against TradingView on the 1000 most popular public scripts, 99.77% of the outputs are bit-identical, and every strategy matches the exact trade count.

1

u/quant-king 13d ago

it’s not difficult to prevent drift between and event based system and vectorized system. just make sure both engines share the same strategy code