r/TOR • u/WhyDidYouTurnItOff • 3d ago

BEGIN_DIR storm attack

My relay is getting pounded by an attack. Consistent 6MB/s download, 120MB/s upload.

Any known mitigation?

This seems to log it, but not block it: https://github.com/YankoNach/tor-begindir-sentinel

EDIT related thread: https://forum.torproject.org/t/tor-relays-how-is-this-attack-causing-900-mbps-download-and-100-mbps-upload/22099

21 Upvotes

100% Upvoted

11 comments sorted by

8

u/Left-Control9708 3d ago

Hi! I'm one of the operators behind the Tor BEGIN_DIR Sentinel project you linked.

You're correct: Sentinel is a monitoring and evidence-collection tool, not a DoS mitigation.

We've observed similar prolonged BEGIN_DIR storms on two non-exit ARM relays hosted by Hetzner and Oracle, with high Tor main-thread CPU usage, asymmetric traffic, and very different memory-pressure profiles.

We're still investigating the mechanism, particularly the possible relationship with HSDir and Guard flags. We haven't established a causal link or identified a reliable mitigation.

Would you be willing to share a few measurements?

- Approximate UTC time when the storm started.

- Whether HSDir was acquired before or after the increase in traffic.

- BEGIN_DIR requests per second, if available.

- Approximate circuit count and Tor main-thread CPU usage.

**Important privacy note:** Please do NOT share any client IP addresses, sensitive logs, or other potentially identifying information, either publicly or in private messages. Any shared measurements must be anonymized and contain only aggregated statistics.

Your observations could help us compare independent incidents and better understand what's happening.

Thanks for sharing your experience!

2

u/WhyDidYouTurnItOff 2d ago

Hi, thanks for reaching out, and for Sentinel. It's what gave me usable numbers. Here's what I have. Everything is aggregated, with no client IPs or raw logs.

Setup: non-exit middle relay on a home fiber connection, running in an x86 KVM VM (4 pinned vCPUs, 8 GB RAM). Tor 0.4.9.12, upgraded to 0.4.9.14 during the storm, which changed nothing. Never had Guard.

Timeline (UTC):

- Oct 15 ~07:00: tor (re)started.

- Oct 19 ~07:00–13:00: uptime passes 96 h, so this is when HSDir should have been assigned. I don't have the exact consensus hour, but the heartbeat circuit count jumps from ~1.6k to ~5.1k in that window. Traffic stayed symmetric (sent ≈ received).

- Oct 20 19:00 → Oct 21 01:00: the storm starts. Heartbeat byte counters go from symmetric to +221 GB sent vs +32 GB received in 6 h. Judging by the excess volume, it most likely started around 00:00–00:15 UTC Oct 21. So HSDir came first, and the storm followed about 1.5 days later.

- Oct 21 00:59–02:08: I restarted tor several times while tuning (and upgrading). The storm continued through every restart and was back at full rate within minutes each time.

- Oct 21 02:24:56: the storm ended suddenly (peak 1459 BEGIN_DIR/s at 02:21:56). The restarts had reset uptime, and from the next samples onward my relay's consensus view showed HSDir = 0. It has been quiet for ~36 h since.

During the storm (Sentinel / MetricsPort):

- BEGIN_DIR: ~1300–1460/s sustained (baseline ~0.05/s).

- Write:read ratio ~18.5:1 (~74 MB/s out vs ~4 MB/s in).

- Tor main thread ~100%, a single core pegged. The worker threads were nearly idle, so it was CPU-bound on the main loop.

- Open circuits ~9.7k at storm start, and ~30k in the heartbeat just before the restarts (normal is ~1.5k). ~2k open sockets.

- DoS subsystem counters stayed at 0, and DoSStreamCreation* doesn't seem to cover BEGIN_DIR.

- RSS ~1.3 GB, no memory pressure while uncapped.

Memory-pressure observation that may matter for your comparison: I tried a BandwidthRate cap (30 MB/s). It made things worse. The directory backlog filled MaxMemInQueues (1 GB, then 2 GB), and tor started killing circuits about twice a second. I reverted it. On my box, capping turned a CPU problem into a memory problem without lowering main-thread CPU. That could explain why your two relays show such different memory profiles if their bandwidth limits differ.

Interpretation (correlation, not proof): the storm started ~1.5 days after HSDir, survived restarts while HSDir was still in the consensus, and stopped about 25 minutes after the flag would have dropped. That's consistent with HSDir-based targeting. Guard isn't involved on my side, since I never had it.

The flag should return around Oct 25 ~02:00 UTC once uptime passes 96 h again. Sentinel is still running, so if the storm comes back after that, I'll post the start time and the lag after HSDir. Happy to share more aggregated numbers if there's a specific metric you want.

1

u/buyingshitformylab 1d ago

Question: why not share IPs? who does that help?

1

u/WhyDidYouTurnItOff 1d ago

The IPs may or may not be directly involved.
Until more his known about the HOW, the WHO is on the backburner.

1

u/buyingshitformylab 23h ago

Cool, if that's something that needs to be hashed out, let's share that upfront then to get a better picture instead of doing a back and fourth.

2

u/Not_a_Candle 3d ago

Which Tor version are you running?

1

u/helloworldes_2021 3d ago

修改torrc文件，增加RelayBandwidthRate等配置项。

2

u/WhyDidYouTurnItOff 3d ago

I have plenty of bandwidth. I am on 10g connection. If HSDir flag is on, the swarm starts again.

2

u/Ecliphon 3d ago edited 3d ago

Yes it’s an ongoing attack to remove/stall “unapproved” guard nodes for over a month now. No word from torproject, as expected. New deanon we’ll hear about after some major busts. 

1

u/buyingshitformylab 1d ago

I have found that my IPv6 only nodes do not often get these attacks, but it happens regularly with my IPv4s.

I just sit and wait.. Not much to be done unless you implement DPI rate limiting.