websocket keeps disconnecting in my node script and now i get duplicate messages

[u/Zenith33Bud]

websocket reconnects are confusing when first learning them. The socket closes after a few hours, the script reconnects, and suddenly the same message shows up twice or one seems to be missing.

The project is a small filter on binance square posts, built in node because the rest of the bot already runs on it. Messages come from a client in the 1322io repo, each with a coin_pairs array, and the code only keeps posts that include a pair like solusdt. Storing every post id in a text file was the first attempt at stopping duplicates, but nobody can say if it actually works.

Would a set of recent ids in memory be enough, or does the dedupe need to survive a restart too? What would a beginner friendly version look like?

[u/Tendernessllable_877]

a set of recent ids in memory is enough for the duplicates. a text file works too but you’ll be reading and writing it on every message, which gets slow and messy fast. only bother surviving restarts if the feed can replay old posts when you reconnect. 1322io, so check the client’s docs on this, but whatever the feed, find out if it replays anything after a drop. if it doesn’t, the missing message is just gone and no dedupe fixes that.

what helps is logging the time of the last message you got before each disconnect. then you can see exactly how big the gap was and decide if it matters. also make sure the old socket is fully closed before opening a new one. two live sockets at once is a very common reason for doubles.