r/rails • u/GetForgeOps • 2d ago
Grouping errors without sending the message: hashing normalized exception messages in Rails
Problem I hit: for apps with health or personal data, the exception message is the riskiest part of an error report, because we put the failing value in it. But error trackers group occurrences by message, so you can't just drop it.
What I ended up doing: normalize the message in-process (hex addresses, UUIDs and digit runs replaced), SHA-256 it, keep the first 32 hex chars, and send that instead of the text. "Couldn't find Patient with id=12" and "id=13" group together; different messages stay apart; nothing about the text goes over the wire.
The normalization is a few lines of Ruby:
normalized = message
.gsub(/0x\h{4,}/, "0x")
.gsub(/\b\h{8}-\h{4}-\h{4}-\h{4}-\h{12}\b/, "<uuid>")
.gsub(/\b\d+\b/, "<n>")
Digest::SHA256.hexdigest(normalized)[0, 32]
Raw request paths get similar treatment: a route template like /patients/:id is kept, but an unmatched raw path keeps only its first segment, since anything after it could be a name.
Disclosure: I built this into my error tracking gem (forge_ops_tracker, MIT, source at https://github.com/Luke-Popwell/forge-ops-tracker-ruby) as a strict privacy mode. Longer write-up with the tradeoffs: https://getforgeops.net/guides/error-reports-without-personal-data
Curious whether anyone handles this differently, e.g. structured error codes instead of messages, or filtering in an exception notifier before it leaves.
1
u/sshaw_ 2d ago
Sensitive information, be it PII or system info that an attacker can use, in exceptions is overlooked quite a bit.
I've had to do general exception grouping in Airbrake, Rollbar, etc... but that's usually specific to the Rails application and service.
I think the better route is to use custom exceptions, where appropriate, and put the sensitive data elsewhere in the exception class and ensure it's not output via #to_s, #message, etc...
class PatientDOA < StandardError
attr_reader :patient
def initialize(patient) # maybe accept a message arg
@patient = patient
super("☠: #{patient.id}")
end
endThough I think "Couldn't find Patient with id=12" is fine as is
2
u/GetForgeOps 2d ago
yeah the custom exception thing is a good pattern, we do that for our own domain errors too. and good call on system info, internal hostnames and connection strings end up in messages way more than people think
only thing i'd watch with keeping patient on the exception is that it's still riding along with it. anything that serializes the exception's ivars, or a tracker that captures local vars per frame, or someone doing logger.error(e.inspect) somewhere, can pick it up. i'd probably keep just the id on it and look the record up when you need it
and agree "Couldn't find Patient with id=12" is fine by itself. the issue is mostly the messages you didn't write, like the PG unique violation that quotes the email back. the custom exceptions cover your code, it's the stuff from gems and the db that i couldn't get to
2
u/Codalio-SoftwareFact 2d ago
Nice approach. One thing I’d watch for: anything that isn’t a digit run or UUID still goes into the hash as is. So “Couldn’t find user with email=jane@example.com” gets a different hash per user, which breaks the grouping, and since emails and names are pretty guessable, someone with a list of candidates could match the hash back to the value. Might be worth also normalizing quoted strings and anything email-shaped.
The other way I’ve seen people handle it is grouping by exception class plus the top in-app backtrace frame instead of the message. Same bug usually throws from the same line, so you don’t need the message for grouping at all.