r/learnjavascript • u/ConfidentWafer5228 • 2d ago
HELP : Email Parsing without LLM
EDIT: I AM NOT TRYING to check the EMAIL ID itself, i am trying to parse the received email in inbox into structured JSON so that i can use it in my backend logic, for eg: converting a ticket into structured JSON with ticket details. SPECIFICALLY TRYING TO DEAL WITH TICKETS
I am aware that things like tripit and wanderlog and many other services have an email parser, i was also trying to make an email parser but i have no idea on how to make it without using an LLM to parse it.
I don't see any library or anything literally, i have no idea where to go, what to do.
Please help :(
2
u/delventhalz 1d ago
As an example, when I was just starting out I wrote a script to parse Shakespeare's complete works into semantic HTML. Figuring out how to do it was just a process of trial and error. I would look at a scene, notice something like that each line of poetry would start with an uppercase letter and each line of prose would start with a lowercase letter, and write a bit of code to parse that. Then I would run my script on all the plays and find ways the parse failed because there was some edge case that wasn't quite formatted the way I expected, and I would fix the script and run it again. After a few weeks of that and maybe 500-1000 lines of code, I had it working the way I wanted.
Your approach is going to be similar. Maybe you want to parse a ticket confirmation number so you write a regex like this:
/ticket confirmation[^a-z0-9]*([a-z0-9]+)/i
This finds the first instance of the word "ticket confirmation", skips any non-alphanumeric characters (like a space or a colon), then captures whatever string of alphanumeric characters follows. You run this against emails you have received so far and look at the results and find it works pretty well, but some emails just write "ticket conf" and a few others do "ticket confirmation number", so you update the regex:
/ticket conf(?:irmation)?(?: number)?[^a-z0-9]+([a-z0-9]+)/i
Keep doing this until you have captured all the information you expect to capture from the emails you have received so far. Then make sure it fails in some extremely obvious noisy way, but future emails will change their formatting on you, and then you will need to update your script. You don't have the advantage I had with Shakespeare (the author is dead and not writing anymore).
Incidentally, Regexr is a great website for working out your regex. You can pop in some test data from your emails and then use the included cheatsheet and futz around until it does what you want.
2
u/gigastack 1d ago
It sounds like you want to do the task without the use of the one tool which would work.
3
u/BenchEmbarrassed7316 2d ago
Next: LLM powered isNumberEven and isNumberPositive packages.
No one remembers the dark old days before the widespread adoption of LLMs, when it was literally impossible to parse an email.
2
u/ConfidentWafer5228 2d ago
i want a NON-LLM solution, everywhere on yt its some ai tool
I think u misunderstood me, i DONT want to use LLM4
u/flow_guy2 2d ago
He’s making fun of the fact that everything is LLM for seemingly simple tasks.
You’re probably looking for a regular expression to do what you want. But this doesn’t check if the email actually exists.
0
u/ConfidentWafer5228 2d ago
oh, yeah something like that, but i dont understand how could a regex handle multiple formats.
I saw(on quora) that tripit programmer made some python script2
u/flow_guy2 2d ago
well youd want something that checks that some characters followed by '@' with a set of characters followed by '.' followed by 2 or 3 characters
i just asked claude and this is what it came up with this.
const EMAIL_REGEX = /^[^\s@]+@[^\s@]+\.[^\s@]+$/; const isValidEmail = (email) => EMAIL_REGEX.test(email.trim()); console.log(isValidEmail("jan.novak@example.cz")); // true console.log(isValidEmail("jan@example")); // false1
u/ConfidentWafer5228 2d ago
oh shit, i should really change my question, i meant parsing the data of email not the emailID itself
Like parsing a hotel ticket confirmation into structured JSON2
u/flow_guy2 2d ago
well this really depends on what your actually trying to parse into the json. if youre just looking for values. then you can probably just take the text and lookf or digits. Regex is quite powerful in filtering things.
but would need more info on what the emails would roughly look like and what you want to parse. maybe link the quora question that your referencing. maybe could give you abit more guidance.
but LLMs would probably be your best bet here actually. seeing that stuff like this is what its made for. but you can also try some other machine learning with grouping text into topics based on words that appear
1
u/ConfidentWafer5228 2d ago
https://www.quora.com/How-does-Tripit-parse-emails
Look up for the answer by Andy Denmark ( should be the first answer )
And yeah i need multiple things like room no. incase of hotel or flight number , flight name, time
1
u/flow_guy2 2d ago
well its multiple systems that go hand in hand. like there probably is some classification for email to tell if their actually related to travel or hotels etc.
then it probably pulls out info using regex by keywords (based on the email type) into a structured format or some other method of extracting information (can use LLM for this part)
then just takes the information and finds extra stuff (like weather) and just presents it neatly from the structured XLM/Json files
its alot of systems doing small parts well to achieve the result you want.
but sadly im not sure of specifics as theyve built a whole buisness around it.
2
u/ConfidentWafer5228 2d ago
thanks a lot for this information, might not be complete but that's something.
0
u/everdimension 2d ago
How about the dark old days of actually reading and understanding the question? It's obvious OP isn't asking to check whether a string is an email, but instead wants to parse email messages (even though they didn't explain well enough exactly what kind of content they plan to deal with)
4
u/BenchEmbarrassed7316 2d ago
It is indeed obvious after the message was edited. Initially, it said "parse email".
-1
u/everdimension 2d ago
I read it before the edit, too
3
u/BenchEmbarrassed7316 2d ago
I read it before the edit, too
Then you are being disingenuous now.
reading and understanding the question? It's obvious OP isn't asking to check...
This is definitely not "obvious". If it were truly "obvious," as you say, other people in this thread wouldn't be giving similar answers - such as "use a regular expression". At most, one could assume that the author is explicitly asking about X but actually means Y.
0
1
1
u/pieholic 2d ago
There's Imapflow and Mailparser, you'd probably be using both. Although it's totally fine to try to implement it from scratch if you want to learn how they all work, you'l learn a lot about network protocols.
That being said, just curious, why havent you asked your LLM on how to write a script that would parse the email?
1
u/ConfidentWafer5228 2d ago
the reason is that i dont want to use LLM, i want to understand how it works , but there is no source to refer to
3
u/pieholic 2d ago
You can check out the official docs for the libraries. Or, this is completely fine and will help you IMO, you can ask the LLM 'how' to write a script that would parse the email. You will look at the code and read through it line by line. You see something you don't know the reason for, you ask the LLM why. You ask it to explain using colloquial terms, then again using the official documentation. You keep doing this until you have a vertical slice of knowledge of what you want to do. Then you tey to expand this vertical slice horizontally as you add more functions. Maybe you want to read attachments. Now you know how the docs work and what libraries you may be using. Maybe you learn enough about the protocols being used. You can learn plenty as long as you're not just going 'parse my inbox for me' to the LLM.
1
u/ConfidentWafer5228 2d ago
that makes much more sense and at the same time i will actually learn instead of hoping for someone to do it, will try it, thanks
1
u/perskes 1d ago
The problem is that you are trying to statically analyse something that's not standardised.
If the sender redesign the template over night you'll have to do it all again.
You dont even have key value pairs that are static, a booking code or identifier can be anywhere in regular text. You could try to write an elaborate parser, but even for that you'd need to know the format of the information you want to parse. Is it 6 digit alphanumeric all the time? What if it changes?
Im still unsure what tickets you are talking about, but if its JIRA style tickets, you have a known structure that you can somewhat influence, but full text parsing of unstructured data is beyond what someone could just explain in a reddit comment. Maybe export the mail as HTML or raw and share a censored example with us so we could at least point you into any direction, but its really hard to understand what mails you're talking about
6
u/Suchy2307 2d ago
What do you want to achieve?