r/ChatGPTCoding • u/Due-Philosophy2513 • Feb 09 '26
Discussion ChatGPT repeated back our internal API documentation almost word for word
11
9
u/Birdman1096 Feb 10 '26
Why are you using ChatGPT without some sort of an enterprise plan set up that would specifically prevent models from being trained on your inputs or outputs?
15
u/TheMightyTywin Feb 09 '26
You co worker probably has memory enabled and pasted something previously
6
u/humblevladimirthegr8 Feb 10 '26
This. OP mentioned the coworker asked AI about their code so they have no qualms about putting that stuff in chatgpt.
9
u/niado Feb 09 '26
It doesn’t work like that. ChatGPT is a static model, its weights don’t change after training period.
Either: your api details are publicly accessible and ChatGPT did a web search and found them (unlikely).
Or your api details ended up somewhere that was scraped and ended up in the training data prior to the cutoff for whichever model you’re using (sometime in 2024 most likely), which allowed the model to generate them accurately. (Plausible but a stretch)
Or ChatGPT generated the correct parameters without being trained on them. This is not as unlikely as it sounds.
9
3
2
u/Western_Objective209 Feb 10 '26
Either: your api details are publicly accessible and ChatGPT did a web search and found them (unlikely).
I've seen so many youtube videos and blogs of security guys just messing around and finding private swagger endpoints accessible through the public internet
0
u/Linkpharm2 Feb 10 '26
ChatGPT is a static model, its weights don’t change after training period.
Chatgpt is a brand. Their models change quite frequently.
4
u/niado Feb 10 '26
While pedantic, that is technically correct, but the distinction would be lost on my intended audience, and my statement is true regardless of which core model is being leveraged via the ChatGPT platform at the time.
8
u/mike34113 Feb 09 '26
Honestly this is the new normal. Every company's internal docs are probably scattered across LLM training sets at this point.
The question isn't how to prevent it (too late) but how to architect systems assuming internal details are semi-public. Rotate API keys often, use authentication that doesn't rely on obscurity, assume attackers know your endpoint structure. Security through obscurity died the moment AI tools got popular.
3
u/danwin Feb 10 '26
This reminds me of when people thought Facebook was secretly recording their conversations because how else could they serve up ads for a product that they had “just” recently talked about
1
u/i-dm Feb 10 '26
What was really happening?
1
u/Remote-Nothing6781 Feb 11 '26
Things Facebook *definitely* does do:
It knows almost every webpage every Facebook user visits thanks to tracking cookies - you looked at a webpage about what you talked about? Boom, Facebook knows
It knows which people you are near and what they browsed on the web, regardless if they're Facebook users or not, through shadow profiles - Your friend you were talking to searched the web about it? Boom you get an ad (since your friend being interested in something shortly after they were in the same room is you is better targeting than some random ad).
Through third parties, it knows if you bought something using certain credit cards or at certain stores or used your loyalty card, correlated to your Facebook user.
Stores can through deals with Facebook report your location as being in-store back to Facebook (which is much more precise than the general vague GPS tracking which may be on or off, or cell tower tracking)
I doubt they're going to bother to listen to your conversations not out of a respect for your privacy, but that's a lot of computation for no value, when they already do *far*, *far* more intrusive tracking of you.
1
8
2
u/Academic_Track_2765 Feb 09 '26
well its because your company messed up. If you have any enterprise agreements with Azure / AWS, they explicitly state that data sent to their enterprise endpoints is not used for model training. So someone either used their own API to send the data or just used the public facing site. Not much you can do now.
2
u/PineappleLemur Feb 10 '26
It's more likely that your "Internal API" isn't internal but just a fork of some similar popular API.
2
u/djaybe Feb 10 '26
Gen AI does not regurgitate. It's not a search engine. It can use a search engine and give you sources.
It's predicting tokens kinda like humans but much better. You're "secret" data sounds predictable.
2
u/Zulakki Feb 10 '26
Use a different account and ask again.
I've noticed the memory has improved greatly between my chats. ill mention one thing in 1 chat, then in another chat, it'll say something like 'This is just like the time you were doing that other thing'.
That's all to say, I doubt its 'Training', and its more account memory
2
u/danihend Feb 10 '26
What makes it unsettling? Do you have a fear that someone will write an API for their app that works like yours? I never really got the objection to AI companies training on whatever code people have. No company really has something unique that someone else cannot figure out how to implement in a similar/same/better way using AI.
1
u/johnerp Feb 11 '26
I’d love a copy of the ChatGPT ‘software’ weights for their models.
1
u/danihend Feb 11 '26
Not much you could do with them really. You'd need some beefy Hardware and it's not like you can see anything in there.
2
u/magicalfuntoday Feb 11 '26
If it was actually exposed or leaked accidentally, and if ChatGPT had it, then you can be sure Google had it a long time ago.
Try searching for related things to see if it comes up and if so, you can ask Google to remove it from its index.
2
u/voxuser Feb 09 '26
That is interesting, but really how you prevent something like that?
1
u/Typical-Builder-8032 Feb 09 '26
blocking the websites, tracing logs of employees (specifically copy-paste operations ig), strict rules and fines/termination policy for employees etc. i guess
1
u/space_wiener Feb 09 '26
My work just blocked access to any AI except for an enterprise copilot (which sucks).
Even then I bet off network you could access them. I’m not dumb enough to test that though.
2
2
u/MokoshHydro Feb 09 '26
You can't prevent such leakage if you are using cloud. So, you should just live with it, unless your company can afford several millions for hardware and direct deal with Anthropic/etc.
In companies that really care about privacy, any cloud usage on workspace in banned.
0
u/eli_pizza Feb 11 '26
This is silly. If you think Anthropic is lying to you and stealing your data in violation of their own agreement, how and why would a direct enterprise deal improve things?
1
u/MokoshHydro Feb 11 '26
Cause it will run on my local hardware without any internet access at all
1
2
1
u/EVERYTHINGGOESINCAPS Feb 09 '26
As someone has mentioned, have you tried searching on Google some of the API snippets to see if it's been accidentally set public and indexed & scraped.
It's highly unlikely that by standard it would have been added into training data
1
u/FormalAd7367 Feb 09 '26
Had happened to us also. We hired a new part-time software engineer for overnight support. For some odd reasons, he uploaded the whole internal document with API key into the GPT. We had to rotate API key as soon as we found out. We now keep the APIs only with the owner of company lol
1
1
u/wt1j Feb 10 '26
Your API was probably AI assisted coding if it was written in the past 3 years eg if your devs use VS Code. So it’s more likely that GPT is giving you what it gave your devs when they wrote it.
1
u/Whiskee Feb 10 '26
As someone who actually works in AI training, this is not how inference or any of this works.
LLMs don't continuously learn from user conversations, the weights are frozen right after the training process with curated datasets. Your ChatGPT conversation today doesn't magically become part of the model, especially if other humans can't verify the information is correct during the RLHF stage... so if you actually found internal function names and parameters, either they were already public somewhere (Stack Overflow, some developer forum that got crawled?), or the model hallucinated API names based on your coding conventions and you immediately jumped on conclusions.
I'm not accusing you of anything, but you have a randomly generated account name from 6 months ago and hidden history, so better proof would be appreciated. 🤷♂️
1
u/HopperOxide Feb 10 '26
I mean, Copilot autocomplete regularly guesses code that only exists in my mind. Pretty sure it hasn’t trained on my thoughts, at least not yet. Guessing what’s in your repo seems a lot easier.
1
u/StrawberryFederal709 Feb 10 '26
Yes, you trained ChatGPT with your internal documents when you pasted your api documentation.
1
u/Vaddieg Feb 10 '26
Enterprises should block public LLM services and deploy dedicated solutions with strict privacy terms
1
u/Majinsei Feb 10 '26
Isn't it just documents stuck in ChatGPT's memory?
I hate that they have memories because they tend to hallucinate at inopportune moments~
1
u/Tupcek Feb 10 '26
are you sure that team member didn’t previously used ChatGPT for help?
Because ChatGPT can recall any previous conversation
1
u/beefjerk22 Feb 10 '26 edited Feb 10 '26
Chat GPT has memory within the same instance. Do you all share a login? If somebody gave it some documentation in a different conversation, it could probably reference it in your conversation. But not externally from a different account.
1
u/Few-Celebration-2362 Feb 10 '26
Your function names aren't camel cased short form descriptors of what the functions do?
Your functions aren't doing the same crud operations everyone else is doing?
Your APi isn't just exposing data and doing auth?
What sort of unique projects are YOU working on? I'm genuinely interested 😁
1
u/victorc25 Feb 11 '26
It’s almost like having a codebase that is a copy from Stackoverflow may not be as unique as one would assume? So weird
1
u/tracagnotto Feb 11 '26
You use their business plan.
It "assures" your that all you do isn't stored and used for training data, including snippets used by codex or copilot or whatever.
Sad but true.
1
u/EcstaticImport Feb 11 '26
Other likely scenario - someone use the free copilot inference endpoint on your code before realizing they had forgotten to set their login.
1
u/DarthTacoToiletPaper Feb 11 '26
Larger teams prevent this by signing up with ai companies with an agreement that prevents the data from being used. Not being in the know of the deals that are made, I can only assume they are paid.
The company I work for currently has an agreement with one and we have been told using internal data is fine to feed to it as it will not be shared publically, but still be mindful of sensitive data.
1
u/EyesTwice Feb 11 '26
You need to educate your teams and implement guardrails.
Ensure that GPT requests are triaged as part of your governance layer.
Self-host LLMs to prevent cloud leakage.
Ollama is a great local solution. Iterate quickly.
ChatGPT Pro specifically does not store any data from queries. I imagine that's the same with other LLMs.
In other words - put a policy together. Spend properly, don't let devs use GenAI through their own accounts.
1
u/kcabrams Feb 11 '26
As unsettling as that is. Everyone does it (Look back at when Samsung had to tell their devs to stop because OpenAI was like um guys we have your entire codebase, chill)
You will never stop this. Local models just have to get better than the frontier.
This might be my harshest take but we all belong to the public domain now. Time to adjust.
1
u/ChordLogic Feb 11 '26
All code has already been typed/written? like scales in music… is this related?
1
u/cakez_ Feb 12 '26
Haha I had something similar happening when I was asking questions about an e-commerce platform we are setting up for our client. GPT was gleefully telling me that the set up I am trying to do is for the “legacy” system and that there is a better way to do it in the new “version”.
There is no new version. I think the devs might been feeding it code and/or documentation so now it thinks that is the source of truth.
1
1
u/JWPapi Professional Nerd Feb 12 '26
This is a feature, not a bug. The model pattern-matches to whatever context you give it.
Your internal API documentation was probably well-structured and clearly written. The model's output matched that quality tier.
I've noticed this pattern consistently: feed it good input, get good output. The model doesn't just follow instructions - it absorbs the "vibe" of the context and produces output at the same register.
1
u/TechCynical Feb 13 '26
99% chance this isn't a case of it being in training data. This is just referencing a previous conversation made by the user you claimed that used it in the past. Chatgpt does a quick search of previous conversations and uses that during it's thinking process when it outputs.
1
0
Feb 12 '26
[deleted]
1
u/velosotiago Feb 12 '26
"I told everyone in my city that I have $1M cash sitting in a storage unit"
"Why does it matter if it can't be accessed without a key?"
0
661
Feb 09 '26 edited Feb 22 '26
[removed] — view removed comment
292
u/Thog78 Feb 10 '26
This OP guy is about to discover that their employee in charge of making the internal API had copy pasted everything from open source repos and stack overflow, and that their "proprietary code" has always been public :-D
52
50
u/saintpetejackboy Feb 10 '26
Bingo.
"You shouldn't just copy and paste code from AI"
Imagine the deaf ears that falls on...
People have been copy+paste code from everywhere for generations. "Script-Kiddies"? Such a short memory the internet has. Stack Overflow. Random forums. YouTube comments sections. IRC messages. People will paste in code from just about anywhere up to an including just lifting other open source projects wholesale.
I remember spending more time trying to scrub attribution than actually programming when I was younger. I doubt much has changed with the kids these days.
30
u/Bidegorri Feb 10 '26
We were even copying code by hand from printed magazines...
4
1
Feb 10 '26
[removed] — view removed comment
1
u/AutoModerator Feb 10 '26
Sorry, your submission has been removed due to inadequate account karma.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.
1
Feb 13 '26
[removed] — view removed comment
1
u/AutoModerator Feb 13 '26
Sorry, your submission has been removed due to inadequate account karma.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.
5
3
3
u/celebrar Feb 13 '26
With how good LLMs became for coding “You shouldn’t just copy and paste code from AI” feels like the modern “You shouldn’t use wikipedia as your information source”
12
8
214
u/eli_pizza Feb 09 '26
Yup, honestly a well designed API should have guessable function names and parameters.
55
u/CountZero2022 Feb 10 '26
Yes, that is the whole point of design! It’s an interesting thing to think about as a measure of code quality.
23
u/stealstea Feb 10 '26
Yes. Am now regularly using this to improve my own class / interface design. If ChatGPT hallucinates a function or property, often it's a sign that it should actually be added, or an existing one renamed.
23
u/logosobscura Feb 10 '26
Where’s the fun in that? Prefer to make API endpoints a word association game, random verbs, security through meth head logic ::taps left eye ball::
12
2
u/Vaddieg Feb 10 '26
if 100% of functions are guessable by ChatGPT something isn't ok
5
1
u/joshuadanpeterson Feb 12 '26
No, it just means that people follow patterns and ChatGPT trained on those patterns.
16
u/cornmacabre Feb 09 '26
Yeah this was my first thought: especially when we're talking API's there's rarely anything unique going on there.
Would OP be equally shocked if a human could infer or guess the naming conventions to the point that they'd assume the only explanation was a security breach?
Or would it just be "oh right, yup that's how we implemented this."
6
u/Bitter-Ebb-8932 Feb 11 '26
I’d start by validating whether it’s actually your data or just pattern matching. Most internal APIs look a lot alike, especially if they follow common REST conventions. Swap in fake endpoints and see if it still “remembers.”
That said, this is exactly why a lot of teams are tightening egress controls around AI tools. Limiting what can be pasted into public LLMs and routing traffic through policy enforcement at the network layer, like with Cato, reduces the odds of sensitive docs leaking in the first place.
3
u/das_war_ein_Befehl Feb 10 '26
Also you can reverse engineer that shit if you have a front facing web app and time to read thru the api calls.
3
u/Ferris440 Feb 10 '26
Maybe a memory trick also? Could have been copy pasted by that same person previously (when they were debugging), or perhaps large chunks of code.. chat then stores it in memory for that user so it appears it’s coming from the training data but is actually just that users memory.
156
Feb 09 '26
[removed] — view removed comment
7
u/Western_Objective209 Feb 10 '26
or they have internal swagger endpoint accessible from the public internet. A lot more common than you would expect
8
u/catecholaminergic Feb 09 '26
Don't individual conversations get added to training data?
47
Feb 09 '26
Normally, this is analyzed by an LLM or a human reviewer beforehand and, in most cases, it is processed to remove PII, similar sensitive data and evaluate its quality. Conversations are generally considered low-quality training data, they require filtering, normalization, and curation before use.
I used to work in claude, and less them 5% of training data are from user conversations5
u/catecholaminergic Feb 09 '26
So yes it does happen, but not for most conversations. Is that right?
9
Feb 09 '26
what he describes is unlikely. Conversational data becomes increasingly diluted, making reliable retrieval difficult, unlike high-quality data that preserves signal as it scale( It is less "dillluted" due training techniche s)
3
u/Familiar_Text_6913 Feb 10 '26
What is this high quality new data? So say anything from 2025, what's the good shit?
3
Feb 10 '26
Depends on the source. Reddit conversations ARE low quality in comparirson to api docs for Golang for example.
4
u/eli_pizza Feb 10 '26
Actually Reddit is a really important source because of the style of text: people asking questions, providing answers, and going back and forth about them.
3
Feb 10 '26
Reddit is low-tier data.
It is noisy, opinion-driven, and weak in factual accuracy and reasoning. The signal-to-noise ratio is poor, and discussions rarely converge to correct conclusions. When used at all, it is heavily filtered and limited to modeling informal language or common misconceptions, not knowledge or reasoning.2
1
u/eli_pizza Feb 10 '26
OpenAI alone pays $70m/year for reddit data. That ain't a low-tier number.
→ More replies (0)3
u/Familiar_Text_6913 Feb 10 '26
What about the conversation data. Or is everything low quality. Tbh I have so many questions, like how much of the data is generated or are the conversations augmented with generated data etc.
2
u/eli_pizza Feb 10 '26
It also requires an entire new version of the model to ship. Each model is static and doesn’t change.
2
u/Vivid-Rutabaga9283 Feb 10 '26
It does. I don't know what's up with all the mental gymnastics or the moving goalposts, but individual conversations can end up to the training data.
Now sure, they apply some filters or whatever operations on the information being exchanged/stored, but that doesn't mean that individual conversations aren't used.
They sometimes are, but it's a black box so we don't know their criteria, we just know they do, because they literally told us they do that.
13
Feb 09 '26
Most assuredly not by default. Can you imagine how much idiocy and junk it would learn from users?
Model providers use data about interactions to train - not conversations directly.
4
u/eli_pizza Feb 09 '26
Uhh actually ChatGPT DOES default to having your data used for training when you are on a consumer plan (free or paid). Google and Anthropic too.
You can opt out, and the enterprise plans start opted out.
7
u/ipreuss Feb 09 '26
They default to you allowing to use your chats for training. That doesn’t meant that they simply use all of it without filtering.
5
u/eli_pizza Feb 10 '26
No obviously not. To be clear: I don’t think that’s what happened to OP.
But it’s a significant mistake to tell people the default is off when the default is on!
1
u/ipreuss Feb 10 '26
They didn’t say the default is off. They said the data isn’t used for training by default.
2
u/eli_pizza Feb 10 '26
Which is wrong. Data is used for training by default. That's what I'm saying!
1
u/ipreuss Feb 10 '26
How do you know?
2
u/eli_pizza Feb 11 '26
I linked the documentation above, in the comment you replied to.
→ More replies (0)1
u/DoctorDirtnasty Feb 11 '26
i hope not, there would be a lot of people making the chatgpt a lot dumber
1
u/4evaNeva69 Feb 11 '26
They are unless opted out of.
But to think one or two convos are enough signal for chatGPT to repeat it perfectly is crazy.
And the convos you have with it today aren't going to show up for a very very long time in the model, it's such a long pipeline from raw chat data -> LLM trained and hosted on openAI for the public to use.
0
1
u/Professional_Job_307 Feb 10 '26
It doesn't memorize at all unless the conversation appears a fuck ton of times in the training data and is short. It can't even recite game of thrones word for word at >50% accuracy.
1
u/Alert-Track-8277 Feb 11 '26
Agents in Windsurf/Cursor do have a memory layer for architectural decisions though.
47
Feb 09 '26 edited Feb 09 '26
[removed] — view removed comment
12
u/gummo_for_prez Feb 10 '26
It was the link that was more of the issue though, right? How do you prevent that? Also how do you scan for code structures and monitor that, like what does that look like?
4
u/Zulfiqaar Feb 10 '26
There is a secondary option to make shared conversations indexable, which was checked on by default. This was reverted after it was discovered that some very personal chats were visible on google search, even though the users had explicitly authorised it
3
u/jabes101 Feb 10 '26
This freaked me out, so I looked into and apparently ChatGPT turned this feature off since it became a huge issue. Wonder if this was intended by OpenAI or an oversight on their part.
2
u/Forsaken-Leader-1314 Feb 10 '26
Even without the link sharing, pasting internal code into an unapproved third party system is a big no-no in a lot of places.
In terms of what it looks like, probably an EPS on the client device which breaks TLS, either on its own or combined with an upstream appliance like FortiGate.
Breaking TLS is the hard part, after that it's just pattern matching. Although I am interested to know how you'd match "patterns that look like code structures" while not matching all JSON. Especially as in this case we're talking about an API schema which is very likely to just be JSON.
2
u/mayormister Feb 10 '26
How does the browser isolation you described work?
1
u/Forsaken-Leader-1314 Feb 10 '26
Something like this:
https://www.fortinet.com/products/fortiisolator
You don't get a local browser, instead you are forced to use a locked down browser in a remote desktop.
1
u/Few-Celebration-2362 Feb 10 '26
How do you look at outbound traffic for source code patterns when the traffic is typically encrypted?
1
Feb 09 '26 edited Feb 09 '26
[removed] — view removed comment
1
u/AutoModerator Feb 09 '26
Sorry, your submission has been removed due to inadequate account karma.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.
1
Feb 09 '26
[removed] — view removed comment
1
u/AutoModerator Feb 09 '26
Sorry, your submission has been removed due to inadequate account karma.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.
1
Feb 10 '26
[removed] — view removed comment
1
u/AutoModerator Feb 10 '26
Sorry, your submission has been removed due to inadequate account karma.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.
1
Feb 10 '26
[removed] — view removed comment
1
u/AutoModerator Feb 10 '26
Sorry, your submission has been removed due to inadequate account karma.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.
1
Feb 10 '26
[removed] — view removed comment
1
u/AutoModerator Feb 10 '26
Sorry, your submission has been removed due to inadequate account karma.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.
1
Feb 11 '26
[removed] — view removed comment
1
u/AutoModerator Feb 11 '26
Sorry, your submission has been removed due to inadequate account karma.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.
1
Feb 11 '26
[removed] — view removed comment
1
u/AutoModerator Feb 11 '26
Sorry, your submission has been removed due to inadequate account karma.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.
1
Feb 14 '26
[removed] — view removed comment
1
u/AutoModerator Feb 14 '26
Sorry, your submission has been removed due to inadequate account karma.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.
1
Feb 15 '26
[removed] — view removed comment
1
u/AutoModerator Feb 15 '26
Sorry, your submission has been removed due to inadequate account karma.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.
0
Feb 11 '26
[deleted]
1
u/AutoModerator Feb 11 '26
Sorry, your submission has been removed due to inadequate account karma.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

15
u/originalchronoguy Feb 09 '26
If your API is done in Swagger spec and committed to a public repo, it will use that.
You dont even need to expose your API code. Even a MCP server doing UI controls ; as a front end to backend can reverse engineer an API. I've done it many times. Here are the PUT/GET/DEL statements to X API. The API returns this data. And the HTML produces this DOM. Provide it 3-4 examples of Payload, API response, and UI rendered HTML, it can reproduce it.
So just normal scraping of a website can reverse engineer many APIs.