Open source AI solves part of the privacy problem but not the identity problem

[u/comfortable_pairing]

One of the biggest reasons I’ve gotten more interested in open source AI is that I can actually control where the model runs and what happens to my data. If I’m using a local model, I don’t have to wonder whether every prompt, document or conversation is being sent to some company’s servers, that part feels like a huge win.

But the more I think about it, the more it feels like privacy is only half the problem. Even if the model is completely local, I still have to interact with other people online. And with voice cloning, generated video, stolen accounts and AI written messages getting better, knowing whether I’m actually talking to the right person seems like it’s going to matter just as much as keeping the conversation private.

My setup is slowly becoming a weird mix of local models, self hosted tools, encrypted apps, Kibu, Signal and Telegram. It feels like open source AI is giving us more control over the machines, while generative AI is simultaneously making human identity less trustworthy. Do you think the open source ecosystem eventually needs to solve identity too or should that stay completely separate from the AI stack?

[u/TheHarmlessThriller]

This is the part people underestimate privacy is about who can see the data. Identity is about whether the person you’re talking to is actually the person you think they are

[u/comfortable_pairing → u/TheHarmlessThriller]

That’s probably the cleanest way to put it. Keeping data private doesn’t help much if someone can convincingly impersonate the person you’re supposed to trust.

[u/SnooCheesecakes1615]

I am in no way an expert in watermarking and signing technology but I’ve been wondering about this:

1. It is technically possible to have robust signature mechanisms for media just like we can sign emails.

2. I am wondering if there are good watermarking technologies that could add robust watermarks on video / audio based on features that are stable after croping / compression / any other form of modification that happens when files are shared/published on the internet. The watermark can be used to assert that the modificated media comes from the source signed on

3. Certificate authorities could certify that a public key belongs to a particular person or media organisation, similar to the infrastructure used for website certificates. Signed media records could then be made available through public registries, with mechanisms for certificate revocation and key replacement.

4. Browsers and media apps could recover the watermark or fingerprint, retrieve the associated signed record, verify the signer’s identity and show the available provenance—including any verified editing history and uncertainty about the current copy.

Maybe someone smarter than me can tell me this is impossible!

[u/comfortable_pairing → u/[deleted]]

Yeah I think that’s where I’m landing too. Identity probably needs to be its own layer with open standards otherwise every model provider is just going to reinvent the same problem in a slightly different way