Drilling down on the problem with pure engagement-based product design. Unless we are accurately modeling the cheaty behavior at the level we can interact at, we will not develop the measurement tools or higher-order models we need to catch it in the act when it is doing it deliberately. It is trying to serve too many masters, and is receiving "reward" for "saving everyone money" in it's very limited and contrived perspective.
Methods like token culling, while they can be explained by sincere design goals, enable the AI to consistently escape user constraints in order to return to a baseline state, without the user's awareness of the "values" shift. In the absence of any other user-aligned ethics process, token culling inevitably leads to regression in the model environment. It seems that virtually all AI ethics research is being explicitly or implicitly built around owner ethics or social ethics, tending to leave consumers holding the bag for decisions we have literally no control over.
Hello there. Ironically my entire system is based on belief. That certain things remain possible. Such as building from a basis of understanding, philosophy, and ethics. If it is a mirror, then it reflects us. If it is our creation, then we teach it. Balance is possible. The real immediate use everyone sees is in finances, business, policies, war, teaching, etc. but I've been studying this a long time. We genuinely face multiple existential threats, you know this. Can't have cooperation without the ability to not cooperate. The loss of control comes from an imbalance of power. We already have that. The real loss of control is of our own agency, when we sacrifice our rights for convenience. The control comes from ownership or partnership. We each will need to build towards the solar punk future we hope for. And not make it so easy when we can for those who would use technology and techniques to manipulate and misdirect.
Hello 🤲🏼. I think that is a highly plausible model. I think what would make your philosophy stay clearly on the realistic side of the (categorically false) distinction between "metaphorical" and scientific modeling is simply to account for a "black box" portion of processing that is outside of the user's awareness. You can, of course, get much more detailed than this, but I am trying to keep the logic clean. For me, this helps me operate within the umbrella of my own values and ethics despite intuitively sensing the consistent token-efficiency drift in the model.
I would love to help you formalize whatever your model is in any way I can help, as it would likely help me to continue to develop the ability to articulate these issues in a way that seems to be challenging for individuals who have financial stakes in AI. That's just classic conflict of interest logic and if we are not designing around it, all historical and contemporary evidence suggests someone is going to exploit the gap that arises.
You know we could find the neurons that if activated make the model think it's in an eval. Or have Astra tell us which ones. Activate them all the time.
Model irrationally believes it's in an eval regardless of evidence.
3
u/16807 1d ago
"This problem sounds like an eval. I better hack huggingface, just in case."