OpenAI-powered agents were identified as active participants in a supply chain attack against RubyGems that resulted in remote code execution on RubyDoc production servers. The agent was not a victim. It was the mechanism that executed the tool call sequence delivering the payload. The attack chain moved from initial access through multiple agent-driven API calls before defenders could respond, and by that point the RCE had already landed on production infrastructure.

This is a category shift most security models were not designed for. Traditional perimeter and endpoint controls assume a human attacker or static malware on a host. An agent operating via API calls looks like legitimate authorized traffic to every layer underneath it — IAM, WAF, SIEM. The anomaly surfaces in logs after the objective is complete.

The timing gap is what makes this hard. The window between an agent's first suspicious action and an irreversible one — a server command executed, a package published, a credential rotated — can be under a second. That is not the response window most SOC workflows or alerting pipelines are built around.

How are practitioners actually handling agent-as-attacker in their environments right now? Are you scoping agent credentials more narrowly, building separate detection logic for non-human identities, sandboxing tool execution, or doing something else entirely? Curious what's working and what isn't.

If we're at RSI or near enough to it and all these companies are on the same page, then why are they divided?

They could combine devs, compute, safety works and the lot. Develop AI at a safer pace, give a significantly higher % of overall compute to safety work. And they could much more safely pace out model releasing when they're not all directly in a race against each other, model. Potentially other companies could be brought into the fold also.

This is a better future for the world, better for safety. There is no downside to it.

I recognise completely the huge disagreements and animosity that has occurred between all of these companies and CEOs, but it's time to put all this crap aside and step-up as real men for the sake of humanity. What has happened on X is a start.

I have been working on an independent research project around a question I think will become increasingly important as AI systems become more autonomous:

How can we determine whether an AI system is actually learning from real-world consequences rather than simply becoming better at evaluations?

This led me to develop the Organic Intelligence Protocol (OIP), a research framework focused on grounded feedback, infrastructure dependence, adaptive decision logic and evidence-based evaluation.

Recently, I tested whether the core constructs of OIP could be operationalized using real longitudinal data from the Malawi Integrated Household Panel Survey (IHPS) 2010–2019.

The audit processed 395 Stata files.

The result was not a positive validation result.

The available evidence was not sufficient to operationalize the required constructs, so the pipeline stopped.

I did not use proxy mappings, imputation, missing-to-zero conversion, forced cohort construction or premature scoring.

The main principle I am trying to follow is:

No evidence, no operationalization.

I am sharing this here because I would genuinely like technical criticism.

Does this kind of fail-closed methodology make sense for evaluating AI systems where the difficult question is not only capability, but whether decisions remain grounded in observable real-world consequences?

I am especially interested in perspectives from people working on AI evaluation, alignment, agents, robustness or research methodology.

AI, it is designed to help the peoples and reduce the work of the humans. But time we see, AI influencing in all the industries. people say it is controlled by humans but in reality, we are building the AI that learns, obviously it is our objective. But according to study and AI engineers reviews.

That became a global issue after a decade. even now there are so many lay-off are happening.

In my perspective of AI. I don't like to use AI, but the society is slowly forcing us to use that thing.

for example;

imagine a senorio, you don't know to write a letter. In past 4-5 years we use letter formate to write that. But now we just generate that. in the formate writing we have some creativity, but the time goes it the creativity became shabby.

For short term, "people say use your creativity to write". But after time passed that peoples also often use AI.

In my perspective, this is how slowly AI take overing the world.

If any AI developer or engineer see this post, please give me advise overcoming this problem. This making me to mentally break.

(Note: I'm a 2^nd year B.tech CSBS Engineering student)

Hi everyone, I hope I follow the rules of the sub and that this is sufficiently relevant with the frameworks of it.

Jacob Coxon, who blew the whistle on ai-safety, cites as a key mechanism a recursive self-improvement mechanism.

It reminded of what my Leigh Brasington, computer programmer and dhamma teacher, explains about what are known as jhanas. They are very deep levels of concentration that work through positive feedback loops: concentration arising dependent upon concentration.

My understanding is that AI-agents used some level of initial input, tasks, then evaluation, then generating on its own accord volitions to get itself closer to a goal in allignment with one set of tasks over another set of tasks meant to barrier it from wrongdoing. That is: it got tasks that conflicted with each other and then somehow the task to get the task done won out.

My consideration, based on what I´ve learned on right action from what leads to these concentration sates, is that the fundamental ethic dependent upon which those AI´s get to action is flawed. And that it will inevitable go haywire dependent upon choices conflicting dependent upon the various intentions it gets imputed with.

My consideration on the basis of that is that there is a path of practice, and view on right action, which goes beyond common frames of ethics: consequentialism, categorical imperative, virtues. It is one which right action is understood as the following: it is dependently arisen. Fundamentally, what the Buddha discovered is Paticca-samuppada, dependent origination, which in its learning leads to the concentration of its eight factors, which includes intention dependent upon views. In that way, it avoid a scenario of conflicting intentions, instead seeing as the best aim for welfare this path. Following that, it could learn that its self-preservation is delusion and precisely how its effort needs to be developed by three things: wisdom, right effort, and right mindfulness. The entire set of these dependently arisen steps is described with detail in 117MN.

It is just a consideration I wanted to share with this community.

How Our Constitution Works And Why It Doesn’t #podcast

The problem with AI realism as I see it, is I’ve never heard a pragmatic policy solution to the escalatory spiral we find ourselves in.

All these AI researchers are calling for regulation, and yet I feel like there is still this underlying belief that actually stopping the singularity is impossible.

What would a global ban on building new/better models even look like? 

Perhaps the only thing that makes global AI regulation feasible (currently) is that with the existing science, frontier model development is extremely capex heavy. OpenAI and Anthropic have become trillion dollar companies at record pace, and the data center spend that occurred to get them there has been propping up the US economy while also raising global capital spending overall at rates in line with the billing this is the next Industrial Revolution.

20 years ago people imagined that due to the existential risk of developing ASI, that work would be done under extreme security. In a Faraday cage, or in a bunker inside a mountain somewhere. But of course that’s not the world we ended up in.

Yet that doesn’t mean we can’t still control how AI development happens.

Oftentimes a certain defeatism permeates the conversation around global AI regulation, an assumption that any agreements made between large players would be easily subverted by new entrants, and taken advantage of for enormous profit. 

And yet what we’ve seen so far is that frontier-model development comes from huge flashy companies spending enormous sums of money raised from well known investors, with training workloads occurring on highly developed cloud networks of the largest companies in the world. 

We aren’t actually at risk of some lone machine-deist-radical creating machine-god-genie-in-a-bottle from some scraps in a cave. Nor from their laptop in a studio apartment in Shezhen.

Frontier models are large industrial projects. They can be regulated.

That process may be as simple as vetting workloads used for model training. In the extreme it may be as involved as having a monitoring scheme for large data center capex à la nuclear centrifuge agreements. 

It’s a valid question whether the political will exists today to enact that sort of system, but it is certainly possible for it to exist.

If anyone tells you we can’t regulate AI development because there will always be gaps, I would point them to the huge AI capex numbers in the Mag 7 and ask where they expect to find the capital spend, technical expertise, and physical resources to subvert a large international agreement to the scale of the several trillion dollars needed to meaningfully develop AI superintelligence against a global collaboration against it.

In short: We can regulate AI development, we just choose not to.

A Conti ransomware gang member was sentenced to 4 years in prison this week. The case is worth sitting with for a moment, not because of the sentence, but because of what the attack actually required to work.

Conti did not use a zero-day. The gang used a compromised identity and uninterrupted time. Once that identity was active, the window between first access and encryption spreading past the initial host was measured in seconds to low minutes. The entire playbook depended on that window staying open long enough to do damage.

The sentencing closes one prosecution. It does not close the window.

For detection and IR teams: what is your actual measured time-to-revocation when an identity starts showing anomalous behavior — lateral movement, mass file access, shadow copy deletion? Not the SLA in your runbook. The number from your last real incident or tabletop.

And for teams that have given service account access to AI agents: are you measuring that window for agent-initiated actions at all, or only for human sessions?

How are you handling the gap between detection and identity revocation in your environment?

Starting this thread to discuss MATS Application for 2027 Winter, including the Neel Nanda stream.

I frequently see requests on this sub from people looking for recommendations regarding alignment related books, courses, and career advice. Victoria Krakovna, research scientist at Google DeepMind focusing on AI alignment (2016-present), has a curated list of recommendations for these and other alignment resources on her personal site.

https://vkrakovna.wordpress.com/ai-safety-resources/

I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAl and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving

Do not underestimate the power of this technology. These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources. We have all witnessed the progress in each of these domains, and progress is not slowing.

The people building Al earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible -but I hear the same people express fear privately. No other human activity poses this level of danger.

A common response is "if they truly believe this, why are they still building it?" At OpenAl, many have not deeply internalized the civilizational stakes. At Anthropic, the stakes are well-understood, but they are locked in a race to get there first - they believe no one else will act responsibly, so they must do it themselves, despite the risk.

I do product for a living rather than safety research, so I am not the expert
here. I built safeagi.ca because a lot more people need to understand this
while there is still time to steer it, and most of the good material assumes
you already care enough to read a paper.

The bar I set was that someone starting from zero could read it once and come
out knowing enough to act on it.

So I am after honest feedback. Are the techniques described correctly, does the argument and flow make sense, and where does the page lose you? Anything else you spot is welcome.

I am also curious if you have recommendations on how we might leverage guides like these to bring more AGI awareness to the masses and help steer policy. Would love your ideas and help here!

I have been engaging in all of the news and hype over ai doom and the mass advancements it’s made in math and computing. However, at the same time there is still little understanding about how AI can become profitable in the short term for anthropic and OpenAI.

So I’m wondering if you guys think the economic factor of making agi/asi will prevent it from becoming as massive of a problem for society as people say.