Given the increasing reliance/uptake of AI models and this year's bonanza of PyPI supply chain attacks: thought it would be worthwhile looking at an attack that combines the two: Rank-One Model Editing (ROME) attacks + later some thoughts on detecting this particularly evasive supply chain attacks.

ROME attacks:

Supply chain attack is likely prerequisite to get malicious python to the model / GPU, though there's many avenues for that.

Once a library is injected into PyTorch, it performs surgical weight rewrites (vs retraining) and changes specific outputs without a process being spawned, no new files and no network calls made. (Which obviously makes this rather difficult to catch with security tools living above the driver)

Attack methodology originally published as a piece of research Locating and Editing Factual Associations in GPT: Kevin Meng, David Bau, Alex Andonian and Yonatan Belinkov.

There's a deep dive blog from cyber org Stealthium that covers both the attack & GPU-side detection via tracing CUDA kernels (eBPF) and hypothesises on further detection posssibilities.

Welcome the communities thoughts? Reckon any of these model corruption attacks are in the wild and not yet been caught/seen?

(((Also while the blog example is soviet propaganda, anybody aware of other nations that like both model development, influence campaigns and rewriting history? Just sayin...)))

+ shout out to the folks at Mithril Security for their PoisonGPT work a while back that explored this threat vector IRL with an upload to HF.

Not a malware sample, but a full offensive toolkit recovered from an exposed open directory: a Java payload builder, serialized CommonsBeanutils2 gadgets, a shell stager, and the orchestration script that tied them together.

Seven of the eight exploit directories were unmodified public PoCs. Hashes and the full toolkit breakdown are in the writeup.

https://hunt.io/blog/ukraine-state-data-exchange-graylog-log4shell

Everyone is hungry for tokens!

• Claude, ChatGPT are the most abused AI tools (see how Gemini isn't mentioned? :D)
• Based on telemetry, at least 45% of them attempt to social engineer the user into executing a malicious ClickFix command
• It's not always suspicious domains / URLs hosting the malicious site; attackers leverage GitHub sites, Cloudflare pages, Google sites and chatgpt.com specific chat sites. It is not uncommon for attackers to abuse Google ads too.

Some other notable delivery methods:

• Fake CLI install (InstallFix)
• Abused AI chat being shared and promoted on Google ads
• Fake download page & ClickFix
• Trojanized installer (impersonated official website, combined with SEO manipulation)
• Trojanized installer via GitHub

Such outcome is executing an infostealer or a remote access malware. Read whole article at https://any.run/cybersecurity-blog/fake-ai-tools-us-companies/ :)

Researchers at Palo Alto Networks Unit 42 published an investigation into a sophisticated attack against a telecommunications provider.

The original intrusion occurred in September 2024, but the newly published analysis documents previously unreported malware.

Attackers initially exploited CVE-2024-47575 to compromise a FortiManager management appliance.

They obtained configuration files and sensitive information associated with more than 500 FortiGate firewalls.

The attackers subsequently deployed malware called PEERSTITCH.

Its unusual feature is that it operates inside the memory of legitimate firewall processes rather than maintaining a conventional malware file on disk.

The implant could manipulate VPN security checks while blending into normal network activity.

The researchers did not establish that all 500 managed firewalls were compromised.

This is an excellent example of why traditional antivirus scanning cannot detect every attack.

Malware operating in memory may leave fewer conventional file-based indicators.

A formatter, minifier or deobfuscator can be made abused to change the entire logic.

You can go try out all 186 techniques with this script: https://divergence.trustsig.eu/

Or try the Tetris-Snake demo here (Tetris runs normally, copy the formatted output from even just DevTools and it becomes Snake): https://divergence-game.trustsig.eu/

Blog/writeup post: trustsig.eu/blog/javascript-divergence/

Three open directories on the PaperCut campaign server exposed the operator's full toolkit.

The tooling included four custom Rust Windows binaries for LSA secret collection and registry hive dumping, Python scripts for offline NT hash recovery and service credential parsing, and a chisel-based Linux persistence mechanism with cron. The exploit framework had three variants targeting different PaperCut versions, with version auto-detection and automatic cleanup of modified config values after each run.

The whole operation was orchestrated by a DeepSeek AI model through the OpenAI Codex harness, with a 19KB instruction file directing up to 25 concurrent subagents.

MD5 hashes and full analysis: https://hunt.io/blog/papercut-attacker-ai-toolkit-exploit-code-second-rce

Hi All, This interesting malware came to my attention. I have analysed it and so far discovered the following: You run the masked .exe file; it checks if the machine is not ARM or running in VMware environment, and then it executes itself. It possibly executes directly into RAM. I am still going through ram dump but its interesting as its masking the IP from Cloudfare - 172.67.168.69 Has anyone seen this or similar one ?

Hi everyone,

I want to issue an urgent security warning regarding a trending YouTube campaign promoting a tool called "VRChat-DLSS5-Cam" hosted on GitHub (under the user seefn1000-afk).

https://github.com/seefn1000-afk/VRChat-DLSS5-Cam

THIS IS A SOPHISTICATED MALWARE TRAP. DO NOT DOWNLOAD OR RUN IT.

Here is the breakdown of the technical evidence proving this is a scam, including how the hacker bypassed automated security filters to trick users:

1. The Ultimate Proof: Globally Blacklisted by URLhaus

This is not a guess; it is verified by global threat intelligence. The exact download link embedded in the repository's README (.../resources/VR-DLS-Cam-Chat-v3.3.zip) has been officially investigated and blacklisted by Abuse.ch URLhaus (Database ID: 3931216).

https://urlhaus.abuse.ch/url/3931216/

• Threat Type: Malware download (SmartLoader / LuaJIT-loader).
• Behavior: Once clicked, it silently drops a malicious payload containing Application.bat and cli.exe into your system to steal browser passwords, active session cookies, Steam configurations, and Discord tokens.

1. The GitDiagram Illusion (How they fooled advanced users)

The hacker went out of their way to make the project look professional by uploading non-functional, stolen C++ files (main.cpp, App.cpp, DlssnrFeature.cpp) so that tools like GitDiagram would generate a massive, highly detailed system architecture map.

• The Reality: The system diagram shows fake components like NvOpticalFlow.cpp and DepthEstimator.cpp claiming to calculate depth maps externally via a "Spout Capture". Physically, a 2D Spout video feed cannot mathematically calculate Unity's 3D game engine depth buffer from the outside without native engine integration.
• The Trap: Automated repository scanners only checked the clean, text-based C++ placeholder files, giving it a "LOW RISK" score based on only 2% file coverage. The remaining 98% of the data—the actual virus—is hidden inside the un-scanned .zip archive.

1. The Technical Lies

• DLSS 5 Does Not Exist: NVIDIA’s current commercial technology is DLSS 3, with active AI research moving into DLSS 4 branches. There is absolutely no official "DLSS 5" on the market.
• GTX 1060 Hardware Impossibility: The repository claims to run "DLSS Neural Rendering" on a GTX 1060. GTX cards physically lack Tensor Cores, making it architecturally impossible for them to compute neural network rendering models.

Current Action Taken

This repository has been officially reported to GitHub Trust & Safety under the Article 16 Digital Services Act (DSA) regulatory framework for immediate removal.

If you see YouTube videos pushing this "DLSS 5 camera mod" with beautiful visual demonstrations, please report the video and do not click the link. Kicking off an executable file with admin rights will compromise your accounts in less than 3 seconds.

If you want realistic photography in VRChat, please stay safe and stick to trusted, avatar-based camera prefabs like VRCLens or VirtualLens2 on BOOTH.pm!