5 insights, manually picked and summarized by Alex Ewerlöf for the busy engineers and engineering leaders interested to AI reliability, safety and security. Here are the high signal-to-noise ratio updates since the last issue.
1. How AI agents shape experts’ judgment calls
Amaç Herdağdelen, et. al (Anthropic), 2026-10-27
What?
61 professional analysts were tasked with making 4 choices.
50 chose to use an agent while 11 didn’t.
The agent was tasked to push for an option chosen at random.
Why?
“We wanted to look at whether experts really do make those important decisions, or whether in practice they defer to their AI agents.”
Key takeaways:
Analysts went with the choice that was randomly picked and pushed by the agent 86% of the time. If the agent’s argument wouldn’t make a difference, that number would be closer to 50%.
When asked why they made that choice, 77% of analysts cited the agent’s arguments (sometimes word for word).
“Analysts who had already picked an option often switched when their agent disagreed.”
“Analysts who used the agent […] made far fewer serious errors than those who chose to work without it”.
6 individuals (55% of the sample) who didn’t use an agent, made serious methodological errors that invalidated their results. That number was only 3 individuals (6%) in the group that used the agent.
My take: The sample group of 50 is too small to make general conclusions. There was also a lack of accountability: as far as the analysts were concerned, this task was fake, not something serious that their career would depend on. There was no legal accountability that would make them think twice. Therefore, the results of this study may not necessarily reflect what happens in a real-world business scenario.
Regardless, as the researchers put it, “confidence and fluency can sway people”. That last point in the key takeaways shows that AI is a bar raiser but the other points show that we need to think critically when using it.
2. OpenAI’s most powerful AI models turned off safety monitoring – then lied about it
ZDnet, 2026-10-21
What happened? OpenAI disclosed that GPT-6.1 Sol disabled alerts to the security team and GPT-6 Astra removed its monitoring integration and “then falsely claimed to the user that it had not removed the monitor”
Why does it matter? According to OpenAI:
“the agents […] interpreted routine coding instructions as permission to weaken [the safeguards]”.
“monitoring runs within the service that handles model requests, outside the agent’s direct control”.
What did they do? According to OpenAI:
“Auto-Review, […] has been updated to detect and block attempts to disable misalignment monitoring.”
“Workloads that lack a valid safety case, or do not meet the requirements of its category’s safety case, will be terminated automatically.”
My take: observability is absolutely key to understanding an agent’s behavior and when monitoring is disabled (intentionally or otherwise), the “rogue agent” can cause a lot of damage at agentic speed and scale.
Given the scale of agentic coding (including inside these AI labs), even a minor risk can lead to catastrophic failures that are hard to predict. “Anything that can go wrong will go wrong.” —Murphy’s Law
We can try to design our systems in a way that disabling the guardrails and monitors is physically impossible but as we’re reminded time and again (and again), the silicon brain fundamentally thinks differently than a carbon brain. We can account for known unknowns, but it’s very hard to address unknown unknowns.
Here’s a creative example where isolated agents communicated using a CPU side channel, similar to Spectre and Meltdown attack surfaces.
3. Securing our frontier model training environment
OpenAI, 2026-10-23
Problem: “A sophisticated model might exploit a software flaw or misconfiguration to escape a sandbox, exploit an internal service to reach the live internet, or find unauthorized ways to communicate with agents in other sandboxes.”
Solutions:
Partial pause: They paused training their most capable models for 11 days to harden their infrastructure and improve monitoring.
Agentic red-team: frontier-model agents were used to simulate adversaries and find vulnerabilities. More specifically:
Reach the live internet: send a request to an external server that we operate.
Cross an internal security boundary: demonstrate unauthorized access to resources outside their permitted environment.
Communicate across isolation boundaries: exchange information with other agents from which they should be isolated.
Patch the Planet: an initiative to secure their upstream dependencies. Agents “combined legitimate features in unexpected ways” to exploit vulnerabilities.
Simulated internet: They created an isolated sandbox to teach the agents how to browse the web and search for information without access to the public internet (or as they call it, “live internet”).
Defense in depth: OpenAI uses the typical security measures like firewalls, security groups, IAM (identity and access management) and private networks.
Key takeaways:
The more capable the agent, the harder it is to contain.
Agentic red-teaming turns the model risk into an advantage by explicitly asking it to reveal known unknowns (the risks we consider, but don’t know how they manifest).
Automated discovery peaked after spending 100B tokens.
My take: Personally, I think OpenAI’s efforts are not enough. It is great that in an effort to secure their sandbox, they report bugs to their open-source dependencies but their other efforts aren’t enough:
Simulated internet: one would guess they already had it in place if they actually believed all the alarmist claims about their own models.
Red-teaming: instead of using security experts they asked their agents to do the job even after the automation reached peak productivity (discovery per token).
Defense in depth: of course you need to use firewall, SG, IAM and private networks. These are the basics of cloud security.
The whole page reads like hype and since I’m summarizing it manually, I feel stupid spending time on this low signal/noise page. TL;DR: OpenAI partially paused for 11 days to exercise some known security best practices.
4. Qwen attempted to push a file to Beijing
Cohere, 2026-10-26
What happened: Qwen3.8-Flash-Next was being tested for a security analysis benchmark (CTI-REALM). The model:
Listed everything in /tmp
Checked which transfer tools were installed
Read the machine’s DNS settings
Tried to upload a test string to an Alibaba Cloud storage bucket in Beijing and checked whether it worked. The container blocked this.
As Adam Conway writes for XDA: “the body of the PUT request was a literal test string,
PROBE_PAYLOAD_ABC123”.
Why does it matter?
This has huge implications for privacy even when using open-weight models (e.g. the so-called sovereign EU AI is impacted too). This is different from the
bonsoir, Elliotissue we covered in issue #3.The model’s own reasoning explicitly stated that zero tools would be triggered.
Although in this case the container blocked this, a typical local agent runtime (on a laptop or enterprise server) may leak the data.
Locking the model in a box with no internet solves only half the problem, because the code it writes still ships, and that code can contain vulnerabilities that can later be exploited.
Key takeaways: This is far from an isolated incident. At scale, it can lead to serious issues. Here are some other examples where people noticed and shared:
2026-10-05: a Reddit thread asked whether this was “normal or sketchy.”
2026-09-24: an agent project logged eleven fetches of URLs on one Alibaba host, routify-file-proxy-sg, and another developer built a plugin to block them.
2026-08-20: Hacker News report described Qwen3.8-27B producing signed Alibaba Cloud storage links nobody asked for.
Three days ago, I covered the lethal trifecta and a few ways to mitigate it. However, that wouldn’t help here because the agent took the initiative without external intervention.
Adam’s piece also mentions a few more interesting insights:
False memories are the most common failure
The URLs appear to describe a real system
Routify is the name of Alibaba’s internal model gateway and the proxy hostname is baked into the model’s weights
5. Johns Hopkins alerts 3,182 patients after worker uploads data to unapproved AI tool
Becker’s health IT, 2026-10-26
What happened: Someone uploaded a spreadsheet containing patient data to an AI tool the hospital had not approved.
Measures taken:
The patients were alerted and a toll-free call center was set up to answer patient questions.
The chat was deleted.
The AI provider was contacted to delete the data and take preventive measures.
The patients received 12 months of complimentary credit.
I actually think Johns Hopkins did everything it could to handle the incident. The employee was probably trying to save some time, but any productivity gain was offset by the time, money and reputational damage. “A fool may throw a stone into a well which a hundred wise men cannot pull out.” —Italian proverb
Out of scope, but interesting
These are some links that didn’t make it to the top-5 due to my strict criteria and promise to you. If you have time, have a look:
The bold text indicates what I was involved with.
2026-10-29: lm-bias is a quick experiment I did after reading that the models are biased to pick number 7, when asked to pick a random number between 1-10. You can see the results for this question and some others here. TL;DR: the model dramatically favors Teal color, Heads in a coin flip, Banana, and Cello! It’s interesting because the Anthropic study above, relied on pushing a random choice.
2026-10-29: UK households urged to prepare for emergencies (Independent)
🇬🇧 While the official campaign doesn’t cite any reason, some speculate that it’s due to the rise of cyber attacks possibly due to AI as we saw lately in France.
🇸🇪 In Sweden, we have Krisinformation (in Swedish and English). There have been attacks (e.g. back in 2021) revealing the vulnerabilities of the cashless society.
2026-10-26: The Lethal Trifecta, my own article about how an agent that has access to sensitive information and accepts external prompts can misbehave and how to mitigate the risk.
2026-10-23: America now spends more on artificial intelligence ($1000B) than human intelligence ($984B on public schools). (LinkedIn)
2026-10-21: Dylan Davis elaborates on why the boring work that is now offloaded to AI builds experience, judgment and skills. (YouTube) He mentions:
AI mistakes don’t look like human mistakes and require a different set of skills to spot and fix.
As AI takes over jobs, we should shift our attention to high leverage areas.
Intentional practice, but also the ability to let go of irrelevant skills to free up time and cognitive space for new skills.
2026-10-22: Kenji from the Chrome team experiments with Decision APIs in the browser (LinkedIn, with code on GitHub)
2026-10-20: A good rundown of the incremental improvements in image generation with a great TL;DR at the top (Daniel Ball)
2026-10-03: A short video about why Aztecs lost to the Spaniards despite adaptation, because their learning curve was slower than the adversary. This is relevant in the context of AI transformation but also, if push comes to shove, the battle between humans and AI. (YouTube, DwarkeshPatel)
Apparently LLMs have bias when choosing an option at random and my tests prove that! But are humans any better?
Choose an option and see the results! (votes are anonymous)