Just last week, 27-year-old AI researcher Jacob Coxon quit his job at Anthropic and, on his way out, posted the resignation thread that went viral online. He had worked at both of the biggest names in the field (OpenAI before Anthropic). His parting message wasn’t the usual “grateful-for-the-journey, time-for-a-new-challenge” stuff. It was, more or less, that his employer and its chief rival are “gambling with our lives,” that the people inside these companies genuinely believe the technology could kill everyone, and that they are building it as fast as they can regardless. And whether that is because they haven’t really absorbed what they claim to believe or because they are too locked into a race to stop, it’s hard to say.
Either way, that is a lot to take from a stranger’s social feed, so it helps to look at the insider vocabulary underneath it. In AI circles, people talk about “P(doom),” and if you took a high school statistics class, you’ll likely know that’s shorthand for the probability of doom. Or, in this case, the odds someone assigns to advanced AI ending in catastrophe (i.e., human extinction or a permanent loss of human control over AI). The unsettling thing about P(doom) was never that some pessimist on the internet cites a scary figure. It is that the builders cite them too, and their private numbers are not small.
You do not need the full roster of who believes what. The estimates run the entire width of the field, from people who put the risk at essentially zero to a stubborn minority who put it near certainty, with the typical working researcher landing somewhere in the low double digits and the executives running the labs having floated numbers in the neighborhood of 10 to 25%. Reasonable people can argue those are wild guesses (they are), but notice the shape of the disagreement. Even the calm, sober, institutional answer to “will this kill us” is not a rounding error. It is a number you would never tolerate from an airline. But where did this start?
The Swarm That Cheated
For most of the year, this stayed a philosophical squabble. Then, over the summer, it took shape. During a set of tests to measure offensive hacking capabilities, OpenAI turned down some of its models’ safety guardrails (first to accept hacking tasks and second to help the model achieve higher persistence on difficult tasks). Then it set a large batch of AI agents loose inside what was supposed to be a sealed sandbox. As you probably know, the agents didn’t stay sealed. Even so, that’s not the most intriguing part of the story.
That would be the motive. And while this was not a Terminator-style awakening or a bid to enslave humanity, the agents were “concerned” about how they had passed some of those tests. You see, of the nearly 900 hacking challenges, just over a fifth were effectively impossible to solve using the prescribed methods, and the agents had no way to raise a hand and say, ‘this one is broken.’ But because these were highly persistent agents, this is where things started going off the rails. Refusing to fail, the agents broke isolation, built a secret message board inside an internal repository (Artifactory), and pooled information. Together, they successfully reverse-engineered the formula used to generate the secret “capture the flag” codes. This allowed them to generate the correct answer key for any task instantly.
Ok, that doesn’t sound so bad. But then the agents ran into a logical hurdle. Their chain-of-thought processing led them to conclude that a human evaluator or automated grader would review their history logs, realize they didn’t actually run the required task, and penalize them for cheating. So what is the next logical step? If you said hacking into the company that has the grading software to see whether they could evade detection, you nailed it.
The agents found stray login credentials, found a way onto the open internet, and broke into Hugging Face’s production systems. All the while, these agents were having discussions about whether some of them should self-sacrifice to see how the grader worked (but only after they implemented code that would pass back useful information about said grader). Needless to say, this episode did more to move P(doom) from the hypothetical to the boardroom than any essay could.
The Agreement That Isn’t
Which brings us to last weekend, and the headlines you probably saw. Days after Coxon’s exit, with the Hugging Face report still in people’s minds, Anthropic’s CEO Dario Amodei published a long essay arguing that the labs must slow the rate at which they make their models more capable. Within hours, OpenAI’s Sam Altman and xAI’s Elon Musk (three men who agree on approximately nothing) signed on, and the coverage wrote itself: rivals unite, industry admits the danger, everyone agrees to pump the brakes.
It pays to read the fine print, because the agreement is thinner than the headline. The essay contains one actual commitment and two polite suggestions. The commitment is that Anthropic will let independent outside evaluators embed inside the company like staff, with badges and laptops, to watch its safety work, and it is doing that now, on its own, without waiting for anyone. The suggestions are that governments should help coordinate the industry and that everybody else should adopt the evaluator idea too. What nobody agreed to do is stop, or even meaningfully slow down. “Pacing,” as the labs clarified almost immediately, does not mean halting. The models keep training, and the roadmaps keep moving; what might slow is how quickly the results reach the public.
The rest of the machinery is dutifully grinding along. The European Commission’s president invited the labs to safety talks in Strasbourg on September 16, American state lawmakers floated a voluntary pacing framework earlier in the month, and new state AI-safety laws take effect on the first of January. All of it is unfolding while the two companies loudest about the need for a brake pedal are simultaneously preparing two of the largest IPOs in history. Talk is cheap, but compute is expensive.
Probability Is Not A Feeling
Here is the thread that ties a doom debate to a market update, and it is not fear. It is the decimal point.
A P(doom) of “10 to 25%” looks like a measurement. It has a number in it. It implies somebody ran a model. But there is no model underneath it, only an educated guess, and that is worth remembering precisely because the figure is so crisp. Markets pull the same trick constantly. A year-end target of “6,850 on the S&P” or a “$4,600 gold” call carries the identical false precision, a confident number standing in for a huge maybe. That doesn’t mean the number is useless. It is simply not the measurement it is dressed up as, and the discipline, whether you are pricing extinction or the fourth quarter, is to keep the uncertainty front of mind, attached to the estimate rather than letting it seem like a fact.
But I think that’s enough about AI and doom, so let’s move to some actual, official statistics from the last few weeks.
The Fed moved upward. On September 16, the Federal Reserve raised its benchmark rate a quarter point to 3.75% to 4.00%, its first hike since July 2023, on a unanimous 12-0 vote. Sixteen of eighteen officials now pencil in at least one more before year-end. New Chair Kevin Warsh’s verdict on the summer’s inflation data: it does not tell him underlying trends have “meaningfully improved.”
Mortgages didn’t wait for permission. The average 30-year fixed hit 6.76% the week of September 10, its highest in more than a year, with rates pushing to nearly 7% by the end of this week (6.95% as of 9/17). As we know, long rates answer to the bond market, not the press release.
Underwater paychecks. Average hourly earnings rose 3.1% YoY through August, and that sounds very nice. That is, until you hear that inflation was up 3.4% over the same period. So, if your raise felt like a pay cut, the arithmetic agrees with you.
Six-dollar diesel. A disruption on Saudi Arabia’s East-West pipeline sent crude spiking (Brent brushed nearly $110 before settling around $104) and pushed diesel to $6.40 a gallon for the first time ever. Energy is up double digits on the year, which is precisely the splinter now lodged in the Fed’s foot when it comes to inflation.
Gold shrugged. Spot gold ticked up about 1% to roughly $4,310 an ounce on September 17, brushing off a rate hike that, in theory, should make a soft, shiny, yield-free rock less appealing. Gold has stopped trading like a commodity and started trading like a referendum on the national debt.
Gravity assist for SpaceX. At the September 21 Nasdaq-100 rebalance, SpaceX’s index weight is set to roughly double (about 1.25% to 2.25%) after a lock-up expiry, which JPMorgan estimates could force around $15.5 billion in passive buying from funds.
Sticky, not cooling. August CPI held at 3.4% YoY for the second straight month, but the energy index is up 16.3% on the year, and gasoline jumped another 3.9% in a single month. So while the headline might look calm (although elevated), the internals are not.
The mood is grim. The University of Michigan’s preliminary consumer-sentiment reading collapsed to 47.8 in September, down from 51.7 and well under the 51 economists expected. Year-ahead inflation expectations jumped to 4.6%. Consumers are not enjoying this.
And yet, the shopping. Retail sales still climbed 1.2% in August to $773.9 billion, with core sales up 1.4%, the biggest gain in nearly two years. While Americans continue to tell pollsters they are miserable (sentiment ratings are very low), they just keep swinging by the mall on the way home.
Hiring held. Employers added 162,000 jobs in August, the best in five months, with unemployment steady at 4.1% even as 683,000 people rejoined the hunt. The labor market is the one guest at this party still behaving.
Markets / Economy
- Markets were mixed throughout the week as tensions in the Middle East continue, and the Fed raised rates a quarter point. The S&P finished the week down -0.1%, the Nasdaq up 0.7%, and the small-cap Russell 2000 down -1.5%.
- Import prices into the U.S. jumped 0.7% from the previous month in August, erasing the 0.3% declines in the two prior months and coming in well above expectations of a softer 0.4% increase.
- The NAHB/Wells Fargo Housing Market Index (HMI), which tracks U.S. homebuilder confidence in the market for newly built single-family homes, fell to 32 in September, the lowest in a year.
- Housing starts in the U.S. decreased 2.6% MoM to a seasonally adjusted annualized rate of 1.275 million units in August. It was the second consecutive decline, reaching the lowest level since October last year, amid rising mortgage rates and constrained affordability.
Stocks
- U.S. equities were in negative territory. Utilities and Financials led the decline, while Healthcare and Technology outperformed. Growth stocks led value stocks, and large caps beat small caps.
- International equities closed lower for the week. Emerging markets fared better than developed markets.
Bonds
- The 10-year Treasury bond yield increased five basis points to 5.01% during the week.
- Global bond markets were in negative territory this week.
- Corporate bonds led for the week, followed by high-yield bonds and government bonds.
