I ran Anthropic's 335-page IPO filing through an AI-writing detector. Well, well, well…

Description

You're currently following this author! Click to unsubscribe from email alerts.

Senior Correspondent covering technology and culture

Anthropic's mostly human SEC filing, according to Pangram. Pangram/BI

Anthropic's much-awaited S-1 filing landed today, revealing all sorts of interesting things about the company's plans and current finances.

But the biggest question I had wasn't "does Anthropic think that accidentally destroying humanity is a business risk?" (yes) or "how much money are they losing?" (lots) or "does it feel weird to be part of the 50% of employees who don't do the pledge to donate some of their equity for the greater good?" (still wondering — if this is you, let me know).

My question was: Did they use AI to write this thing?

To find out, I ran the whole 335-page S-1 filing through Pangram, a service that checks for signs of AI-writing. Ok, well technically I tried to run it through myself, but ran into some problems with such a massive file. I reached out to Pangram CEO and cofounder Max Spero, and he ran the full file for me.

(I also reached out to Anthropic for comment about the tool's results. I haven't heard back — I suspect it's a busy day for them.)

And before you ask me why I didn't use Anthropic's official AI watermarking system to check, it's not available to normies yet.

Here's what Pangram's result was for the full 259,031 words on the S-1 filing:

92% of Anthropics S-1 filing is human, according to Pangram. Pangram / Business Insider

Well, well, well!

So there was a tiny bit of AI use, per Pangram. It flagged some sentences and passages it said with a "high" degree of confidence contained AI use, things like: "The pace of AI development is extremely rapid, with successive versions of models emerging in compressed timeframes, often measured in months rather than years."

"My read on a result like this is that it indicates light AI editing or AI usage at the sentence level," Spero told me. "It does not seem like any significant sections of the document are fully AI-generated."

An example of a sentence that Pangram flagged with "high" confidence as AI-generated. Pangram

This is one of the limitations of something like Pangram, which isn't 100% perfect, and the way that people use AI tools these days for editing and proofreading might affect results.

My colleague Aki Ito ran all her LinkedIn posts and published articles here on Business Insider through Pangram as an experiment, and ended up with a tiny surprise: one sentence in one article was flagged as AI-written. When she looked back at it, she saw that ChatGPT had suggested she rephrase her original, human-written sentence, and she accepted the new wording. (I use Grammarly, an AI-powered grammar tool that mainly helps with my repeated crimes against commas, but occasionally suggests rewording, which I sometimes accept and sometimes reject, though I worry that this may get flagged as AI).

So it would appear Anthropic still had real humans — slow, expensive, fallible meatsacks — write this massive document, with just a little assist from Claude in the editing. That might be par for the course these days for IPO teams and S-1's, or at the very least the S-1's for literal AI companies. (If you're a banker who works on S-1s, shoot me an email!)

I can't tell if I think this is a good sign or a bad sign for Anthropic (and the world)?

On one hand, shouldn't Anthropic be eating its own dog food, so to speak, and proudly use its own AI for something like this? An S-1 filing is full of the kind of boilerplate, charmless, technical-ish writing that you'd assume AI would be perfect to replace a human for. If Anthropic is going to the hassle of having real (presumably highly paid humans) write out each of the 250,000+ words here, well, where is its faith in its own product? I you can't trust Claude to write your S-1, can you trust it to cure cancer?

On the other hand, thank god this isn't 100% AI-written. The stakes are pretty high when the SEC is poring over every word. If you're going to tell people your company is worth hundreds of billions of dollars, at least have a human show up and type it out, eh?

I guess bankers everywhere can breathe a sigh of relief.