Claude AI can now see your PDFs

It’s been a big couple of weeks for AI company Anthropic. Late last month, the company released a computer-controlling version of its flagship Claude AI model that could type, swipe and click.

This week, the company upgraded Claude’s visual capabilities to see, rather than just read, PDFs.

Before now, Anthropic extracted text from PDFs and fed it into Claude as a prompt. This meant the model wasn’t getting any information on graphic elements like tables, charts and images.

 Now, Anthropic turns the PDF into an image and sends both the text and visual elements of the document to Claude for analysis. 

Users can drag and drop PDFs into the Claude web interface to try it out. You need to turn on Visual PDFs in the feature preview to access the functionality.

Developers can send PDFs via the API. As it’s still in public beta, bugs are likely for the next few weeks.

It’s not clear how well the API will handle complex visuals, but Anthropic has released a demonstration video of Claude identifying a particular bone from a diagram of a hand.

For developers, the new API feature could offer a better way to extract tabular information from PDFs at scale. Traditional OCR software like Tabula can extract PDF tables and save them as spreadsheets, but it offers relatively mixed results — especially if a page contains other graphics.

Anthropic says Claude’s PDF support should be good at analyzing financial reports, understanding charts, extracting key info from legal documents and assisting with translation.

Many indie hackers are already excited about the update.

“PDF via API??? You guys could go quiet for the rest of November and it would still be a massive month - huge!!” wrote McKay Wrigley on X.

Even Lex Fridman gave it a shout-out on X.

But others say the growing capabilities of AI models puts certain indie businesses at risk.

Dima Rubanov, who co-founded German-language AI-PDF chat service AskthePDF.ai said that running standalone LLM tools was "becoming increasingly difficult" as big AI firms integrate more services.

Indie makers needed to provide more comprehensive services to stay ahead of the game, he added. His own start-up, for example, now includes dedicated features for academics.

Some alternative AI models already offer similar PDF-reading capabilities to Claude. OpenAI’s ChatGPT can read PDFs dropped into its web interface, for example. But API users need to convert PDFs to PNG images to send them to GPT.

Google’s Pinpoint service can search through batches of PDFs and transform tables into spreadsheets. It’s a Google News Initiative, so access is available on request to journalists.

Users can access Claude’s PDF capabilities by using the “anthropic-beta: pdfs-2024-09-25” header in API requests to the latest 3.5 Sonnet model.

You can send documents up to 32mb and with a maximum page size of 100. You can use other API features like prompt caching and batch processing to streamline requests and reduce costs.

 The Amazon Bedrock and Google Vertex AI platforms will have access to the functionality soon.

Full details are available in the Anthropic API docs.