Effortless Document Conversion to Markdown with Workers AI: Comprehensive Guide & API
In today's AI-driven landscape, Markdown has emerged as a pivotal format for text generation, Large Language Model (LLM) training and inference, and Retrieval Augmented Generation (RAG) systems. Its structured, semantic nature, readability for both humans and machines, and ease of parsing make it an ideal choice for AI agents.
This article explores how Workers AI streamlines document conversion to Markdown, empowering developers to efficiently process various document formats for their AI applications.
Why Markdown Matters for AI
- Structured Input: Provides a clear, organized structure for AI models to understand.
- Semantic Richness: Enhances the model's understanding of the text's meaning.
- Human-Readable: Facilitates easier debugging and maintenance.
- Machine-Parsable: Simplifies the process of extracting information for AI processing.
- RAG Optimization: Enables effective chunking and structuring of data for retrieval and synthesis.
- AI Agent Compatibility: Seamlessly integrates with AI agents due to its simplicity.
Introducing Workers AI toMarkdown
Workers AI offers the toMarkdown
utility, accessible through the env.AI
binding or REST APIs, for quick, convenient, and automated conversion of documents to Markdown.
Key Benefits
- Simplified Workflow: Eliminates manual conversion efforts.
- Multi-Format Support: Handles a wide range of document types.
- Automated Summarization: Leverages AI models for automatic summarization (especially useful for images).
- Cost-Effective: Free for many formats; pay-as-you-go pricing for AI-powered features.
Methods and Definitions
async env.AI.toMarkdown()
Converts an array of documents to Markdown.
Parameter
documents
: An array oftoMarkdownDocument
objects.
Return Value
results
: An array oftoMarkdownDocumentResult
objects.
toMarkdownDocument
Definition
name
:<string>
- The name of the document to convert.blob
:<Blob>
- A Blob object containing the document's content.
toMarkdownDocumentResult
Definition
name
:<string>
- The name of the converted document (matches the input name).mimetype
:<string>
- The detected MIME type of the document.tokens
:<number>
- The estimated number of tokens in the converted Markdown document.data
:<string>
- The Markdown content of the converted document.
Supported Formats
Workers AI supports a growing list of document formats for Markdown conversion. Check the table below for the current list.
Format | Description |
---|---|
Portable Document Format | |
JPEG | Joint Photographic Experts Group image format |
PNG | Portable Network Graphics image format |
GIF | Graphics Interchange Format image format |
TIFF | Tagged Image File Format image format |
WEBP | Web Picture image format |
BMP | Bitmap image format |
DOCX | Microsoft Word Open XML Document |
TXT | Plain Text File |
RTF | Rich Text Format |
HTML | HyperText Markup Language |
EPUB | Electronic Publication |
CSV | Comma Separated Values |
Note: This list is constantly being updated.
Example: Converting PDF and Image to Markdown
This example demonstrates how to fetch a PDF and an image from R2 storage and convert them to Markdown using env.AI.toMarkdown
.
import { Env } from "./env";
export default {
async fetch(request: Request, env: Env, ctx: ExecutionContext) {
// https://pub-979cb28270cc461d94bc8a169d8f389d.r2.dev/somatosensory.pdf
const pdf = await env.R2.get("somatosensory.pdf");
// https://pub-979cb28270cc461d94bc8a169d8f389d.r2.dev/cat.jpeg
const cat = await env.R2.get("cat.jpeg");
return Response.json(
await env.AI.toMarkdown([
{
name: "somatosensory.pdf",
blob: new Blob([await pdf.arrayBuffer()], {
type: "application/octet-stream",
}),
},
{
name: "cat.jpeg",
blob: new Blob([await cat.arrayBuffer()], {
type: "application/octet-stream",
}),
},
]),
);
},
};
Example Output:
[
{
"name": "somatosensory.pdf",
"mimeType": "application/pdf",
"format": "markdown",
"tokens": 0,
"data": "# somatosensory.pdf\n## Metadata\n- PDFFormatVersion=1.4\n- IsLinearized=false\n- IsAcroFormPresent=false\n- IsXFAPresent=false\n- IsCollectionPresent=false\n- IsSignaturesPresent=false\n- Producer=Prince 20150210 (www.princexml.com)\n- Title=Anatomy of the Somatosensory System\n\n## Contents\n### Page 1\nThis is a sample document to showcase..."
},
{
"name": "cat.jpeg",
"mimeType": "image/jpeg",
"format": "markdown",
"tokens": 0,
"data": "The image is a close-up photograph of Grumpy Cat, a cat with a distinctive grumpy expression and piercing blue eyes. The cat has a brown face with a white stripe down its nose, and its ears are pointed upright. Its fur is light brown and darker around the face, with a pink nose and mouth. The cat's eyes are blue and slanted downward, giving it a perpetually grumpy appearance. The background is blurred, but it appears to be a dark brown color. Overall, the image is a humorous and iconic representation of the popular internet meme character, Grumpy Cat. The cat's facial expression and posture convey a sense of displeasure or annoyance, making it a relatable and entertaining image for many people."
}
]
Note: In the case of the image, Workers AI uses an AI model to detect and summarize the content, providing a textual representation in Markdown.
Using the REST API
Alternatively, you can use the Workers AI REST API to convert documents to Markdown:
curl https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/tomarkdown \
-H 'Authorization: Bearer {API_TOKEN}' \
-F "[email protected]" \
-F "[email protected]"
This command uploads the specified files (cat.jpeg
and somatosensory.pdf
) to the Workers AI API for conversion.
Pricing
The toMarkdown
utility is free for most format conversions. When AI models are used for object detection and summarization (e.g., image conversion), it may incur additional costs if you exceed the Workers AI free allocation limits. For detailed pricing information, refer to the pricing page.
Conclusion
Workers AI's toMarkdown
functionality provides a seamless and efficient way to convert documents to Markdown, unlocking the potential of your data for AI applications. Whether you are building LLMs, RAG systems, or AI agents, Workers AI simplifies the process and empowers you to focus on innovation.