Guide
File uploads
Upload the documents that never made it onto a website: handbooks, contracts, specs, exported help articles. They index the same way pages do and search alongside them.
From the console
- In Data sources, create a source with type Upload and a name such as Policies. Files belong to the source; collections include the source without owning it.
- Open Upload, choose that source, and drop your files. Each file becomes a document with status
queued, thenprocessing, thenindexed. The list updates as they move. - Add the source to a collection, and its documents are searchable there.
Supported files
| Type | Extensions | How it is read |
|---|---|---|
.pdf | Text extracted per page. Scanned PDFs without a text layer index as empty. | |
| Word | .docx | Paragraph text. |
| Plain text | .txt | As is. |
| Markdown | .md, .markdown | Kept as Markdown, so headings drive chunking and show up in results. |
Files are limited to 50 MB each. Other extensions are refused with 415. Markdown is the best input: it carries the structure that makes results precise.
How files are chunked
Content is converted to Markdown, then split along headings. Short sections are grouped up to a fixed size, long sections are split with some overlap, and every chunk is prefixed with its heading path, for example Handbook › Leave › Parental leave. Plain text and PDF without headings are split by size. The path is stored with the chunk and returned by search and retrieve.
From the API
Requires the ingest scope and a verified email. Send a form upload with the source id and one or more files:
curl https://api.hayfork.dev/documents/upload \
-H "Authorization: Bearer $HAYFORK_KEY" \
-F data_source_id=SOURCE_ID \
-F files=@handbook.pdf \
-F files=@faq.mdThe response is 202 with one document per file. Each has an id and a status to poll:
curl https://api.hayfork.dev/documents/DOCUMENT_ID -H "Authorization: Bearer $HAYFORK_KEY"
curl https://api.hayfork.dev/documents/DOCUMENT_ID/chunks -H "Authorization: Bearer $HAYFORK_KEY"Create the upload source itself with POST /data-sources and {"name": "Policies", "source_type": "upload"}. Delete a document with DELETE /documents/DOCUMENT_ID; the file, its chunks and its index entries go together. Deleting a source removes all of its documents.
Updating a file
Upload the new version and delete the old document. Uploads are counted as documents on your plan, so a replaced file frees its slot when the old one is deleted. If you change the embedding model on a self-hosted deployment, Reindex on the source re-extracts and re-chunks the stored files without uploading again.