Documentation
Search infrastructure for everything your company knows.
Hayfork indexes your websites and files, lets you group them into collections, and answers questions over them with sources. People use it through a console or your product. Agents use it through one API.
Quickstart
Workspace to first API call in about ten minutes.
Start here →Search for agents
Citation-numbered passages sized to a token budget, for any LLM.
Retrieve endpoint →Website sources
Sitemaps, llms.txt, scope, page limits and re-syncs.
Crawler guide →API reference
Every endpoint, with limits and error codes.
Reference →Four things to know
Everything in Hayfork is one of these, and every guide uses the same words for them.
| Term | What it is |
|---|---|
| Workspace | Your isolated account. Sources, documents, collections, keys and usage all belong to exactly one workspace. Nothing is shared between workspaces. |
| Data source | Where content comes from: a website you crawl or a set of uploaded files. A source owns its documents and can be re-synced on its own. |
| Collection | A search scope made of one or more data sources. Every search names a collection. A source can sit in many collections, and its documents count once. |
| API key | How your product and your agents authenticate. Keys carry scopes, so a key embedded in a website can only search, never change anything. |
How content becomes an answer
When you connect a source, the text of each page or file is pulled out, converted to Markdown, split into chunks along its headings, and written to the index. Documents move from queued to processing to indexed, and you can watch that happen in the console.
A search runs two rankers over the collection at once: a full-text ranker for exact words and a vector ranker for meaning. The two rankings are merged, and each hit comes back with the chunk that matched, its heading path such as Billing › Invoices › Refunds, and a link to the source. The retrieve endpoint returns the same matches as full passages with citation numbers, ready to paste into a prompt.
Where things live
- Console: app.hayfork.dev. Sign in with your email. Collections, data sources, uploads, the web crawler, a search playground, and settings for keys and your plan.
- API:
https://api.hayfork.dev. Every endpoint takes a bearer key. The interactive Swagger UI lets you try calls with your own key. - Self-hosted: the same stack runs from Docker Compose on your own server, with a bundled embedding model so no text leaves your infrastructure. Write to us for a licence.
Plan limits
Limits are enforced by the API. A request over the limit returns 429 with a message saying which limit and how much room is left. The console shows monthly search usage in the sidebar.
| Plan | Data sources | Documents | Searches / month |
|---|---|---|---|
| Free | 2 | 300 | 1,000 |
| Team | 10 | 5,000 | 50,000 |
| Business | Unlimited | 25,000 | 500,000 |
| Self-hosted | Unlimited | Unlimited | Unlimited |
A document is one page, file or record, counted once however many collections include it. Re-syncing a source is free. Searches from people and from agents count the same.