Start a website import

POST 
/v1/scrape-jobs/

Queues a background job that discovers a website's pages (or a single page plus the pages it links to), selects the most relevant ones with an LLM against the given context, cleans them, and ingests them into the knowledge base as markdown documents. Returns 201 with the queued job. Idempotent per agent and exact URL: while a job for the same URL is queued or running, the same request returns that job with 200 instead of creating a duplicate. Paywalled preview pages are detected and skipped.

Request

Responses

201

Successful Response

422

Validation Error