Skip to main content

PaginatedResponse[ScrapeJobRead]

items object[]required
  • Array [
    • idIdrequired

    Unique numeric identifier of the record.

    product_idProduct Idrequired

    ID of the product (workspace) that owns this record. Most endpoints scope access by product.

    agent_idAgent Idrequired

    ID of the agent (a configured AI assistant / bot persona) this record belongs to.

    directory_id objectrequired

    Knowledge directory receiving the documents; set as soon as ingestion starts.

    anyOf
    integer
    urlUrlrequired

    The URL the job was created for.

    modeScrapeDiscoveryModerequired

    Resolved discovery mode (site or page).

    Possible values: [site, page]

    context objectrequired

    Client context used by page selection and content cleaning.

    anyOf
    string
    max_pages objectrequired

    Effective page cap for this job.

    anyOf
    integer
    statusScrapeJobStatusrequired

    Job lifecycle state (queued, in_progress, completed, failed, expired).

    Possible values: [queued, in_progress, completed, failed, expired]

    stage objectrequired

    Current pipeline phase while in progress (discover, select, fetch, normalize, ingest).

    anyOf
    ScrapeJobStage

    Possible values: [discover, select, fetch, normalize, ingest]

    selection objectrequired

    Audit trail of every considered URL with its score, category, one-line reason, and whether it was kept.

    anyOf
  • Array [
    • property name*any
  • ]
    • stats objectrequired

    Live page counters (pages_discovered, pages_selected, pages_fetched, pages_normalized, pages_ingested, pages_failed), updated continuously while the job runs.

    anyOf
    object
    error_message objectrequired

    Failure reason for failed or expired jobs.

    anyOf
    string
    status_changed_atdate-timerequired

    Last status or liveness update time (UTC, ISO 8601).

    started_at objectrequired

    When a worker claimed the job (UTC, ISO 8601).

    anyOf
    date-time
    completed_at objectrequired

    When the job reached a terminal state (UTC, ISO 8601).

    anyOf
    date-time
    created_atdate-timerequired

    Creation time (UTC, ISO 8601).

    updated_atdate-timerequired

    Last modification time (UTC, ISO 8601).

  • ]
    • pagePagerequired
    limitLimitrequired
    totalTotalrequired
    pagesPagesrequired
    PaginatedResponse[ScrapeJobRead]
    {
      "items": [
        {
          "id": 0,
          "product_id": 0,
          "agent_id": 0,
          "directory_id": 0,
          "url": "string",
          "mode": "site",
          "context": "string",
          "max_pages": 0,
          "status": "queued",
          "stage": "discover",
          "selection": [
            {}
          ],
          "stats": {},
          "error_message": "string",
          "status_changed_at": "2024-07-29T15:51:28.071Z",
          "started_at": "2024-07-29T15:51:28.071Z",
          "completed_at": "2024-07-29T15:51:28.071Z",
          "created_at": "2024-07-29T15:51:28.071Z",
          "updated_at": "2024-07-29T15:51:28.071Z"
        }
      ],
      "page": 0,
      "limit": 0,
      "total": 0,
      "pages": 0
    }