API Reference

Pipelines exposes a small, focused REST API — the External API v1 — for programmatic task creation and status polling.

If anything in this page disagrees with the live OpenAPI spec at /api/v1/openapi.json, treat the spec as the source of truth.

What the External API can do

The following are not in the External API: creating/editing/activating pipelines, creating projects or organizations, editing or deleting tasks, querying datasets, and triggering evaluation runs. Use the web app for those.

Base URL

https://api.pipelines.tech/api/v1

Authentication

Every endpoint except GET /health requires an organization-scoped API key. Keys are the literal prefix pk_live_ followed by 40 hex characters (48 characters total) and are scoped to a single organization — they can only access workflows inside the org they were created in.

Pass the key as either Authorization: Bearer <key> or X-API-Key: <key> — both schemes are accepted.

curl -H "Authorization: Bearer pk_live_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" \
  https://api.pipelines.tech/api/v1/workflows/42

Missing, malformed, expired, or revoked keys return 401 with error code missing_api_key (no key in the request) or invalid_api_key (wrong prefix, not found, expired, or revoked). Revocation takes effect immediately.

Creating an API key

Only Org Admin users can manage API keys. Create Key under Organization Keys (in the admin API Keys page) takes:

• Key name (required) — used to identify and revoke the integration later.
• Expiry — one of No expiry, 30 days, 90 days, or 1 year, measured from creation.

The full pk_live_... value is shown once on creation and cannot be retrieved afterward. If lost, revoke and create a new key.

After a key exists

From the Organization Keys table you can:

• Update expiry — pick a new duration (30 days, 90 days, 1 year, or No expiry). The new expiration is always computed from now, so picking "30 days" on a key that already has a year left will shorten it to 30 days from today. Updating expiry on a revoked key returns 400.
• Revoke — permanently disables the key. Revocation is immediate and cannot be undone. Revoked keys remain in the list for audit and sort below active keys; re-revoking is a no-op.
• See Last Used, Expires, Created, and Created By columns. Last Used is debounced to roughly once per minute per key, so a freshly-used key may take a moment to refresh.

There is no rename or rotate action today. To rotate a key, revoke the old one and create a new one.

API keys carry the identity of the user who created them. If that user is later removed, the three ZIP-upload endpoints (init, confirm, and status) start returning 422 because they can no longer resolve an uploader identity; in that state, revoke the key and have an active Org Admin create a new one. JSON task creation and status endpoints do not depend on the creator identity and continue to work.

Rate limiting

Each API key has its own rate-limit bucket. /health and other unauthenticated paths are bucketed by API key when a key is supplied, and by client IP otherwise. Repeated failed auth attempts against protected paths are rate-limited per client IP to mitigate brute-forcing.

On 429 Too Many Requests, the Retry-After header and body field retry_after both return the wait in integer seconds. The exact per-key limit is configured per deployment; treat Retry-After as authoritative.

Request size limits

• POST /workflows/{workflow_id}/tasks — up to 5,000 tasks per request.
• All JSON endpoints — request body capped at 32 MB. Exceeding the cap returns 413 payload_too_large.
• ZIP archives do not count against this limit; they upload directly to cloud storage via a presigned URL (see the upload flow below).

Error responses

All errors use this shape:

{
  "error": "validation_error",
  "message": "tasks: ensure this value has at most 5000 items",
  "retry_after": 30
}

• error is a machine-readable code. Stable values: missing_api_key, invalid_api_key, invalid_request, unauthorized, forbidden, not_found, payload_too_large, validation_error, rate_limit_exceeded, internal_server_error, service_unavailable, api_error.
• message is a human-readable description.
• retry_after is present only on 429 responses.

Status code mapping:

Successful responses are 200 OK for GETs and POST /tasks, and 202 Accepted for POST /tasks/upload and POST /tasks/upload/{upload_id}/confirm (both kick off async work).

Pagination

Only GET /workflows/{workflow_id}/tasks is paginated. It uses cursor-based pagination — there is no offset or total count.

Query parameters:

• limit — integer, 1–200, default 50.
• cursor — opaque string; omit on the first page. Pass the next_cursor from the previous response to get the next page.
• status — optional filter. Valid values: pending, in_progress, finished, failed, paused, escalated, quarantined. Any other value returns 400 invalid_request with the accepted values listed in the message.

Response shape:

{
  "tasks": [
    /* TaskStatusResponse[] */
  ],
  "next_cursor": "eyJsYXN0X2lkIjogMTAwMX0=",
  "has_more": true
}

Results are sorted by task ID descending (newest first). next_cursor is null when has_more is false. An invalid cursor returns 400 invalid_request.

Interactive documentation

• Swagger UI: https://api.pipelines.tech/api/v1/docs — authorize with ExternalApiBearerAuth or ExternalApiKeyHeaderAuth.
• ReDoc: https://api.pipelines.tech/api/v1/redoc
• OpenAPI JSON: https://api.pipelines.tech/api/v1/openapi.json

Typical flow: create tasks from JSON

1. Discover field names. Call GET /workflows/{workflow_id}/schema to get the exact field name values you need to send. The name is the API key in fields; title is the human label shown in the UI. For select fields, format_hint lists valid options.
2. Check prerequisites.
   • The workflow must be ACTIVE or PAUSED. Any other workflow status (DRAFT, COMPLETED, ARCHIVED, PENDING_ACTIVATION) rejects task creation with 422 and the message Workflow must be ACTIVE or PAUSED to create tasks.
   • If the schema response has has_file_upload_fields: true, POST /tasks is not the right endpoint — use the ZIP-upload flow below instead; /tasks will return 422.
3. POST the batch.

curl -X POST https://api.pipelines.tech/api/v1/workflows/42/tasks \
  -H "Authorization: Bearer pk_live_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" \
  -H "Content-Type: application/json" \
  -d '{
    "tasks": [
      {
        "fields": {
          "input_text": "Translate this to French: Hello world",
          "target_language": "French"
        }
      }
    ]
  }'

4. Handle the partial-success response. A 200 does not mean every task succeeded — only that at least one did. Always inspect both counts:

{
  "success_count": 1,
  "error_count": 1,
  "errors": [
    {
      "index": 1,
      "field_name": "target_language",
      "message": "Missing required columns: target_language"
    }
  ],
  "task_ids": [1001]
}

errors[].index is the zero-based position in the tasks array you sent. errors can contain both pre-validation failures (unknown or missing fields) and downstream seeding failures, mixed in the same array. Only when every task fails does the endpoint return 422 instead of 200.

Common mistakes:

• Sending field titles (what you see in the UI) instead of field names (what /schema returns) — yields "Unknown field" errors.
• Forgetting that the workflow must be in ACTIVE or PAUSED state — DRAFT / COMPLETED / ARCHIVED workflows always reject task creation.
• Sending tasks to a workflow that has predefined file_upload fields — use the upload flow instead.

Typical flow: create tasks with files (ZIP upload)

When the workflow's schema reports has_file_upload_fields: true, the only way to create tasks is the three-step upload flow:

1. Init. POST /workflows/{workflow_id}/tasks/upload with an optional tasks array (1–5,000 items) and optional file_size_bytes. Returns 202 Accepted. The response contains:

   • upload_id — UUID used for the rest of the flow.
   • upload_url — a presigned cloud-storage PUT URL.
   • expires_at — when upload_url stops working. Expiration scales with file_size_bytes and is clamped between 60 minutes (the default when file_size_bytes is omitted) and 12 hours. Send a realistic file_size_bytes for large ZIPs so the URL doesn't expire mid-upload.

   If has_only_file_upload_fields: true on the schema, the tasks array may be omitted — in that case every task row is derived from the ZIP's top-level contents:

   • When there is a single file_upload field, each top-level file in the ZIP becomes one task and that file is assigned to the field.
   • When there are multiple file_upload fields, each top-level folder becomes one task; files inside the folder are matched to fields by name.

   Otherwise tasks is required.

2. Upload the ZIP directly to upload_url using an HTTP PUT with Content-Type: application/zip. This does not go through the API server, so the 32 MB limit does not apply.

3. Confirm. POST /workflows/{workflow_id}/tasks/upload/{upload_id}/confirm starts background processing. Returns 202 Accepted with { "upload_id": "...", "status": "processing" }.

   • Calling /confirm before the ZIP is actually in storage returns 422 ("ZIP archive has not been uploaded yet").
   • Calling /confirm twice returns 409 ("Upload has already been confirmed or is no longer pending"). The endpoint is effectively single-use on success.
   • If the background worker is unreachable, /confirm returns 503 and leaves the upload in pending so a retry is safe.

4. Poll status. GET /workflows/{workflow_id}/tasks/upload/{upload_id} until status is completed, failed, or cancelled. A terminal completed response includes success_count, error_count, per-row errors, and task_ids.

Task status and node progress

GET /workflows/{workflow_id}/tasks/{task_id} returns:

{
  "id": 1001,
  "status": "in_progress",
  "node_summary": { "total": 5, "finished": 2, "in_progress": 1, "pending": 2 },
  "created_at": "2026-03-30T12:00:00Z",
  "updated_at": "2026-03-30T12:05:00Z"
}

• status is one of pending, in_progress, finished, failed, paused, escalated, quarantined.
• node_summary collapses the many internal node statuses into three buckets:
  • finished — finished nodes.
  • in_progress — available, claimed, submitted, escalated, quarantined.
  • pending — pending, deferred.
• Nodes with status na are excluded from all counts, including total.
• Tasks are always scoped to the workflow_id in the path. A task_id that exists in a different workflow (even in the same org) returns 404.

This same payload (per task) is what populates the tasks[] array in the list endpoint.

Verifying your integration

1. GET /health without a key — expect {"status": "ok"}.
2. GET /workflows/{workflow_id} with your key — expect the workflow name and status. A 404 here means the key is not scoped to that workflow's org, or the workflow ID is wrong.
3. GET /workflows/{workflow_id}/schema — confirm the field name values match what your integration sends.
4. Create a single task with POST /tasks and then GET /tasks/{task_id} with the returned ID. The task should appear in the Data Explorer tab for the workflow in the UI.