Prompt Configuration

How to configure prompts for every type of LLM-powered field — model response fields, inline evaluations, and criteria — including context mechanisms, placeholder mapping, and system prompts.

Three kinds of fields in a pipeline use LLM prompts, and each has a different configuration surface. This page explains how prompt configuration works for all three, when to use each approach, and how context reaches the model.

Field type	What it does	Where the prompt lives
Model response field	Generates a value (summary, translation, draft, classification)	Prompt Configuration Modal on the field
Inline evaluation field	Evaluates other fields using LLM-as-judge	Prompt Configuration Modal on the field (with evaluation targets)
Criteria field	Evaluates using a predefined criterion from the criteria library	Inside the criterion definition (`prompt_template`)

All prompt configuration starts in the Prompt Configuration Modal, opened from a field's settings in the Form Builder. The modal has two tabs — User prompt and System prompt — and adapts its layout depending on the field type.

For general LLM field setup (model, temperature, trigger timing, tools, regeneration), see LLM-Generated Fields.

Core concepts

Before diving into each field type, here are the building blocks that appear throughout.

Prompt sources

Every LLM-powered field gets its prompt from one of four sources:

Source	Description
Use prompt from dataset	The prompt is loaded from a CSV column at task seeding time.
Use custom prompt	A freeform prompt you write in the built-in editor, with optional template variable support.
Use prompt from library	A versioned prompt selected from the Prompt Library, pinned to a specific version.

Context mechanisms

Field values from the current node and upstream nodes can reach the LLM in one of three ways. These are mutually exclusive — which one applies depends on how the prompt is configured.

Appended context — The system automatically appends selected field values in a structured format after your prompt. Configured via the Context Sources selector, which defaults to "all" (every accessible field). At runtime, values appear in labeled sections at the end of the prompt:

--- ADDITIONAL CONTEXT ---
Field Title:
"field value here"

Variable insertion — When writing a custom prompt, you place {{placeholder_name}} tokens inline at the exact position you want a value to appear. Type {{ in the editor to open the autocomplete menu and browse available fields. At runtime, each token is replaced with the mapped field's value. Context is passed only through the mapped placeholders.

Placeholder mapping — When using a library prompt that declares {{name}} tokens, the Map Placeholders section appears where you wire each placeholder to a pipeline field. Context is passed only through the mapped placeholders.

Evaluation targets

Evaluation targets are field references that identify what is being evaluated. They apply to inline evaluation fields and criteria fields, not to regular model response fields.

How targets reach the LLM depends on the context mechanism in use:

Appended context — targets are auto-appended under a --- CONTENT TO EVALUATE --- section.
Variable insertion — targets are referenced via {{placeholder}} tokens in the prompt text, just like any other field.
Placeholder mapping — targets are wired through the Map Placeholders UI with kind: target.

Targets are selected via the Evaluation Target Picker, which shows fields from the current node (above the current field) and all upstream nodes.

Placeholder mappings

When a library prompt or criterion prompt declares {{name}} tokens, the pipeline builder shows a Map Placeholders section where you wire each token to a pipeline field.

Each mapping has:

Name — the declared placeholder (e.g., {{response}})
Source — any field from the current node or upstream nodes
Metadata — optionally include LLM execution metadata (tokens, cost, tool calls) alongside or instead of the field's value

Model response field

This is the most common use case — generating content for a field. The Prompt Configuration Modal opens with the full set of options.

Custom prompt path

Custom prompt: Select "Use custom prompt" in the Prompt Source radio group. The template editor opens, where you write your prompt.

Without variables — write plain text. Context sources are appended automatically after your prompt.
With variables — type {{ to insert field references inline. The autocomplete menu shows fields from the current node (above this field) and upstream nodes. Context is passed only through the mapped placeholders.
Without any context — set Context Sources to none. Only your prompt text is sent.

Dataset prompt: Select "Use prompt from dataset." The prompt will be loaded from a CSV column when tasks are created. No further configuration is needed in the modal.

Library prompt path

Select "Use prompt from library" and choose a prompt from the picker. The picker shows all available user-role prompts with version badges and expandable previews.

After selecting a prompt:

If the prompt declares {{placeholders}}, the Map Placeholders section appears. Wire each placeholder to a pipeline field using the dropdown. Context is passed only through mapped placeholders — no additional context is auto-appended.
If the prompt has no placeholders, you can still configure additional context sources below the picker.

Multi-variant comparison: For non-evaluation fields, you can select multiple library prompts to create sibling fields that run the same inputs through different prompts for A/B testing. Each variant gets its own placeholder mapping configuration.

System prompt

Switch to the System prompt tab in the modal. The platform default system prompt is always applied (shown as a read-only preview). You have three options for additional instructions:

Option	Description
Default only	No additional instructions. Just the platform default.
Append custom instructions	Write additional instructions in a text editor. These are appended after the platform default.
From prompt library	Select a versioned system-role prompt from the library. Also appended after the platform default.

System prompts are additive — the platform default is always sent first. Your custom or library system prompt is appended, never a replacement.

Step-by-step: configuring a custom prompt with appended context

Set the field's data source to LLM.
Click the prompt chip on the field to open the Prompt Configuration Modal.
Select Use custom prompt under Prompt Source.
Write your prompt in the editor.
Configure Context Sources below the editor to control which fields are appended (default: all).
Optionally switch to the System prompt tab to add custom system instructions.
Click Done.

Step-by-step: configuring a custom prompt with variables

Set the field's data source to LLM.
Click the prompt chip on the field to open the Prompt Configuration Modal.
Select Use custom prompt under Prompt Source.
Write your prompt in the editor. Type {{ to insert a variable — the autocomplete shows fields from the current node and upstream nodes.
Optionally switch to the System prompt tab to add custom system instructions.
Click Done.

Inline evaluation field

An inline evaluation field is a model response field that has been configured to evaluate other fields. It uses the same Prompt Configuration Modal but with additional evaluation-specific controls.

Enabling evaluation mode

In the Prompt Configuration Modal, toggle the evaluation switch at the top of the User prompt tab. This reveals:

Evaluation Target Picker — select which fields this evaluator judges.
Evaluation Mode — choose between:
- Independent — the evaluator runs once per target, producing a separate score for each. The system creates sibling evaluator instances automatically.
- Grouped — the evaluator runs once with all targets included together, producing a single combined score.

Custom prompt path (with targets)

When writing a custom prompt for an evaluation field:

Targets are auto-appended under a --- CONTENT TO EVALUATE --- section with their label, helper text, and value. If a target is an LLM-response field, its execution metadata can optionally be included.
Additional context fields are appended under --- ADDITIONAL CONTEXT ---.
Variable insertion works the same as model response fields — type {{ to insert field references. The special token {{evaluation_targets}} inlines all evaluation targets at that position in the prompt (suppressing the auto-appended targets section for those entries).

Library prompt path (with targets)

When using a library prompt for an evaluation field:

The picker is restricted to single selection (no multi-variant comparison).
The Map Placeholders section appears with source options split into two pools:
- Targets — the fields being evaluated
- Available Fields — additional context fields
Each placeholder mapping has a kind:
- target — the value is always included. You can optionally multi-select metadata keys (tokens, cost, tool calls) to include alongside the value.
- context — choose between the field's value or a single metadata key.

Step-by-step: configuring an inline evaluator

Add a field (e.g., Rating or Numeric) and set its input mode to LLM Response.
Open the Prompt Configuration Modal.
Toggle the evaluation switch on.
Select evaluation targets using the target picker.
Choose Independent or Grouped mode.
Write your evaluation prompt (custom) or select one from the library.
If using a library prompt with placeholders, map each placeholder to a target or context field.
Click Done.

Criteria field

A criteria field uses a predefined criterion from the criteria library. The prompt configuration works differently because the prompt lives inside the criterion definition, not in the field config.

Criterion types

Type	Prompt behavior
LLM Judge	Uses the criterion's `prompt_template` — an author-written evaluation prompt. This is the only type that involves prompt configuration.
LLM Metric	Runs a predefined metric evaluation pipeline (e.g., answer relevancy, toxicity). No user-written prompt.
Programmatic	Deterministic evaluation (regex match, JSON validity, keyword check). No LLM call.
Human Rating	Renders as an input field for a human reviewer. No LLM call.

The rest of this section applies only to LLM Judge criteria.

Target + context selection (appended)

When a criterion's prompt template does not use {{placeholders}}:

Select evaluation targets — the fields being evaluated. Their values are appended under --- CONTENT TO EVALUATE ---.
Select additional context — other fields to provide background. Appended under --- ADDITIONAL CONTEXT ---.
Execution metadata — if a target is an LLM-response field, you can optionally include its execution metadata (tokens, cost, latency, tool calls) via the Execution Context selector.

All three layers are concatenated and appended to the criterion's prompt template at runtime.

Placeholder mapping (when the criterion uses variables)

When a criterion's prompt template declares {{name}} tokens (e.g., {{response}}, {{question}}), the Map Placeholders section appears:

Source options are split into Targets and Available Fields pools.
Each placeholder is mapped with a kind:
- target — value is always included. Optionally multi-select metadata keys to include alongside the value.
- context — choose between the field's value or a single metadata key.
In Independent evaluation mode, target placeholder mappings are automatically rewritten for each sibling evaluator to point at its own target field.

Step-by-step: configuring a criteria field

Add a Criteria field to the form.
Select a criterion from the library (or create one inline).
Select evaluation targets using the target picker.
If the criterion's prompt has {{placeholders}}, map each one to a target or context field in the Map Placeholders section.
If the criterion's prompt does not have placeholders, configure additional context sources and execution metadata.

Quick reference

Field type	Prompt source	Context mechanism	How it works
Model response	Custom prompt	None	Plain text prompt with no context
Model response	Custom prompt	Variables	`{{placeholders}}` inserted inline; context passed only through mapped placeholders
Model response	Custom prompt	Appended	Plain text prompt; context sources auto-appended
Model response	Dataset	Appended	Prompt from CSV column; context auto-appended
Model response	Library prompt	Placeholder mapping	Each `{{name}}` wired to a pipeline field
Model response	Library prompt	Appended	No placeholders; context sources auto-appended
Inline eval	Custom prompt	Targets + context appended	Targets under "CONTENT TO EVALUATE"; context under "ADDITIONAL CONTEXT"
Inline eval	Custom prompt	Variables	`{{placeholders}}` reference targets and context; `{{evaluation_targets}}` inlines all targets
Inline eval	Library prompt	Placeholder mapping	Placeholders split into target and context kinds
Criteria (LLM Judge)	Criterion template	Targets + context appended	Target values and context appended to the criterion's prompt template
Criteria (LLM Judge)	Criterion template	Placeholder mapping	`{{name}}` tokens mapped to targets and context with kind annotations

Tips

Appended context is simpler; variables give precision. Use appended context when you want the LLM to see everything. Use variables when the prompt needs specific values at specific positions.
Library prompts are best for reuse and A/B testing. Save a prompt to the library when you plan to use it across multiple fields or pipelines. Use multi-variant selection for comparing prompt strategies.
Context order matters. Fields appear in display order — current node fields above the target field first, then upstream node fields from all completed ancestors.
Save custom prompts to the library. The "Save to Library" button in the modal footer lets you promote any custom prompt for reuse.

On this page