seed1

Evaluation using live or near-live traffic, feedback, or production outcomes — measures what actually happens when users interact with the deployed system.

Evaluation run outside live user traffic — on a saved dataset — before changes reach production. Used to catch regressions before users see them.

A product or workflow requirement that model answers cite the underlying sources used — usually driven by legal, compliance, or user trust needs.

An answer tied to source data, documents, or tool results that users can inspect or verify — not a confident-sounding guess.

A versioned set of prompts, instructions, examples, and tool guidance used by an AI application or agent. Versioned alongside the release it belongs to.

The scoring criteria used to judge whether model or agent output is good enough for the use case — defined by what the customer actually cares about, not by fluency.

A set of representative examples used to measure model, agent, retrieval, or workflow performance before and after changes.

A curated set of examples with trusted expected outcomes — used to test model or workflow behavior and catch regressions. Built from real cases, not invented ones.

The test infrastructure used to run model or agent evaluations repeatedly against examples, expected behavior, tools, and scoring logic. FDEs add real failure cases from production.

A customer-specific agentic solution built for a particular workflow — useful as a first deployment, but needs to be evaluated for what belongs in product before the second account asks for the same thing.

The technical and organizational model for building, approving, monitoring, supporting, escalating, and improving agents after launch.

A control that limits what an agent can see, say, decide, or do, which is enforced through policy, permissions, validation, or runtime checks.

The designed transfer of work between an agent and a human, another agent, or a system, which is triggered when confidence, ownership, or permissions change.

The logic that coordinates agents, tools, prompts, retrieval, state, and human checkpoints across a workflow. FDEs reduce this until it's as simple as the use case allows.

A smaller specialized agent or model role used inside a larger agent workflow to handle a specific task.

A bounded capability an agent can perform, which is defined by instructions, tools, inputs, outputs, permissions, and evals.

The staged introduction of an agent to users, workflows, permissions, or action types. Recommendations before write-back is a common first stage.

The ordered steps an agent follows: context gathering, reasoning, tool calls, decisions, human checkpoints, and outputs. FDEs simplify these after watching them in practice.

An enterprise operating model where agents participate in many workflows under shared governance, monitoring, and integration patterns. Needs standard tool permissions before teams build dozens of agents independently.

An application where AI agents perform meaningful workflow steps through tools, context, and controls (not just generating text for a user to copy somewhere else).