Agent operations

Running agents in production: tracing and observability, evaluation and red-teaming, sandboxed execution, and hosting.

Tool	What it is	Verified
LangSmith	LangChain's framework-agnostic observability and evaluation platform for LLM applications and agents, providing run tracing, dashboards, prompt management and automated evals. Offers managed cloud, bring-your-own-cloud and self-hosted options.	2026-10-22
Langfuse	An open-source (MIT) agent evaluation and observability platform covering tracing, prompt management, datasets, experiments and evals, with a free self-hostable core. OpenTelemetry-native with 100+ integrations.	2026-10-22
Braintrust	A hosted observability, evaluation and gateway platform for LLM applications, offering multi-step agent tracing, LLM-as-a-judge and custom scorers, regression testing in CI and production monitoring.	2026-10-22
Arize Phoenix	An open-source AI observability and evaluation platform that traces model calls, retrievals, tool use and custom logic to debug agents. Built on OpenTelemetry and the OpenInference spec, runnable locally, self-hosted, on Kubernetes or via managed cloud.	2026-10-22
promptfoo	An open-source CLI and library for systematic LLM evaluation, prompt testing and AI security, running YAML-defined test cases across 50+ providers with automated scoring, plus red-teaming for prompt injection and data leakage.	2026-10-22
DeepEval	An open-source evaluation framework styled as Pytest for LLM applications, with 50+ research-backed metrics including LLM-as-a-judge, G-Eval and hallucination detection, plus synthetic data generation.	2026-10-22
E2B	An open-source cloud platform that runs one isolated microVM per agent session for executing AI-generated or untrusted code, with snapshots, shells, filesystems, browsing, networking and pause/resume. Supports both its own cloud and bring-your-own-cloud.	2026-10-22
Daytona	A sandboxed code-execution platform for AI-generated code and agent workflows, providing container-based sandboxes created in under 90ms, stateful persistent workspaces, environment snapshots and optional GPU compute. Available as a managed cloud with customer-managed compute.	2026-10-22
Modal	A serverless cloud for AI workloads that runs Python functions, model endpoints, batch jobs and isolated Sandboxes for executing untrusted agent code, with per-second billing and scale-to-zero autoscaling.	2026-10-22
Letta	A platform and open-source framework for building, hosting and running stateful agents with persistent memory, context management and tool execution. Agents can run in the managed cloud, a self-hosted app server, or fully local backends.	2026-10-22

Tool

What it is

Verified

LangChain's framework-agnostic observability and evaluation platform for LLM applications and agents, providing run tracing, dashboards, prompt management and automated evals. Offers managed cloud, bring-your-own-cloud and self-hosted options.

2026-10-22

Langfuse

An open-source (MIT) agent evaluation and observability platform covering tracing, prompt management, datasets, experiments and evals, with a free self-hostable core. OpenTelemetry-native with 100+ integrations.

2026-10-22

Braintrust

A hosted observability, evaluation and gateway platform for LLM applications, offering multi-step agent tracing, LLM-as-a-judge and custom scorers, regression testing in CI and production monitoring.

2026-10-22

Arize Phoenix

An open-source AI observability and evaluation platform that traces model calls, retrievals, tool use and custom logic to debug agents. Built on OpenTelemetry and the OpenInference spec, runnable locally, self-hosted, on Kubernetes or via managed cloud.

2026-10-22

promptfoo

An open-source CLI and library for systematic LLM evaluation, prompt testing and AI security, running YAML-defined test cases across 50+ providers with automated scoring, plus red-teaming for prompt injection and data leakage.

2026-10-22

DeepEval

An open-source evaluation framework styled as Pytest for LLM applications, with 50+ research-backed metrics including LLM-as-a-judge, G-Eval and hallucination detection, plus synthetic data generation.

2026-10-22

E2B

An open-source cloud platform that runs one isolated microVM per agent session for executing AI-generated or untrusted code, with snapshots, shells, filesystems, browsing, networking and pause/resume. Supports both its own cloud and bring-your-own-cloud.

2026-10-22

Daytona

A sandboxed code-execution platform for AI-generated code and agent workflows, providing container-based sandboxes created in under 90ms, stateful persistent workspaces, environment snapshots and optional GPU compute. Available as a managed cloud with customer-managed compute.

2026-10-22

Modal

A serverless cloud for AI workloads that runs Python functions, model endpoints, batch jobs and isolated Sandboxes for executing untrusted agent code, with per-second billing and scale-to-zero autoscaling.

2026-10-22

Letta

A platform and open-source framework for building, hosting and running stateful agents with persistent memory, context management and tool execution. Agents can run in the managed cloud, a self-hosted app server, or fully local backends.

2026-10-22