Core Concepts

This page digs into the underlying concepts of how Langfuse structures and captures your data. Understanding these will make debugging and working with traces easier.

Ready to start? Check out the Get Started guide to ingest your first trace.

Traces, Observations and Sessions

Langfuse organizes an application’s data into three core concepts: sessions, traces and observations.

Traces

A trace typically represents a single request or operation. For example, when a user asks a question to a chatbot, that interaction, from the user’s question to the bot’s response, is captured as one trace.

It contains the overall input and output of the function, as well as metadata about the request ( i.e. user, session, tags, etc.).

Observations

Each trace can contain multiple observations to log the individual steps of the execution. Example observations include LLM text generations, toolcalls, RAG retrieval steps, etc.

Nesting

Observations can be nested. The example below shows a trace with a nested observation.

Types

Langfuse supports a number of LLM application specific observation types:

• event is the basic building block. An event is used to track discrete events in a trace.
• span represents durations of units of work in a trace.
• generation logs generations of AI models incl. prompts, token usage and costs.
• agent decides on the application flow and can for example use tools with the guidance of a LLM.
• tool represents a tool call, for example to a weather API.
• chain is a link between different application steps, like passing context from a retriever to a LLM call.
• retriever represents data retrieval steps, such as a call to a vector store or a database.
• evaluator represents functions that assess relevance/correctness/helpfulness of a LLM’s outputs.
• embedding is a call to a LLM to generate embeddings and can include model, token usage and costs
• guardrail is a component that protects against malicious content or jailbreaks.

The advantage of using observation types is that you can filter on them, and certain types have special handling in the Langfuse UI.

Sessions

Optionally, traces can be grouped into sessions. Sessions are used to group traces that are part of the same user interaction. A common example is a thread in a chat interface.

Optionally, sessions aggregate traces

Example session in Langfuse UI

Using sessions is recommended for applications with multi-turn conversations or workflows. Please refer to the Sessions documentation to add sessions to your traces.

How Langfuse Captures Data

Now that you understand the data model, let’s explore how Langfuse actually captures and processes your traces.

Built on OpenTelemetry

Langfuse is built on OpenTelemetry, an open standard for collecting telemetry data from applications.

This means you’re not locked into using only Langfuse-specific SDKs. You can also send your traces to multiple destinations at once, like Langfuse for LLM observability and Datadog for infrastructure monitoring.

Instrumentation

Instrumentation is the process of adding code to record its behavior. Once this recording is turned on, Langfuse (through OpenTelemetry) can automatically capture these events and structure them into traces and observations.

The Get Started guide walks you through the process of instrumenting a function in your application.

Background Processing

In order to avoid slowing down your application, Langfuse doesn’t send traces synchronously the moment they’re created. Instead, Langfuse batches traces locally and sends them in the background, keeping your application fast and responsive.

Long-running applications

The approach above works well for long-running applications (like web servers or APIs) because the background exporter continuously runs and has plenty of time to flush batches on its own.

Short-lived applications

For applications that start, execute something, and shut down quickly (short-lived applications), there’s a risk that the application terminates while there are still unsent traces in the queue.

To avoid losing data, short-lived applications must explicitly call flush() before exiting. This forces the exporter to send all buffered traces immediately, so nothing is lost when the process terminates.