Observability programmes often begin with an ambitious target: every service will produce structured logs, meaningful metrics and complete distributed traces. The target is reasonable. The difficulty is that teams do not begin from the same place, services do not carry the same risk, and telemetry is only useful when people can operate it.
I developed a Three Levels of Observability approach to make that journey more practical. It was adopted within a large organisation as a way of establishing clearer expectations for engineering teams. It is not a universal standard. It is a model to adapt to the organisation, platform and service in front of you.
The central idea is simple: define a useful baseline, build operational depth, then add end-to-end diagnostic capability where its value justifies the work.
Level one: a dependable service baseline
The first level should answer the questions every team faces when a service is released:
- Is the service running?
- Is it accepting and completing work?
- When something fails, can we find the relevant event?
- Can an operator connect a failure to a deployment or configuration change?
At this level, focus on structured application logs, basic health and workload metrics, and a small set of actionable alerts. Logs should include a consistent timestamp, severity, service identity, environment and a correlation identifier where one is available. They should avoid secrets and unnecessary personal information.
Metrics should describe availability and the shape of work: request volume, errors, latency, queue depth or job completion, depending on the service. A standard HTTP dashboard may be useful for an API, but it is a poor fit for a scheduled data pipeline. The requirement should describe the operational questions, not prescribe identical charts for every workload.
Level one is complete when a team can recognise a service problem and begin an investigation without logging directly into a host or searching several disconnected systems.
Level two: explain service behaviour
The second level adds enough context to understand why the service is behaving differently. It connects technical signals to releases, dependencies and user-visible operations.
Useful improvements include:
- consistent correlation identifiers across internal calls;
- deployment and version information attached to telemetry;
- service-level indicators for important operations;
- dependency health and saturation metrics;
- dashboards organised around operational questions;
- alerting based on symptoms rather than every low-level event.
This is also the point to improve telemetry quality. Can a person move from an alert to a dashboard, then to the relevant logs, without reconstructing identifiers by hand? Do labels have controlled cardinality? Does an error message explain the failed operation and the next useful context, or merely repeat a stack trace?
An organisation can support this level with reusable logging libraries, OpenTelemetry configuration, dashboard templates and conventions for resource attributes. Standardisation matters most at the boundaries: service name, environment, deployment version and correlation context should mean the same thing everywhere.
The aim is not to collect more data. It is to reduce the time and guesswork required to form a defensible explanation.
Level three: understand the distributed path
The third level provides end-to-end visibility through a distributed system. Distributed tracing is the obvious capability, but mature tracing involves more than deploying an agent.
A useful trace needs context propagation across the important path, sensible span boundaries, controlled attributes, error status and links to logs or metrics. Sampling must preserve valuable failure and latency information without creating unacceptable cost. Async messaging, batch processing and boundary systems need deliberate instrumentation because the default HTTP path will not cover them.
OpenTelemetry is valuable here because it separates instrumentation from a single analysis backend. Teams can use common APIs and semantic conventions while exporting to platforms such as Elastic or AWS X-Ray. That portability does not remove the need for governance: collector configuration, redaction, sampling and attribute policy still require ownership.
Level three should be driven by system shape and diagnostic need. A small independent service may operate well at level two. A critical transaction crossing several teams and platforms may justify level-three tracing early. Maturity should never become a badge that encourages expensive telemetry without a use case.
Adapt the levels to service criticality
The model works best when paired with a simple service classification. Consider factors such as:
| Factor | Questions to ask |
|---|---|
| User impact | What happens when this service is slow or unavailable? |
| System complexity | How many components and teams take part in an operation? |
| Change rate | How frequently is the service deployed or reconfigured? |
| Data sensitivity | What must be redacted, restricted or retained carefully? |
| Support model | Who responds, during which hours, using which tools? |
A high-impact distributed service may need all three levels. An internal utility may need a strong baseline and a few level-two practices. Write down the decision so that teams understand both the expectation and the reasoning.
Make adoption an engineering product
Standards alone do not create observability. Teams need paved paths: supported libraries, collector endpoints, examples, test environments, dashboards and clear onboarding documentation. Platform teams should test those paths as products.
It is also useful to define evidence for each level. Evidence might include a dashboard review, a sample incident walkthrough, telemetry field checks or an automated assertion that a deployed test service produces a trace. The evidence should show that the capability works, not that a document exists.
Start with a small number of representative services. Learn where instrumentation is difficult, where costs rise and which conventions are confusing. Improve the platform before asking every team to adopt it.
A progression, not a score
The Three Levels of Observability approach gives teams a shared language for improvement. Its value comes from making the next useful step visible while allowing expectations to reflect operational reality.
Use the model to guide conversations, investment and implementation. Change the names, evidence and technical requirements to fit your organisation. The outcome that matters is not reaching “level three”. It is enabling engineers and operators to understand their systems when it counts.