Telemetry
In order to stay aligned with Cloud Native efforts, Hiveio has integrated with OpenTelemetry to provide built-in logging, tracing, and metrcs if enabled. OpenTelemetry has stated that it is suitable for use in its current phase of development. We have followed their implementation guidelines to provide flexible integration with their API allowing you to extend that functionality in your code.
About
OpenTelemetry defines itself as an observability framework that provides the tools you need to capture information on how your services perform in the environments they are deployed. To use their own words:
OpenTelemetry is composed of the following:
- One API and SDK per language, which include the interfaces and implementations that define and create distributed traces and metrics, manage sampling and context propagation, etc.
- Language-specific integrations for popular web frameworks, storage clients, RPC libraries, etc. that (when enabled) automatically capture relevant traces and metrics and handle context propagation
- Automatic instrumentation agents that can collect telemetry from some applications without requiring code changes
- Language-specific exporters that allow SDKs to send captured traces and metrics to any supported backends
- The OpenTelemetry Collector, which can collect data from OpenTelemetry SDKs and other sources, and then export this telemetry to any supported backend
OpenTelemetry is a CNCF Sandbox member, formed through a merger of the OpenTracing and OpenCensus projects.
Concepts
OpenTelemetry combines existing concepts and introduces some new ones to create the comprehensive solution they provide. They define these as such:
In software, observability typically refers to telemetry produced by services and is often divided into three major verticals:
- Tracing, aka distributed tracing, provides insight into the full lifecycles, aka traces, of requests to the system, allowing you to pinpoint failures and performance issues.
- Metrics provide quantitative information about processes running inside the system, including counters, gauges, and histograms.
- Logging provides insight into application-specific messages emitted by processes.
These verticals are tightly interconnected. Metrics can be used to pinpoint, for example, a subset of misbehaving traces. Logs associated with those traces could help to find the root cause of this behavior. And then new metrics can be configured, based on this discovery, to catch this issue earlier next time. Other verticals exist (continuous profiling, production debugging, etc.), however traces, metrics, and logs are the three most well adopted across the industry.
Architecture
Apart from the previously mentioned components of the solution, which are built into Hiveio specialized containers, OpenTelemetry provides a Collector that receives data from the traces and metrics generated and export to your choice of supporting services such as Zipkin, Prometheus, and more.
The Collector provides a single binary and two deployment methods:
- An agent running with the application or on the same host as the application (e.g. binary, sidecar, or daemonset).
- A gateway running as a standalone service (e.g. container or deployment) typically per cluster, datacenter or region.
Once your deployment method is defined, the Collector needs to be configured to integrate with the supporting services previously mentioned. Check out the examples in the next section to get an idea on basic configurations.