OTel agent

Prerequisites

The following environment variables must be set to configure the agents to send data to SixthSense collector.

  • export OTEL_RESOURCE_ATTRIBUTES=service.name={Provides service Name},service.instance.id=$HOST,sixthsense.team.id={Provide teamID}
  • export OTEL_EXPORTER_OTLP_COMPRESSION=none
  • export OTEL_INSTRUMENTATION_COMMON_DB_STATEMENT_SANITIZER_ENABLED=true

Rakuten's SixthSense Customer Support provides the following two endpoints for the SixthSense collector:

  • Endpoint for receiving traces
  • Endpoint for receiving management data

OTel sends telemetry data in the following two ways:

  1. Directly from Agents/Sdks: In this case Rakuten provides the configuration for Agents/Sdks to send the telemetry data to SixthSense collector.
  2. OTel collector: Agents/Sdks will send the data to OTel collector and Rakuten provides configuration for OTel collector so that it sends the data to SixthSense collector.

Configuring OTel collector

For configuring OTel collector, add exporters for sending the traces and management data in the otel-config.yml file.

The otel-config.yml file has three main sections for configuration:

  • Receivers
  • Exporters
  • Processors

Sample otel-config.yml file:

receivers:
otlp:
protocols:
grpc:
otlp/2:
protocols:
http:
exporters:
prometheus:
endpoint: "0.0.0.0:8889"
const_labels:
label1: value1
logging:
loglevel: debug
zipkin:
endpoint: "http://zipkin-all-in-one:9411/api/v2/spans"
format: proto
jaeger:
endpoint: jaeger-all-in-one:14250
tls:
insecure: true
otlp/2:
endpoint: http://host.docker.internal:15800
compression: none
tls:
insecure: true
otlp:
endpoint: https://otlp.nr-data.net:4317
headers:
"api-key": f0e99733db0c8b53f53a5dc78c6d360cFFFFNRAL
processors:
batch:
service:
extensions: [pprof, zpages, health_check]
pipelines:
traces:
receivers: [otlp,otlp/2]
processors: [batch]
exporters: [logging,otlp/2,otlp,jaeger,zipkin]
metrics:
receivers: [otlp]
processors: [batch]
exporters: [logging, prometheus,otlp]
  1. In the exporter section, add two OTLP exporters:
  • One GRPC exporter for sending traces
  • One Http exporter for sending management Data

Adding exporters

exporters:
otlp/2:
endpoint: <endpoint url>
headers:
Authentication: <access token>
teamId: <team id>
compression: none
tls:
insecure: true
otlphttp:
endpoint: <endpoint url>
compression: none
headers:
Authorization: <access token>
teamId: 0cdf4e3c-96a6-4f26-8fbd-9bbf52db8fcd
  1. Add the following exporters to the services section:
  • GRPC exporter in traces pipeline
  • Http exporter in metrics pipeline.

Adding exporters in the service pipeline

service:
pipelines:
traces:
receivers: [otlp]
processors: [batch]
exporters: [otlp/2]
metrics:
receivers: [otlp]
processors: [batch]
exporters: [otlphttp]
note

In the exporter services, all the exporters with endpoints, protocol, and authentication headers are defined. In the services section, the exporter used for sending traces and metrics are defined.

Configuring Auto-Instrumented Agents

OTel's main aim is to provide a standard format for collecting the telemetry data and for sending this data to any backend. OTel provides various environment variables in order to configure the agents to send the data to any backend. For these variables, see Prerequisites.

Configuring Traces

  • export OTEL_TRACES_EXPORTER=otlp
  • export OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=grpc
  • export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT={grpc endpoint}

Example:

export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=http://localhost:15800
  • export OTEL_EXPORTER_OTLP_TRACES_HEADERS=Authentication {access token from sixthsense UI},teamId={teamID}

Example:

export OTEL_EXPORTER_OTLP_TRACES_HEADERS=Authentication=eaccess token
teamId=teamID;

Configuring Management Data:

  • export OTEL_METRICS_EXPORTER=otlp
  • export OTEL_EXPORTER_OTLP_METRICS_PROTOCOL=http/protobuf
  • export OTEL_METRIC_EXPORT_INTERVAL=5000;
  • export OTEL_EXPORTER_OTLP_METRICS_ENDPOINT={http endpoint}

Example:

export OTEL_EXPORTER_OTLP_METRICS_HEADERS=Authorization Bearer {access token from sixthsense UI},teamId={teamID}

Example:

export OTEL_EXPORTER_OTLP_METRICS_HEADERS=Authorization Bearer <access token,teamId=<teamID>;