diff --git a/pipeline/inputs/opentelemetry.md b/pipeline/inputs/opentelemetry.md index 7bb7d1636..5c59c58de 100644 --- a/pipeline/inputs/opentelemetry.md +++ b/pipeline/inputs/opentelemetry.md @@ -1,35 +1,36 @@ --- -description: An input plugin to ingest OTLP Logs, Metrics, and Traces +description: An input plugin to ingest OpenTelemetry logs, metrics, and traces --- # OpenTelemetry -The OpenTelemetry input plugin allows you to receive data as per the OTLP specification, from various OpenTelemetry exporters, the OpenTelemetry Collector, or Fluent Bit's OpenTelemetry output plugin. +The OpenTelemetry input plugin lets you receive data based on the OpenTelemetry specification from various OpenTelemetry exporters, the OpenTelemetry Collector, or the Fluent Bit OpenTelemetry output plugin. -Our compliant implementation fully supports OTLP/HTTP and OTLP/GRPC. Note that the single `port` configured which defaults to 4318 supports both transports. +Fluent Bit has a compliant implementation which fully supports `OTLP/HTTP` and `OTLP/GRPC`. The single `port` configured defaults to `4318` and supports both transport methods. -## Configuration +## Configuration -| Key | Description | default | -| ----------------- | -----------------------------------------------------------------------------------| ------- | -| listen | The network address to listen. | 0.0.0.0 | -| port | The port for Fluent Bit to listen for incoming connections. Note that as of Fluent Bit v3.0.2 this port is used for both transport OTLP/HTTP and OTLP/GRPC. | 4318 | -| tag | Tag for all the data ingested by this plugin. Important: This will only be used if `tag_from_uri` is set to false. Otherwise, the tag will be created from URI. | | -| tag_key | Specify the key name to overwrite a tag. If set, the tag will be overwritten by a value of the key | | -| raw_traces | Route trace data as a log | `false` | -| buffer_max_size | Specify the maximum buffer size in KB/MB/GB to the HTTP payload. | 4M | -| buffer_chunk_size | Initial size and allocation strategy to store the payload (advanced users only) | 512K | -|successful_response_code | It allows to set successful response code. `200`, `201` and `204` are supported.| 201 | -| tag_from_uri | By default, tag will be created from uri. e.g. v1_metrics from /v1/metrics. This must be set to false if using `tag` | `true` | -| threaded | Indicates whether to run this input in its own [thread](../../administration/multithreading.md#inputs). | `false` | +| Key | Description | Default | +| -------- | ------------| ------- | +| `listen` | The network address to listen on. | `0.0.0.0` | +| `port` | The port for Fluent Bit to listen for incoming connections. In Fluent Bit 3.0.2 or later, this port is used for both transport `OTLP/HTTP` and `OTLP/GRPC`. | `4318` | +| `tag` | Tag for all data ingested by this plugin. This will only be used if `tag_from_uri` is set to `false`. Otherwise, the tag will be created from the URI. | _none_ | +| `tag_key` | Specify the key name to overwrite a tag. If set, the tag will be overwritten by a value of the key. | _none_ | +| `raw_traces` | Route trace data as a log. | `false` | +| `buffer_max_size` | Specify the maximum buffer size in `KB`, `MB`, or `GB` to the HTTP payload. | `4M` | +| `buffer_chunk_size` | Initial size and allocation strategy to store the payload (advanced users only)` | `512K` | +| `successful_response_code` | Allows for setting a successful response code. Supported values: `200`, `201`, or `204`. | `201` | +| `tag_from_uri` | By default, the tag will be created from the URI. For example, `v1_metrics` from `/v1/metrics`. This must be set to false if using `tag`. | `true` | +| `threaded` | Indicates whether to run this input in its own [thread](../../administration/multithreading.md#inputs). | `false` | -Important note: Raw traces means that any data forwarded to the traces endpoint (`/v1/traces`) will be packed and forwarded as a log message, and will NOT be processed by Fluent Bit. The traces endpoint by default expects a valid protobuf encoded payload, but you can set the `raw_traces` option in case you want to get trace telemetry data to any of Fluent Bit's supported outputs. +Raw traces means that any data forwarded to the traces endpoint (`/v1/traces`) will be packed and forwarded as a log message, and won' be processed by Fluent Bit. The traces endpoint by default expects a valid `protobuf` encoded payload, but you can set the `raw_traces` option in case you want to get trace telemetry data to any of the Fluent Bit supported outputs. -### OTLP Transport Protocol Endpoints +### OpenTelemetry transport protocol endpoints -Fluent Bit based on the OTLP desired protocol exposes the following endpoints for data ingestion: +Fluent Bit exposes the following endpoints for data ingestion based on the OpenTelemetry protocol: + +For `OTLP/HTTP`: -__OTLP/HTTP__ - Logs - `/v1/logs` - Metrics @@ -37,7 +38,7 @@ __OTLP/HTTP__ - Traces - `/v1/traces` -__OTLP/GRPC__ +For `OTLP/GRPC`: - Logs - `/opentelemetry.proto.collector.log.v1.LogService/Export` @@ -49,8 +50,7 @@ __OTLP/GRPC__ - `/opentelemetry.proto.collector.trace.v1.TraceService/Export` - `/opentelemetry.proto.collector.traces.v1.TracesService/Export` - -## Getting started +## Get started The OpenTelemetry input plugin supports the following telemetry data types: @@ -60,12 +60,12 @@ The OpenTelemetry input plugin supports the following telemetry data types: | Metrics | Unimplemented | Stable | Stable | | Traces | Unimplemented | Stable | Stable | -A sample config file to get started will look something like the following: - +A sample configuration file to get started will look something like the following: {% tabs %} {% tab title="fluent-bit.yaml" %} + ```yaml pipeline: inputs: @@ -76,10 +76,12 @@ pipeline: - name: stdout match: '*' ``` + {% endtab %} {% tab title="fluent-bit.conf" %} -``` + +```python [INPUT] name opentelemetry listen 127.0.0.1 @@ -89,13 +91,15 @@ pipeline: name stdout match * ``` + {% endtab %} {% endtabs %} -With the above configuration, Fluent Bit will listen on port `4318` for data. You can now send telemetry data to the endpoints `/v1/metrics`, `/v1/traces`, and `/v1/logs` for metrics, traces, and logs respectively. +With this configuration, Fluent Bit listens on port `4318` for data. You can now send telemetry data to the endpoints `/v1/metrics` for metrics, `/v1/traces` for traces, and `/v1/logs` for logs. -A sample curl request to POST json encoded log data would be: -``` +A sample curl request to POST JSON encoded log data would be: + +```bash curl --header "Content-Type: application/json" --request POST --data '{"resourceLogs":[{"resource":{},"scopeLogs":[{"scope":{},"logRecords":[{"timeUnixNano":"1660296023390371588","body":{"stringValue":"{\"message\":\"dummy\"}"},"traceId":"","spanId":""}]}]}]}' http://0.0.0.0:4318/v1/logs ```