Skip to main content
Version: 0.8.0

Send logs from OTEL Collector

If you already have your own OpenTelemetry Collector and want to export your logs to Quickwit, you need a new OLTP gRPC exporter in your config.yaml:

otel-collector-config.yaml
receivers:
otlp:
protocols:
grpc:
http:

processors:
batch:

exporters:
otlp/quickwit:
endpoint: host.docker.internal:7281
tls:
insecure: true
# By default, logs are sent to the otel-logs-v0_7.
# You can customize the index ID By setting this header.
# headers:
# qw-otel-logs-index: otel-logs-v0_7
service:
pipelines:
logs:
receivers: [otlp]
processors: [batch]
exporters: [otlp/quickwit]

Test your OTEL configuration

  1. Install and start a Quickwit server:
./quickwit run
  1. Start a collector with the previous config:
docker run -v ${PWD}/otel-collector-config.yaml:/etc/otelcol/config.yaml -p 4317:4317 -p 4318:4318 -p 7281:7281 otel/opentelemetry-collector
  1. Send a log to your collector with cURL:
curl -XPOST "http://localhost:4318/v1/logs" -H "Content-Type: application/json" \
--data-binary @- << EOF
{
"resource_logs": [
{
"resource": {
"attributes": [
{
"key": "service.name",
"value": {
"stringValue": "test-with-curl"
}
}
]
},
"scope_logs": [
{
"scope": {
"name": "manual-test"
},
"log_records": [
{
"time_unix_nano": "1678974011000000000",
"observed_time_unix_nano": "1678974011000000000",
"name": "test",
"severity_text": "INFO"
}
]
}
]
}
]
}
EOF

You should see a log on the Quickwit server similar to the following:

2023-03-16T13:44:09.369Z  INFO quickwit_indexing::actors::indexer: new-split split_id="01GVNAKT5TQW0T2QGA245XCMTJ" partition_id=6444214793425557444

This means that Quickwit has received the log and created a new split. Wait for the split to be published before searching for logs.