Urtzi Alfaro
|
1329bae784
|
Fix SigNoz OTel Collector configuration and disable OpAMP
Root Cause Analysis:
- OTel Collector was starting but OpAMP was overwriting config with "nop" receivers/exporters
- ClickHouse authentication was failing due to missing credentials in DSN strings
- Redis/PostgreSQL/RabbitMQ receivers had missing TLS certs causing startup failures
Changes:
1. Fixed ClickHouse Exporters:
- Added admin credentials to clickhousetraces datasource
- Added admin credentials to clickhouselogsexporter dsn
- Now using: tcp://admin:27ff0399-0d3a-4bd8-919d-17c2181e6fb9@signoz-clickhouse:9000/
2. Disabled Unconfigured Receivers:
- Commented out PostgreSQL receivers (no monitor users configured)
- Commented out Redis receiver (TLS certificates not available)
- Commented out RabbitMQ receiver (credentials not configured)
- Updated metrics pipeline to use only OTLP receiver
3. OpAMP Disabled:
- OpAMP was causing collector to use nop exporters/receivers
- Cannot disable via Helm (extraArgs appends, doesn't replace)
- Must apply kubectl patch after Helm install:
kubectl patch deployment signoz-otel-collector --type=json -p='[{"op":"replace","path":"/spec/template/spec/containers/0/args","value":["--config=/conf/otel-collector-config.yaml","--feature-gates=-pkg.translator.prometheus.NormalizeName"]}]'
Results:
✅ OTel Collector successfully receiving traces (97+ spans)
✅ Services connecting without UNAVAILABLE errors
✅ No ClickHouse authentication failures
✅ All pipelines active (traces, metrics, logs)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
2026-01-09 11:51:03 +01:00 |
|