Opentelemetry Filelog Example

Filelog receiver is used to collect logs from the server where otelcol-contrib is running. In otelcol-contrib v111, there are several bugs in transform and some other processors causing log processing to fail. Here is working example to extract and set service_name (as label in Grafana) from the log files following specif pattern. Logs are coming from different apps and in the server they are following the rule:

/some/path/log/services/<service_name/*.log

We will extract <service_name> from directory path and set it as resource.attribute. Loki and Grafana will use this is label and it will show up in Grafana’s drop down for filtering.

receivers:
  filelog:
    include:
      - ./logs/*/*.log
    start_at:
      beginning

    include_file_path: true

    operators:
      - type: regex_parser
        parse_from: attributes["log.file.path"]
        regex: '.*/(?P<service>.*)/.*log'
      - type: add
        field: resource.service_name
        value: 'EXPR(attributes["service"])'