Collecting Kubernetes pod logs

SigNoz can automatically collect all your pod logs and you can perform various action on top of that data.

Collect Kubernetes Pod Logs in SigNoz Cloud

To collect logs from your kubernetes cluster, you will need to deploy k8s-infra chart. Please follow the guide here. Log collection of pods from all namespaces is enabled by default except for pods in kube-system and hotrod. To modify the log collection mechanism, please follow the guides below.

Collect Kubernetes Pod Logs in Self-Hosted SigNoz

When you deploy SigNoz to your kubernetes cluster it will automatically start collecting all the pod logs. It will automatically parse out different attributes from the logs like name, namespace, container name, uid etc. But if you want to parse specific attributes from certain kind of logs you can use different kinds of operators provided by OpenTelemetry here.

If your signoz cluster is hosted in a different cluster then you will have to install k8s-infra chart on your application kubernetes cluster. Please follow the guide here. Log collection of pods from all namespaces is enabled by default except for pods in kube-system and hotrod. To modify the log collection mechanism, please follow the guides below.

Steps to disable automatic pod logs collection

  • Modify/Create the override-values.yaml file

    k8s-infra:
      presets:
        logsCollection:
          enabled: false
    

    You can apply this yaml file by running the following command:

    helm -n platform upgrade my-release signoz/signoz -f override-values.yaml
    

    In case of external K8s cluster where only k8s-infra chart is installed, users can disable log collections by including the following in override-values.yaml :

    presets:
      logsCollection:
        enabled: false
    

    You can apply this yaml file by running the following command:

    helm -n platform upgrade my-release signoz/k8s-infra -f override-values.yaml
    

    Once the above is applied to your k8s cluster, logs collection will be disabled.

Steps to Filter/Exclude/Include Logs Collection

  • Exclude certain log files : If you want to exclude logs of certain namespaces, pods or containers, you can append the following config in your Helm override values file.

    override-values.yaml

    k8s-infra:
      presets:
        logsCollection:
          # whether to enable log collection
          enabled: true
          blacklist:
            # whether to enable blacklisting
            enabled: true
            # whether to exclude signoz logs
            signozLogs: false
            # which namespaces to exclude
            namespaces:
              - kube-system
            # which pods to exclude
            pods:
              - hotrod
              - locust
            # which containers to exclude
            containers: []
            # additional exclude rules
            additionalExclude: []
    
  • Include certain log files only : If you want to only include logs of certain namespaces, pods or containers, you can append the following config in your Helm override values file.

    override-values.yaml

    k8s-infra:
      presets:
        logsCollection:
          # whether to enable log collection
          enabled: true
          whitelist:
            # whether to enable whitelisting
            enabled: true
            # whether to include signoz logs
            signozLogs: false
            # which namespaces to include
            namespaces:
              - platform
              - my-application-namespace
            # which pods to include
            pods:
              - otel  # all pods with otel prefix
              - my-application-pod
            # which containers to include
            containers: []
            # additional include rules
            additionalInclude: []
    
  • Using filter operator in filelog receiver : You can also use the filter operator to filter out logs by changing the operators here charts.

    ....
      operators:
        - type: filter
          expr: 'body matches "^LOG: .* END$"'
          drop_ratio: 1.0
    ....
    

    Here we are matching logs using an expression and dropping the entire log by setting drop_ratio: 1.0 . You can read more about the filter operator here

  • Now you can restart the otel collector pod so that new changes are applied.

Was this page helpful?