Logs Query Builder

We recently released an updated logs explorer page and query builder in SigNoz to make experience of our logs product much more intuitive and seamless.

Some of the key features:

  1. Advanced filtering based on attributes and auto suggestions for filters

    You can now create more complex queries for how you match attributes, and the query field will automatically suggest both attributes and values for your query.

    Logs Query Builder Interface

    After entering your query hit 'run query' to see a default bar chart and results down below:

    Logs Query Builder Interface

    To explore additional ways to filter your query, click 'view details' on any log line to get a list of parameters on the log, and click any value to automatically add a {attribute} IN {value} to your query.

    Logs Query Builder Interface
  2. Aggregation options like Group By, ability to specificy aggregation intervals, etc.

    The Group By drop down is automatically populated with common attributes, and selecting one, like log level in this example, gets us a few things right away

    • A comparitive bar chart, with a legend (you can enter a format for the legend including explanatory text eg in this case you might enter The log level is {{level}} to make the chart more readable by others)

    • A timeseries comparing the relative volume of results by time

    Logs Query Builder Interface
  3. Plot multiple queries and formulae based on them in the same charts

    For comparing two timeseries or values, you can now add multiple queries or formulae on the same chart, which is especially useful if you are comparing similar data across two different services.

  4. Modify your query with a click

    By clicking on the details of any log line, you can see the attributes for that item. Click any value, and your query will be modified to require that attribute and value.

    Logs Query Builder Interface
  5. Create Dashboards and Alerts in a single click from logs query builder

    Directly from results you can add the query to a dashboard and set up an alert. This makes the timeseries view especially useful, as you can now create an alert when a certain event is logged beyond a certain rate.

    Logs Query Builder Interface

With the alert query you can perform sophisticated comparisons or other math on your measurement from your logs.

Writing JSON Filters In The New Logs Explorer

In the new logs explorer, you can query your JSON data present inside the body. JSON data in the body will be rendered like this:
JSON Data in log body
You can click on one of the keys and then filter them out
JSON Data filter

If you want to write the query on your own in the filter bar then you can follow the following rules.

  • For json query inside body it starts with a prefix body.
  • To access a value of a key you can use the notation body.key_name
  • If it is nested key then use . to signify nested keys like body.level1.level2.key
  • If the type of value is array use [*]
  • operators supported for arrays are HAS and NHAS

Example for JSON filter

Lets say we have this example data in your json log body

{
  "message": "hello",
  "request": {
    "services": [
      "service_1",
      "service_2"
    ],
    "service_meta": [
      {
        "name": "service_1",
        "latency": 101,
        "tags": [
          "tag1",
          "tag2"
        ]
      },
      {
        "name": "service_2",
        "latency": 200,
        "tags": [
          "tag1",
          "tag2"
        ]
      }
    ]
  }
}
  • logs where value of message is hello

    body.message = hello
    
  • logs where value of message is contains he

    body.message CONTAINS he
    
  • logs where value of latency of service_1 is >100

    body.request.service_meta[*].latency > 100
    
  • logs where tag1 is present in service tags

    body.request.service_meta[*].tags[*] HAS tag1
    
  • logs where tag2 is not present in service tags

    body.request.service_meta[*].tags[*] NHAS tag2
    

Logs Query Builder in old Logs Explorer

This section will walk you through the query language that is used by SigNoz for filtering logs in the old logs explorer.

This query language for logs is a simplified version of SQL. The current state of the query language is good enough for daily uses. As of now we don't support nesting and parenthesis for explicit ordering due to added complexity.

If you have a use case which you are not able to fullfill with the current features please reach out to us on our slack community or Github issues. We plan to improve the query as we go forward while keeping it simple

Types of queries supported by SigNoz:

  • Freehand query
    When a user writes a plan text query without using any kind of operators, the query is directly searched against the log body. ( inefficient over huge log data)

    eg:-Dispatch Successful

    Freehand
  • Filtering queries
    When a user writes queries using a key, operator and a value separated by and , or operators it is a filtering query. This kind of queries are faster as they reduce the search space by using indexes.

    eg:- id IN ('2DCVZOsKHioCeuvbObzVzzL1eZ5') AND fulltext contains 'Dispatch Successful'

    Filtering

List of Operators supported by SigNoz

  • Here is a list of all the operators that are supported:

    OperatorMultivaluedExamplesMeaning
    INyesnum in (1,2,3)
    strdata in ('a', 'b', 'c')
    In
    NINyesnum nin (1,2,3)
    strdata nin ('a', 'b', 'c')
    Not In
    GTEnonum gte 10
    dict_word gte 'cat'
    Greater than or Equal to
    GTnonum gt 10
    dict_word gt 'cat'
    Greater than
    LTEnonum lte 10
    dict_word lte 'cat'
    Less than or Equal to
    LTnonum lt 10
    dict_word lt 'cat'
    Less than
    CONTAINSnostream contains 'err'Contains
    NCONTAINSnostream ncontains 'err'Doesn't Contain

Fulltext Key

The fulltext key is used when we want to write freehand queries and combine them with filters.

eg:-

id IN ('2DCVZOsKHioCeuvbObzVzzL1eZ5') AND fulltext contains 'Dispatch Successful'

In this cases we are searching Dispatch Successful as fulltext along with the id filter.

πŸ“ Note

The fulltext keyword can be only used with contains and the ncontains operator.

Pointers to note while writing queries

  • Text always needs to be enclosed in single quotes in filtering queries

    eg:- If you want to search for logs with stream error and which contains Mozilla in body, the corresponding query on the ui will be

    stream IN ('stderr') AND fulltext contains 'Mozilla'
    

    as we can see Mozilla is enclosed in single quotes as well as stderr.

  • Order of execution is similar to sql i.e left to right and and has higher precedence over or , but currently SigNoz doesn’t support combining explicitly using parenthesis.

    correct βœ… :-

    stream IN ('stdout') and fulltext contains 'Mozilla' or stream IN ('stderr')
    

    incorrect ❌ :-

    (stream IN ('stdout') and fulltext contains 'Mozilla') or stream IN ('stderr')
    

    While the above to evaluates to the same expression, it’s not necessarily same for the one below

    correct βœ… :-

    stream IN ('stdout') and fulltext contains 'Mozilla' or stream IN ('stderr'
    

    incorrect ❌ :-

    stream IN ('stdout') and (fulltext contains 'Mozilla' or stream IN ('stderr'))
    

Here both the statements are not equivalent of each other i.e it is currently not supported

Query Examples

Here are a few examples of valid and invalid queries:

  • Valid Queries

    QueryDescription
    OPERATION in ('add') AND FULLTEXT contains 'search string'fulltext with filtering query
    my fulltext searchfulltext search query
    status gte 200 AND FULLTEXT contains 'search string'fulltext with filtering query
    service IN ('name>100') AND length GT 100filtering query
    service IN ('name > 100') AND name GT 'myname'filtering query
    hello in 2fulltext search query
    hello in (2,3)filtering query
    hello lt 2filtering query
  • Invalid Queries

    QueryDescription
    OPERATION in ('bcd') AND 'abcd' FULLTEXT contains 'search string'AND missing between 'abcd' and FULLTEXT
    OPERATION in ('bcd') AND 'search string'Operator missing before 'search string'

Was this page helpful?