Filter logs by header

Use of Aperture is governed by the Aperture Beta Terms.

Aperture records the request and response headers of every capture, but by default those headers are not filterable. Instead, they are stored inside the capture as compressed data that cannot be queried. This guide shows you how to index specific headers so you can filter the dashboard Logs page by their presence or value, for example to find every request that carried a given x-request-id or came through a particular proxy.

You configure header filtering entirely through the Aperture configuration, with no code changes required.

Prerequisites

Before you begin, ensure you have the following:

• An Aperture instance with at least one provider configured.
• Admin access to the Aperture configuration, including either the Database settings page or the raw configuration editor.

The configuration is written in HUJSON, which is JSON that also permits comments and trailing commas. Standard JSON is valid HUJSON.

How header filtering works

Header filtering has two parts. The filterable list that decides which headers Aperture indexes, and the filter controls on the Logs page that query those indexes.

The list is the gate. Each header you add becomes a queryable index on that header's name and values. Every other header remains part of the capture and is queryable only through the capture itself. The list starts empty, so Aperture builds no header indexes until you add a header. Until then, Aperture stores no header index data and runs no header queries.

Indexing a header does not by itself let anyone filter on it. A caller, including an admin, can use an indexed header as a Logs filter only when a grant assigns them that header through the filterable_headers capability.

A header index is subject to the same data retention policy as captures: when a capture is purged, its indexed headers are removed too. When you add a header, it becomes filterable for new requests immediately, and Aperture backfills history from captures that are still within the capture retention window. Requests whose captures have already been purged cannot be backfilled.

Header values are scrubbed before storage as a safeguard against durably storing a secret. Aperture always redacts values that match a built-in set of credential patterns, and this set cannot be extended. To keep a header's value out of storage entirely, mark it presence-only. Refer to Store a header as presence-only.

To make the steps concrete, suppose you want to trace a single request across your logs. Every call that carried the x-request-id your gateway assigned it. The steps below index that header, grant access to it, then filter the Logs page down to the requests that match.

Step 1: Choose the headers to index

Decide which headers you want to filter on. Good candidates are low-cardinality, stable headers that identify or classify a request, such as user-agent, a routing or trace identifier, or a header your gateway adds. Avoid per-request unique headers such as date, which are not useful as filters.

Step 2: Add the headers to your configuration

Open the Aperture dashboard and go to Administration > Configuration.

"database": {
  "headers": {
    "filterable": [
      { "name": "user-agent" },
      { "name": "x-request-id" },
    ],
  },
}

Header names are case-insensitive and apply to both the request and response hops. After you save, Aperture indexes the listed headers for new requests and backfills recent history.

Step 3: Grant filter access

Indexing a header does not let anyone filter on it, not even an admin, until a grant assigns it. In the grant that covers the users who should filter logs, add a filterable_headers capability listing the header names:

"grants": [
  {
    "src": ["*"],
    "app": {
      "tailscale.com/cap/aperture": [
        { "filterable_headers": ["user-agent", "x-request-id"] },
      ],
    },
  },
]

Use ["*"] to grant every indexed header. The granted headers appear as options in the Header filter on the Logs page for those users. For the full grant structure, refer to grants.

Step 4: Store a header as presence-only (optional)

By default Aperture stores each indexed header's value, after redaction, so you can match on a substring or an exact value. To index a header for presence checks only and never store its value, set redact_values on that header:

"database": {
  "headers": {
    "filterable": [
      { "name": "user-agent" },
      { "name": "authorization", "redact_values": true },
    ],
  },
}

A presence-only header supports the Present match only. Contains and Is exactly are unavailable for it. Use it for headers whose values are sensitive or too high-cardinality to store durably.

Step 5: Filter the Logs page

On the Logs page, select Filter + and choose Header to add a header filter. For each header filter, set:

• The hop: Req to match the request header, Resp to match the response header, or Any to match either.
• The header name, chosen from the headers you are granted.
• The match: Present (the header exists), Contains (the value contains a substring), or Is exactly (the value equals the string you enter). Contains and Is exactly match case-insensitively, and are unavailable for presence-only headers.

The filter pill summarizes your choice, reading Req x-request-id contains abc, Req x-request-id is exactly abc, or Any x-request-id present.

To match one header against several values, select + header within the filter to add an OR condition. The pill joins the conditions with OR, for example Req x-request-id contains abc OR Req x-request-id contains def.

To narrow further, add more Header filters. Conditions within a single pill combine with OR. Separate Header filters, together with any other active filters such as user or date, combine with AND. A request must match every pill to appear.

Reference

For the full field descriptions, refer to the database.headers configuration reference.