Webhooks

Webhooks let you subscribe to certain events on your Tailscale network and process the event notifications through an integration or app. For example, you could integrate Tailscale events with a Slack channel. If subscribed to an event such as adding a node, your webhook endpoint can send a message in a Slack channel anytime a node is added to your tailnet.

How it works

You provide a webhook endpoint which can receive HTTPS POST requests for subscribed Tailscale events. The body of the request provides information about the event. It is up to you to determine how the webhook should process a notification. Tailscale typically sends an event notification to a webhook within a few seconds of the event's occurrence.

You create and manage webhook endpoints in the Webhooks page of the admin console.

Tailscale provides a digital signature for events that it sends, so you can verify whether an event was signed by a secret that is shared between you and Tailscale.

Webhook events are sent as JSON objects, with the format described in events payload. Optionally, you can configure webhooks to send events in a compatible format for the following destinations:

Prerequisites

• You need a webhook endpoint which will process the Tailscale event notifications.

• Your webhook endpoint must be able to process HTTPS POST requests and must use either port 80 or port 443.

• You need to be an Owner, Admin, Network admin, or IT admin of a tailnet in order to create, modify, or delete webhooks.

Setting up a webhook endpoint

Webhooks apply to a tailnet. If one user creates a webhook, other Owner, Admin, Network admin, or IT admin users in the tailnet can modify or delete it. If a webhook is created by a user who is later removed or suspended from your tailnet, the webhook will still work.

1. Open the Webhooks page of the admin console.

2. Select Add endpoint.

3. In the Add endpoint page:

   1. For Webhook URL, provide the endpoint for your webhook. The endpoint URL must use the HTTPS protocol, and must use either port 80 or port 443.

   2. (Optional) For Destination, select the destination for the endpoint. Tailscale will send your webhook events in the format expected by the destination. (If you enter a Discord or Slack URL for Webhook URL, then the Destination field will be grayed out with the respective destination selected.)

      

      Choose None if you are using the general Tailscale format for the events.

   3. Select the event categories or events you want to receive as notifications.

      A category is a set of related events. For example, the Tailnet Management category contains events related to node actions and tailnet policy file updates. If you select a category, you will be subscribed to any new events that Tailscale adds to the category in the future.

      If you don't want to use categories, you can select specific events.

      

   4. Select Add endpoint.

4. On the subsequent Webhook secret popup, select Copy to copy the newly-created webhook secret. After you close the Webhook secret page, you won't be able to copy the secret again. Also, note that Tailscale-generated webhook secrets are case-sensitive.

5. Select Done.

   Your webhook endpoint is now configured.

6. To ensure your webhook is configured correctly and can receive events from Tailscale, test your webhook.

Events

You can subscribe to the following events:

Events payload

Webhook events are sent as JSON objects with the following fields:

• timestamp: Time the event occurred, formatted as a RFC 3339 string.
• version: Version of the event payload (currently always 1).
• type: Type of the event (as listed in the table above).
• tailnet: Name of the tailnet where the event occurred.
• message: Human-readable summary of the event.
• data (Optional): Per-event payload with additional data. Most events will have an actor field that identifies the user or automated process that did the action that triggered the event. It is the same actor that is included in configuration audit log entries.

Multiple events may be sent in one payload to minimize overhead. The root payload object is always an array of events.

The following shows an example events payload sent to a webhook endpoint:

[
    {
        "timestamp": "2022-09-21T13:37:51.658918-04:00",
        "version": 1,
        "type": "test",
        "tailnet": "example.com",
        "message": "This is a test event",
        "data": null
    },
    {
        "timestamp": "2022-09-21T13:59:02.949217-04:00",
        "version": 1,
        "type": "nodeCreated",
        "tailnet": "example.com",
        "message": "Node alice-workstation1.yak-bebop.ts.net created",
        "data": {
            "nodeID": "nFJw3SRKTM59",
            "deviceName": "alice-workstation1.yak-bebop.ts.net",
            "managedBy": "alice@example.com",
            "actor": "alice@example.com",
            "url": "https://login.tailscale.com/admin/machines/100.12.345.67"
        }
    },
    {
        "timestamp": "2022-09-21T13:59:02.949278-04:00",
        "version": 1,
        "type": "nodeNeedsApproval",
        "tailnet": "example.com",
        "message": "Node alice-workstation1.yak-bebop.ts.net needs approval",
        "data": {
            "nodeID": "nFJw3SRKTM59",
            "deviceName": "alice-workstation1.yak-bebop.ts.net",
            "managedBy": "alice@example.com",
            "actor": "alice@example.com",
            "url": "https://login.tailscale.com/admin/machines/100.12.345.67"
        }
    },
    {
        "timestamp": "2022-09-21T13:59:15.966728-04:00",
        "version": 1,
        "type": "nodeApproved",
        "tailnet": "example.com",
        "message": "Node alice-workstation1.yak-bebop.ts.net approved",
        "data": {
            "nodeID": "nFJw3SRKTM59",
            "deviceName": "alice-workstation1.yak-bebop.ts.net",
            "managedBy": "alice@example.com",
            "actor": "admin@example.com",
            "url": "https://login.tailscale.com/admin/machines/100.12.345.67"
        }
    },
    {
        "timestamp": "2023-04-21T13:59:15.966728-04:00",
        "version": 1,
        "type": "nodeDeleted",
        "tailnet": "example.com",
        "message": "Node alice-workstation1.yak-bebop.ts.net deleted",
        "data": {
            "nodeID": "nFJw3SRKTM59",
            "deviceName": "alice-workstation1.yak-bebop.ts.net",
            "managedBy": "alice@example.com",
            "actor": "admin@example.com",
            "url": "https://login.tailscale.com/admin/machines/100.12.345.67"
        }
    },
    {
        "timestamp": "2022-09-27T09:51:46.512946-07:00",
        "version": 1,
        "type": "policyUpdate",
        "tailnet": "example.com",
        "message": "Tailnet policy file updated",
        "data": {
            "newPolicy": "{\n\t\"acls\": [\n\t\t{\"action\": \"accept\", \"src\": [\"autogroup:member\"], \"dst\": [\"*:*\"]},\n\t],\n}",
            "oldPolicy": "{\n\t\"acls\": [\n\t\t{\"action\": \"accept\", \"src\": [\"*\"], \"dst\": [\"*:*\"]},\n\t],\n}",
            "url": "https://login.tailscale.com/admin/acls",
            "actor": "admin@example.com"
        }
    },
    {
        "timestamp": "2022-11-08T10:26:08.775392-08:00",
        "version": 1,
        "type": "nodeKeyExpiringInOneDay",
        "tailnet": "example.com",
        "message": "Node alice-workstation1.yak-bebop.ts.net key expiring in less than one day",
        "data": {
            "nodeID": "nFJw3SRKTM59",
            "url": "https://login.tailscale.com/admin/machines/100.12.345.67",
            "deviceName": "alice-workstation1.yak-bebop.ts.net",
            "managedBy": "alice@example.com",
            "actor": "expiring-node-key-marker",
            "expiration": "2022-11-08T18:44:46.979292Z"
        }
    },
    {
        "timestamp": "2022-11-08T10:45:08.775392-08:00",
        "version": 1,
        "type": "nodeKeyExpired",
        "tailnet": "example.com",
        "message": "Node alice-workstation1.yak-bebop.ts.net key recently expired",
        "data": {
            "nodeID": "nFJw3SRKTM59",
            "url": "https://login.tailscale.com/admin/machines/100.12.345.67",
            "deviceName": "alice-workstation1.yak-bebop.ts.net",
            "managedBy": "alice@example.com",
            "actor": "expiring-node-key-marker",
            "expiration": "2022-11-08T18:44:46.979292Z"
        }
    },
    {
        "timestamp": "2023-02-27T11:49:25.208092-08:00",
        "version": 1,
        "type": "userRoleUpdated",
        "tailnet": "example.com",
        "message": "User alice@example.com role changed",
        "data": {
            "user": "alice@example.com",
            "url": "https://login.tailscale.com/admin/users?q=alice%40example.com",
            "actor": "admin@example.com",
            "oldRoles": ["Member"],
            "newRoles": ["Member", "IT admin"]
        }
    }
]

Testing your webhook

To ensure an endpoint is properly configured and able to receive events from Tailscale, you can send a test event:

1. Open the Webhooks page of the admin console.

2. Find the endpoint that you want to test, select the menu to the right of the page, and select Test endpoint.

3. Select Send test event. If your webhook is configured correctly, within a few seconds your webhook endpoint should receive an event with type of "test".

Retries for events that fail to send

Tailscale typically sends an event notification to a webhook within a few seconds of the event's occurrence. If an event notification fails to successfully send (such as when Tailscale receives a 3xx, 4xx, or 5xx error, or no response at all from your webhook endpoint), Tailscale will retry sending the event. For an event that fails to send, we'll retry sending the event hourly, up to a maximum of 24 hours.

Updating subscribed events

1. Open the Webhooks page of the admin console.

2. Find the endpoint that you want to update, select the menu to the right of the page, and select Edit.

3. Select the events you want to receive as notifications, and unselect those you don't want to receive.

4. Select Edit endpoint.

Webhook secret

The webhook secret is a signing secret shared between Tailscale and the creator of the webhook endpoint, and is unique per endpoint. If this shared secret is compromised or leaked, whomever knows the secret can send fake events. If you suspect your secret is compromised, create a new secret.

The webhook secret has no expiry.

Rotating a webhook secret

When you rotate a webhook secret, the old secret will no longer work.

1. Open the Webhooks page of the admin console.

2. Find the endpoint that you want to have a new secret, select the menu to the right of the page, and select Rotate webhook secret.

3. Select Rotate to confirm you want a new secret.

4. In the Rotate webhook secret page, select Copy to copy the new secret. After you close the Rotate webhook secret page, you won't be able to copy the secret again.

5. Select Done.

The new secret will be used for any new events that are sent—you need to update your signature verification routine to use the new secret.

Deleting an endpoint

1. Open the Webhooks page of the admin console.

2. Find the endpoint that you want to delete, select the menu to the right of the page, and select Delete.

3. Select Delete endpoint to confirm you want to delete the endpoint.

Using a new webhook for an existing endpoint

To add a new webhook (subscribed event) for an existing endpoint, edit the endpoint instead of setting up a new endpoint.

Audit logging of webhook actions

In configuration audit logging, an action will be recorded in your audit log whenever a webhook is created, deleted, or updated. The log entry will show who performed the action, and when the action occurred.

Verifying an event signature

You can verify whether an event was signed by the webhook secret that was shared between you and Tailscale. Note this doesn't necessarily mean that an event was sent from Tailscale. Rather, it means an event was sent from an entity that has knowledge of the secret shared between you and Tailscale.

A event sent from Tailscale contains a Tailscale-Webhook-Signature header. The Tailscale-Webhook-Signature header includes a timestamp and a signature:

Tailscale-Webhook-Signature:t=1663781880,v1=0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef

The timestamp, prefixed by t=, is the epoch time in seconds when the event occurred. The signature, prefixed by v1=, is a hash-based message authentication code (HMAC) using SHA-256. The only supported scheme for the signature is v1.

Most modern programming languages provide libraries for computing and comparing HMACs. The following flow describes how to verify a signature.

1. Parse the event timestamp and signature from the Tailscale-Webhook-Signature header.

   Using the , character as the separator, split the Tailscale-Webhook-Signature data into a list of elements. Then, using the = character, split each element to get two key-value pairs. The first key is t, with the event timestamp as its value. The second key is v1, with the event signature as its value.

2. Compare the event timestamp (the value for t) with the current time. For example, if your verification process acts on events as soon as they are received, and if the event time is more than 5 minutes prior to the current time, you might consider the event as a replay attack.

3. Create a string, string_to_sign, to sign by concatenating:

   • The timestamp (the value of t) represented as a string.
   • The . character.
   • The decoded event request body. Note this is in the request itself, not in the Tailscale-Webhook-Signature header. The request body contains the encoded events payload, you need to decode the request body for signing purposes.

4. Compute the signature of string_to_sign.

   Create an HMAC with the SHA256 hash function. Use your webhook secret for the signing key, and use string_to_sign as the message to sign.

5. Use an HMAC compare function to compare the signature in the Tailscale-Webhook-Signature header (the value of v1) with the signature you created in the previous step. If they are not identical, that indicates the event's payload was not signed by your webhook's secret, and the event should not be considered an event sent from Tailscale.

Sample verification code

Check out our example Go code for help verifying the signature.