Network flow logs
Network flow logs let you understand which nodes connected to which other nodes, and when, on your Tailscale network. You can export network logs for long-term storage and/or for security analysis, threat detection, and incident investigation. You can also stream logs to a security information and event management (SIEM) system.
The data captured and viewable in network logs is the flow of network traffic, not the contents of network traffic. Tailscale does not and cannot inspect your traffic. For more information about how your data stays private, see our Security page.
Network flow logs are available for the most recent 30 days.
How it works
When Network flow logs are enabled and clients are running a sufficient client version, and when not using
--no-logs-no-support
, nodes report their flow information to the Tailscale logs service. Logs
are reported by both ends of the connection. A connection can be between:
- two nodes.
- a node and a device behind a subnet router. The connection is logged by the subnet router.
- a node and a public device accessed via an exit node. The connection is logged by the exit node.
Logs occur between both transferred and received network connections, and entries exist for both ends.
Logs are gathered at two different layers: virtual and physical. The virtual layer is the Tailscale network (often with IP addresses in 100.x.x.x) that Tailscale provides for each machine. The physical layer involves all the physical network interfaces that exist on a machine. When a packet is transmitted from one node to another, it must cross through the physical layer. The summation of all traffic routed at the virtual layer is generally equal to the traffic routed at the physical layer.
Network logs are centrally stored by Tailscale and can be exported. Tailscale does not log the actual contents or data of the network traffic.
Network logs structure
Network logs contain an array of messages that provide details about network flow activity. Each message contains the following components:
- nodeId: The node that generated this log message. It is a globally-unique opaque identifier that identifies a node.
It is not the same value as the node name. If you want to map
nodeId
to a node name, you can use the/api/v2/device/:nodeId
method and examine thename
field. - logged: The time in UTC that the Tailscale logs service recorded this message. Generally, the logged time is after the end time within a message.
- start: The start time in UTC for the network traffic flow data in this message, as recorded by the node that generated this message.
- end: The end time in UTC for the network traffic flow data in this message, as recorded by the node that generated this message.
- virtualTraffic: Counters for the network flow’s virtual traffic. Virtual traffic is traffic that occurs between nodes on your tailnet.
- subnetTraffic: Counters for the network flow’s traffic routed through an advertised subnet router.
- exitTraffic: Counters for the network flow’s traffic routed through an exit node. For traffic from a node to a public device via an exit node, the source will be the Tailscale IP address, but the protocol, source port, and destination will be empty. For traffic responses from a public device to a node via an exit node, the destination will be the Tailscale IP address, but the protocol, destination port, and source will be empty. Fine granularity information about individual connections is not gathered so that privacy can be preserved.
- physicalTraffic: Counters for the network flow’s traffic on the physical network layer that operates below the virtual Tailscale network. Traffic information at the physical layer is gathered at a slightly different moment in time as the virtual layer, so packets flowing through the virtual layer may not exactly line up with those at the physical layer.
Start and end times are specific to the node that generated the message. Timestamps may be subject to clock skew across different nodes.
The virtual, subnet, exit, and physical traffic counters consist of the following components:
- proto: The IANA Protocol Number for the network flow: 6 for TCP, 17 for UDP. Empty for exit traffic.
- src: The Tailscale IP address and port for the source of the network flow. Port is not included for exit traffic.
- dst: The IP address and port for the destination of the network flow. This IP address can be either a Tailscale IP address or an external IP address, such as on a private network, or a public IP. Empty for exit traffic.
- txPackets: Number of packets transmitted.
- txBytes: Number of bytes transmitted.
- rxPackets: Number of packets received.
- rxBytes: Number of bytes received.
Enabling Network flow logs
By default, Network flow logs are not enabled.
You need to be an Owner, Admin, Network admin, or IT admin of a tailnet in order to enable Network flow logs.
- Open the Network flow logs page of the admin console.
- Click the Start logging button.
- In the Start logging network flows dialog, click the Start logging button.
Disabling Network flow logs
You need to be an Owner, Admin, Network admin, or IT admin of a tailnet in order to disable Network flow logs.
- Open the Network flow logs page of the admin console.
- Click the Stop logging button.
- In the Stop logging network flows dialog, click the Stop logging button.
Accessing network logs
You need to be an Owner, Admin, Network admin, or IT admin of a tailnet in order to access network logs.
Network logs can be accessed by using the Tailscale API, or by log streaming.
Accessing network logs via API
You need an API access token with the network-logs:read
scope to access network logs.
The response from the https://api.tailscale.com/api/v2/tailnet/{$TAILNET_ID}/network-logs
endpoint call is in
the form of the Response
struct:
type Response struct {
Logs []Message `json:"logs"`
}
Using Go syntax, the Response
struct contains a slice of Message
types. The Message
struct is defined as:
type Message struct {
// NodeID is the stable ID of the node that
// generated this network log message.
NodeID string `json:"nodeId"` // e.g., "n123456CNTRL"
// Logged is the timestamp of when the Tailscale logs service
// recorded the network log message from a given node.
// It is guaranteed to be within the start and end time ranges
// specified in the API request.
// All log messages are listed in chronological order
// from oldest to newest.
Logged time.Time `json:"logged"`
// Start and End are the inclusive time ranges for the network
// traffic flow information present in this message.
// These timestamps are recorded by the node and subject
// to clock skew across different nodes.
// Generally speaking, the Logged timestamp is after End.
//
// Network logs are gathered in 5 second windows.
// This may change in the future.
Start time.Time `json:"start"`
End time.Time `json:"end"`
// VirtualTraffic records connection statistics for
// node to node traffic.
// Both the source and address are Tailscale IP addresses
// (e.g., 100.xx.xx.xx). The source is always the
// Tailscale IP address of the current node.
VirtualTraffic []ConnectionCounts `json:"virtualTraffic"`
// SubnetTraffic records node to external traffic
// on an explicitly advertised subnet route.
//
// For nodes using a subnet router,
// the source is the Tailscale IP address of the current node.
// For nodes operating as the subnet router,
// the source is the Tailscale IP address of the node
// using the subnet router.
// The destination address is always the external IP address
// within the advertised subnet range.
SubnetTraffic []ConnectionCounts `json:"subnetTraffic"`
// ExitTraffic records aggregated statistics for all traffic
// flowing through an exit node. For traffic from a node to a
// public device via an exit node, the source will be the
// Tailscale IP address, but the protocol, source port,
// and destination will be empty. For traffic responses from a
// public device to a node via an exit node, the destination
// will be the Tailscale IP address, but the protocol,
// destination port, and source will be empty. Fine
// granularity information about individual connections is not
// gathered so that privacy can be preserved.
ExitTraffic []ConnectionCounts `json:"exitTraffic"`
// PhysicalTraffic records traffic on the physical network layer
// that operates below the virtual Tailscale network.
// The source is the Tailscale IP address of remote nodes
// that the current node is communicating with and
// the destination is the external IP address that traffic
// is physically sent to in order to communicate with that
// remote node.
//
// Traffic information at the physical layer is gathered
// at a slightly different moment in time as the virtual layer,
// so packets flowing through the virtual layer
// may not exactly line up with those at the physical layer.
PhysicalTraffic []ConnectionCounts `json:"physicalTraffic"`
}
Several fields in the Message
struct use the type ConnectionCounts
. The ConnectionCounts
struct is
defined as:
type ConnectionCounts struct {
Proto uint8 `json:"proto"` // e.g., 6 for TCP, 17 for UDP
Src string `json:"src"` // e.g., "100.11.22.33:4567"
Dst string `json:"dst"` // e.g., "192.555.66.77:80"
TxPackets uint64 `json:"txPkts"` // transferred packets
TxBytes uint64 `json:"txBytes"` // transferred bytes
RxPackets uint64 `json:"rxPkts"` // received packets
RxBytes uint64 `json:"rxBytes"` // received bytes
}
The Message.NodeID
field is verified by the Tailscale logs service as the actual node from which the message originated. The Start
, End
, VirtualTraffic
, SubnetTraffic
, ExitTraffic
, and
PhysicalTraffic
fields are produced by individual nodes and recorded by the Tailscale logs service without
validation. It is infeasible for Tailscale to verify the accuracy or truthfulness of this information. It is
possible for malicious nodes to spoof this information.
When investigating network flow logs, you should identify a set of nodes that are considered more trustworthy (such as a server running in production) in contrast to those that may be more easily tampered with (such as an individual employee’s work laptop). Discrepancies in network logs between the two may be indicative of malicious behavior.
You can use the following query parameters with the API:
start
: Required. Start of the timeframe, in RFC3339 timestamp format, for the logs to retrieve. For example:2022-07-20T00:00:00Z
.end
: Required. End of the timeframe, in RFC3339 timestamp format, for the logs to retrieve. Ifend
is greater than the latest known timestamp in the log, the API call will not block the call. This means consecutive queries with the samestart
andend
times range may return different log entries that were not available during the earlier query.
start
and end
times are inclusive within nanosecond resolution.
All log messages are listed in chronological order from oldest to newest.
Currently, there is no pagination support and no maximum page size for the API. All known logged network activity in the specified timeframe is returned.
Example API call
This example assumes you have set up the following variables to use for your API call:
$ACCESS_TOKEN
: An API access token to use when calling the Tailscale API. You can create an API access token in the Keys page of the admin console.$TAILNET_ID
: The organization name for the tailnet whose logs are being retrieved. You can view your organization name in the General settings page of the admin console.$START
: The start of the timeframe for the logs to retrieve.$END
: The end of the timeframe for the logs to retrieve.
export ACCESS_TOKEN=tskey-api-k123456CNTRL-0123456789abcdef
export TAILNET_ID=example.com
export START=2022-10-28T22:40:00.000000000Z
export END=2022-10-28T22:40:04.999999999Z
This example makes a call to the https://api.tailscale.com/api/v2/tailnet/{$TAILNET_ID}/network-logs
endpoint.
curl -u $ACCESS_TOKEN: \
"https://api.tailscale.com/api/v2/tailnet/{$TAILNET_ID}/network-logs?start={$START}&end={$END}"
(You can also copy the command above from the Network flow logs page of the admin console.)
The output will look like:
{"logs": [{
"nodeId": "aBcdef1CNTRL",
"logged": "2022-10-28T22:40:00.290605382Z",
"start": "2022-10-28T22:39:51.890385065Z",
"end": "2022-10-28T22:39:56.886545512Z",
"virtualTraffic": [{
"proto":6, "src":"100.111.22.33:21291", "dst":"100.111.44.55:63281",
"txPkts":2, "txBytes":108, "rxPkts":2, "rxBytes":112
}, {
"proto":6, "src":"100.111.22.33:864", "dst":"100.44.55.66:2049",
"txPkts":6, "txBytes":900, "rxPkts":3, "rxBytes":728
}, {
"proto":6, "src":"100.111.22.33:723", "dst":"100.99.888.77:2049",
"txPkts":4, "txBytes":596, "rxPkts":2, "rxBytes":432
}, {
"proto":6, "src":"100.111.22.33:21291", "dst":"100.111.44.55:63280",
"txPkts":2, "txBytes":108, "rxPkts":2, "rxBytes":112
}],
"physicalTraffic": [{
"src":"100.111.44.55:0", "dst":"192.555.66.77:41641",
"txPkts":4, "txBytes":384, "rxPkts":4, "rxBytes":384
}, {
"src":"100.44.55.66:0", "dst":"192.168.0.101:41641",
"txPkts":6, "txBytes":1136, "rxPkts":3, "rxBytes":848
}, {
"src":"100.99.888.77:0", "dst":"143.110.111.222:41641",
"txPkts":4, "txBytes":752, "rxPkts":2, "rxBytes":512
}]
}, {
"nodeId": "uvwXyz2CNTRL",
"logged": "2022-10-28T22:40:00.344979725Z",
"start": "2022-10-28T22:39:53.286643402Z",
"end": "2022-10-28T22:39:58.286028244Z",
"virtualTraffic": [{
"proto":6, "src":"100.44.55.66:49284", "dst":"100.99.888.77:22",
"txPkts":5, "txBytes":440, "rxPkts":10, "rxBytes":1180
}, {
"proto":6, "src":"100.44.55.66:49282", "dst":"100.99.888.77:22",
"txPkts":5, "txBytes":440, "rxPkts":10, "rxBytes":1180
}, {
"proto":6, "src":"100.44.55.66:49286", "dst":"100.99.888.77:22",
"txPkts":5, "txBytes":440, "rxPkts":9, "rxBytes":968
}, {
"proto":6, "src":"100.44.55.66:49278", "dst":"100.99.888.77:22",
"txPkts":5, "txBytes":440, "rxPkts":10, "rxBytes":1180
}, {
"proto":6, "src":"100.44.55.66:37500", "dst":"100.66.77.88:22",
"txPkts":46, "txBytes":7416, "rxPkts":68, "rxBytes":12612
}, {
"proto":6, "src":"100.44.55.66:49288", "dst":"100.99.888.77:22",
"txPkts":64, "txBytes":6236, "rxPkts":104, "rxBytes":10412
}, {
"proto":6, "src":"100.44.55.66:49280", "dst":"100.99.888.77:22",
"txPkts":5, "txBytes":440, "rxPkts":10, "rxBytes":1180
}, {
"proto":6, "src":"100.44.55.66:2049", "dst":"100.111.22.33:864",
"txPkts":3, "txBytes":728, "rxPkts":6, "rxBytes":900
}, {
"proto":1, "src":"100.44.55.66:0", "dst":"100.99.888.77:0",
"txPkts":5, "txBytes":420, "rxPkts":5, "rxBytes":420
}],
"physicalTraffic": [{
"src":"100.33.444.55:0", "dst":"98.97.111.222:2705",
"txPkts":1, "txBytes":32
}, {
"src":"100.111.22.33:0", "dst":"192.168.0.102:41641",
"txPkts":3, "txBytes":848, "rxPkts":6, "rxBytes":1136
}, {
"src":"100.99.888.77:0","dst":"143.110.111.222:41641",
"txPkts":94, "txBytes":12672, "rxPkts":158, "rxBytes":23104
}, {
"src":"100.66.77.88:0", "dst":"64.71.111.222:41641",
"txPkts":46, "txBytes":9296, "rxPkts":68, "rxBytes":15392
}]
}]}
Optionally, use the Go binary netlogfmt
to make the output more readable:
curl -u $ACCESS_TOKEN: \
"https://api.tailscale.com/api/v2/tailnet/{$TAILNET_ID}/network-logs?start={$START}&end={$END}" \
| go run tailscale.com/cmd/netlogfmt@main
The output will look like:
===========================================================================================
NodeID: aBcdef1CNTRL
Logged: 2022-10-28 15:40:00.290
Window: 2022-10-28 15:39:51.890 (4.996s)
----------------------------------------------------- Tx[P/s] Tx[B/s] Rx[P/s] Rx[B/s]
VirtualTraffic: 2.80 342.66 1.80 277.01
TCP: 100.111.22.33:864 -> 100.44.55.66:2049 1.20 180.14 0.60 145.71
TCP: 100.111.22.33:723 -> 100.99.888.77:2049 0.80 119.29 0.40 86.47
TCP: 100.111.22.33:21291 -> 100.111.44.55:63281 0.40 21.62 0.40 22.42
TCP: 100.111.22.33:21291 -> 100.111.44.55:63280 0.40 21.62 0.40 22.42
PhysicalTraffic: 2.80 454.75 1.80 349.07
100.44.55.66 -> 192.168.0.101:41641 1.20 227.37 0.60 169.73
100.99.888.77 -> 143.111.222.333:41641 0.80 150.52 0.40 102.48
100.111.44.55 -> 192.555.66.77:41641 0.80 76.86 0.80 76.86
=============================================================================================
NodeID: uvwXyz2CNTRL
Logged: 2022-10-28 15:40:00.344
Window: 2022-10-28 15:39:53.286 (4.999s)
------------------------------------------------------- Tx[P/s] Tx[B/s] Rx[P/s] Rx[B/s]
VirtualTraffic: 28.60 3.32Ki 46.41 5.87Ki
TCP: 100.44.55.66:37500 -> 100.103.145.6:22 9.20 1.45Ki 13.60 2.46Ki
TCP: 100.44.55.66:49288 -> 100.99.888.77:22 12.80 1.22Ki 20.80 2.03Ki
TCP: 100.44.55.66:2049 -> 100.111.22.33:864 0.60 145.62 1.20 180.02
TCP: 100.44.55.66:49284 -> 100.99.888.77:22 1.00 88.01 2.00 236.03
TCP: 100.44.55.66:49282 -> 100.99.888.77:22 1.00 88.01 2.00 236.03
TCP: 100.44.55.66:49278 -> 100.99.888.77:22 1.00 88.01 2.00 236.03
TCP: 100.44.55.66:49280 -> 100.99.888.77:22 1.00 88.01 2.00 236.03
TCP: 100.44.55.66:49286 -> 100.99.888.77:22 1.00 88.01 1.80 193.62
ICMPv4: 100.44.55.66 -> 100.99.888.77 1.00 84.01 1.00 84.01
PhysicalTraffic: 28.80 4.46Ki 46.41 7.74Ki
100.99.888.77 -> 143.111.222.333:41641 18.80 2.48Ki 31.60 4.51Ki
100.103.145.6 -> 64.71.162.170:41641 9.20 1.82Ki 13.60 3.01Ki
100.111.22.33 -> 192.168.0.102:41641 0.60 169.62 1.20 227.23
To make it easier to recognize nodes in the output, the netlogfmt
binary provides flags that
resolve IP addresses to readable hostnames:
--resolve-names
: Convert tailscale IP addresses to hostnames. You must also specify--api-key
and--tailnet-name
as parameters tonetlogfmt
when you use the--resolve-names
flag.--api-key
: Specifies the API access token to use for the Tailscale API. This can be the same API access token that you used for thenetwork-logs
endpoint, or another valid API access token.--tailnet-name
: The same organization name that you passed in to thenetwork-logs
endpoint.
This example shows how to use the netlogfmt
flags:
curl -u $ACCESS_TOKEN: -X GET \
"https://api.tailscale.com/api/v2/tailnet/{$TAILNET_ID}/network-logs?start={$START}&end={$END}" \
| go run tailscale.com/cmd/netlogfmt@main -- \
--resolve-names --api-key=$ACCESS_TOKEN --tailnet-name=$TAILNET
The output will look like:
=======================================================================================
NodeID: aBcdef1CNTRL
Logged: 2022-10-28 15:40:00.290
Window: 2022-10-28 15:39:51.890 (4.996s)
------------------------------------------------- Tx[P/s] Tx[B/s] Rx[P/s] Rx[B/s]
VirtualTraffic: 2.80 342.66 1.80 277.01
TCP: carbonite:864 -> prism:2049 1.20 180.14 0.60 145.71
TCP: carbonite:723 -> diamond:2049 0.80 119.29 0.40 86.47
TCP: carbonite:21291 -> glass:63281 0.40 21.62 0.40 22.42
TCP: carbonite:21291 -> glass:63280 0.40 21.62 0.40 22.42
PhysicalTraffic: 2.80 454.75 1.80 349.07
prism -> 192.168.0.101:41641 1.20 227.37 0.60 169.73
diamond -> 143.110.111.222:41641 0.80 150.52 0.40 102.48
glass -> 192.555.66.77:41641 0.80 76.86 0.80 76.86
======================================================================================
NodeID: uvwXyz2CNTRL
Logged: 2022-10-28 15:40:00.344
Window: 2022-10-28 15:39:53.286 (4.999s)
------------------------------------------------ Tx[P/s] Tx[B/s] Rx[P/s] Rx[B/s]
VirtualTraffic: 28.60 3.32Ki 46.41 5.87Ki
TCP: prism:37500 -> glass:22 9.20 1.45Ki 13.60 2.46Ki
TCP: prism:49288 -> diamond:22 12.80 1.22Ki 20.80 2.03Ki
TCP: prism:2049 -> carbonite:864 0.60 145.62 1.20 180.02
TCP: prism:49284 -> diamond:22 1.00 88.01 2.00 236.03
TCP: prism:49282 -> diamond:22 1.00 88.01 2.00 236.03
TCP: prism:49278 -> diamond:22 1.00 88.01 2.00 236.03
TCP: prism:49280 -> diamond:22 1.00 88.01 2.00 236.03
TCP: prism:49286 -> diamond:22 1.00 88.01 1.80 193.62
ICMPv4: prism -> diamond 1.00 84.01 1.00 84.01
PhysicalTraffic: 28.80 4.46Ki 46.41 7.74Ki
diamond -> 143.110.111.222:41641 18.80 2.48Ki 31.60 4.51Ki
glass -> 64.71.111.222:41641 9.20 1.82Ki 13.60 3.01Ki
carbonite -> 192.168.0.102:41641 0.60 169.62 1.20 227.23
Network flow logs streaming
Log streaming lets you stream network flow logs into a security information and event management (SIEM) system. For more information, see Log streaming.
Limitations
- Only nodes that have been updated to Tailscale v1.34 or later will send networking telemetry to the Tailscale logs service.
- Network logs are accessible only by using the API and as a streaming source for SIEM systems. There is no network logs viewing functionality in the Tailscale admin console.
- Network flow logs are not for monitoring—instead they are for logging. The admin console does not contain a “live” view of what nodes are connected to what other nodes. As an example, network logs don’t include information on whether a node is online and idle. The logs indicate only whether there was network traffic.
- Network logs do not include individual packet transfers. Logs capture when connections are active, which could include multiple data flows.
- Only successful connects are logged. If a connection attempt results in denied access, the attempt isn’t logged, as it doesn’t appear differently than if no connection was attempted.
- Traffic information at the physical layer is gathered at a slightly different moment in time as the virtual layer, so packets flowing through the virtual layer may not exactly line up with those at the physical layer.
- Public IP addresses are not logged as either a destination or source. That is, a connection from your tailnet through an exit node does not log where the traffic is going to, and a connection from the public internet through an exit node does not log where the traffic is returning from. Fine granularity information about individual connections is not gathered so that privacy can be preserved.
- The user authenticated on a source or destination host is not logged. If you want to map a node to a user, you can use
the
/api/v2/tailnet/:tailnet/device
method and examine theuser
field. - Network logging is performed on the client side, and Tailscale cannot fully guarantee the delivery or veracity of the clients logs. Tailscale cannot fully guarantee client log latency thresholds—that is, logs will not be delivered in real-time. We believe the latency delay will be in the order of minutes under normal operating conditions. There could however be edge cases, such as when network connectivity is weak or unavailable, which may increase latency delay.
- Enabling network flow logs may result in a slight performance impact, as the client does additional work to track necessary metrics. The incremental load is spread across the fleet, rather than being concentrated on a single host.