Guidance for hardening Tailscale nodes

This document outlines hardening and defense-in-depth steps you can apply to Tailscale nodes to further mitigate and prevent the threat of a single node running Tailscale being compromised as an entrypoint to your network. We also recommend following our Security best practices to harden your network.

This is not general purpose guidance. Tailscale’s default installation represents a good tradeoff between system compatibility and security that is sufficient for the vast majority of uses. This advanced guidance is meant for deployments where other network security controls (such as intrusion detection or use of a private network) may not be in place or be sufficient for security. Hardening nodes as detailed below entails a non-trivial amount of effort and ongoing maintenance.

Implement baseline patching and system hardening

Micro-hardening of the Tailscale process on a node only provides meaningful security gains if lower hanging fruit is already covered. To this end, make sure you have the following tablestakes implemented for node hardening.

Patching and vulnerability management

Keep track of the kernel version of your systems and the versions of any installed software, and apply security patches in a timely manner. Of primary importance is patches for the underlying kernel (with special attention and priority paid to patching vulnerabilities in the network stack), any administrative access software such as OpenSSH or Dropbear, and Tailscale (security announcements can be found in our Security bulletins).

Remove unnecessary network services and software

Remove any unnecessary software running on your nodes to limit exposure to security issues within that software. The more software you have running, the larger the attack surface and hence the greater the risk of security vulnerabilities.

Implement baseline detection and response

Hardening your network helps prevent possible intrusions. Hardening is for prevention, and prevention is not sufficient — you should also put in place some basic detection and response capabilities.

Establish centralized monitoring and digital forensics incident response (DFIR) tooling

Capture, stream, and respond to unusual events on your nodes. Configure your node to capture appropriate audit events and system logs. Stream such logs off the device to maintain immutable activity records and seed detection activities. Establish incident response capabilities through administrative access tools or via specialized agents like GRR.

Develop an incident response playbook for node compromise

Establish a playbook for isolating and collecting forensics from a node in the event of compromise or suspected compromise. In the event of a node compromise, first, quarantine the node. This can be done by editing the tailnet policy file and removing any access rules that allow the device to establish outgoing connections. Consider using an ACL test to verify the node cannot connect to high-risk resources. Alternatively, if the node belongs to a user and the user is believed to be compromised, you can suspend the user to remove access.

Once the node is quarantined, collect forensics from the device. You can obtain client logs from the device directly.

Once the node is no longer needed for forensics, delete and recreate the node.

Architect your network to allow for network segmentation

In a conventional segmented network, access controls are implemented at the subnet level. However, with Tailscale, access controls can be implemented with precision down to specific nodes, ports and protocols, eliminating the need for additional segmentation using subnet routers. Subnet routers can still be used to bridge legacy networks and VPCs to Tailscale, or to connect to embedded devices.

See Security best practices for general guidance on hardening your networking, including segmenting your network following the principle of least privilege.

Minimize deployments of subnet routers

When network traffic flows between two nodes in your tailnet, it is mutually authenticated and end-to-end encrypted using WireGuard® for strong cryptography. This effectively eliminates threats arising from unencrypted, compromised, or even malicious networks anywhere in the routing path between the nodes. When using subnet routers, this encrypted tunnel is terminated at the router and traffic flows unencrypted from the router to the destination device, opening up the possibility of interception or tampering at that point in your network.

Reducing the use of subnet routers means more tailnet traffic traverses in end-to-end encrypted tunnels, and hence the traffic is not vulnerable to any network compromise in the routing path.

Further, given the deployment of a subnet router necessarily implies a privileged position in the underlying network, they should be considered prime targets for attackers seeking to move laterally across your network and be hardened as such.

Relegate subnet routers to essential deployments only, such as bridging connectivity between endpoints running Tailscale and critical micro-segmented infrastructure. If using public network infrastructure, encrypt or otherwise secure network traffic from a subnet router to its destination. Also consider further encryption when using shared network infrastructure, even if it is dedicated or private.

Run subnet routers separately to workloads

Run subnet routers on dedicated, separate machines to create isolation. This can help to minimize lateral movement in the case of a compromise of the subnet router or of a workload.

Sandboxing the Tailscale process on each system

Tailscale’s core functionality, including the tailscaled daemon, is written in Go. Go is a language that provides automatic memory management, and so doesn’t rely on the developer to allocate and free up memory — which prevents a whole class of memory safety vulnerabilities that sandboxing helps mitigate. Further, extensive use of fuzzing helps prevent packet parsing vulnerabilities typically seen with network-exposed software.

The following section describes how to restrict the access and privileges of the tailscaled process to what is strictly necessary for operation.

Necessary Tailscale process privileges

When running Tailscale on a node, as a default client (just running tailscale up), or operating as a subnet router or exit node, tailscaled needs to be able to create and manipulate a TUN device, create and manipulate TCP and UDP sockets, modify the DNS, netfilter, or routing configuration of the device, and perform file I/O on the node’s state and cache directories.

Enumerating more explicitly, the process needs the ability to:

• Load the TUN module if it is not already loaded (needs the CAP_SYS_MODULE capability)
• Create and operate UDP and TCP sockets, performing standard I/O operations
• Set socket mark (SO_MARK) on UDP sockets, and listen for UDP packets on user ports (ports 1024 and higher) (needs the CAP_NET_ADMIN capability)
• Open /dev/tun and perform read, write, and ioctl operations (needs the CAP_NET_RAW capability)
• Launch the iptables binary
• Manipulate interface, netfilter and routing configuration using NETLINK sockets, and read and write to /proc/net and /proc/sys/net (needs the CAP_NET_ADMIN capability)
• Perform basic file i/o within the --state-dir and --cache-dir passed as arguments
• Configure systemd-resolved, implying system D-Bus access is needed with the ability to configure DNS
• Create, read, and write on a UNIX socket

This is in addition to the standard access needed for all processes:

• Read and write to /proc/self/*
• Open /dev/{null,stdin,stdout,random,urandom,zero}

Suggested hardening configuration based on systemd

These instructions apply to a modern systemd-based distro, such as Ubuntu or Debian.

1. Create a new user and group for the process tailscaled.

   adduser --disabled-password --gecos ""
   

2. Install a polkit configuration file that permits tailscaled to configure DNS, for instance to /etc/polkit-1/localauthority/10-vendor.d/tailscaled.pkla:

   [Allow tailscaled to manipulate DNS settings]
   Identity=unix-user:tailscaled
   Action=org.freedesktop.resolve1.*
   ResultAny=yes
   

3. Create a file /etc/systemd/system/tailscaled.service systemd unit with the following hardened configuration:

   [Unit]
   Description=Tailscale node agent
   Documentation=https://tailscale.com/kb/
   Wants=network-pre.target
   After=network-pre.target NetworkManager.service systemd-resolved.service
   
   [Service]
   EnvironmentFile=/etc/default/tailscaled
   ExecStartPre=/usr/sbin/tailscaled --cleanup
   ExecStart=/usr/sbin/tailscaled --state=/var/lib/tailscale/tailscaled.state --socket=/run/tailscale/tailscaled.sock --port=${PORT} $FLAGS
   ExecStopPost=/usr/sbin/tailscaled --cleanup
   Restart=on-failure
   
   RuntimeDirectory=tailscale
   RuntimeDirectoryMode=0755
   StateDirectory=tailscale
   StateDirectoryMode=0700
   CacheDirectory=tailscale
   CacheDirectoryMode=0750
   Type=notify
   
   User=tailscaled
   Group=tailscaled
   
   DeviceAllow=/dev/tun
   DeviceAllow=/dev/net/tun
   AmbientCapabilities=CAP_NET_RAW CAP_NET_ADMIN CAP_SYS_MODULE
   ProtectKernelModules=no
   RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6 AF_NETLINK
   NoNewPrivileges=yes
   PrivateTmp=yes
   PrivateMounts=yes
   RestrictNamespaces=yes
   RestrictRealtime=yes
   RestrictSUIDSGID=yes
   MemoryDenyWriteExecute=yes
   LockPersonality=yes
   ProtectHome=yes
   ProtectControlGroups=yes
   ProtectKernelLogs=yes
   ProtectSystem=full
   ProtectProc=noaccess
   SystemCallArchitectures=native
   SystemCallFilter=@known
   SystemCallFilter=~@clock @cpu-emulation @raw-io @reboot @mount @obsolete @swap @debug @keyring @mount @pkey
   
   [Install]
   WantedBy=multi-user.target
   

   The above service configuration runs the Tailscale agent in its own user and with just the capabilities it needs to operate, with the following additional restrictions:

   • An isolated mount filesystem with: read-only access to the system, a locked-down /dev tree, read-only access to its home directory, locked-down access to /proc, and a private temp directory.
   • The process cannot escalate its privileges (neither its capabilities set nor EUID)
   • The process cannot have writable executable memory
   • The process cannot set up namespaces (such as user namespaces, which are a frequent source of privilege escalation vulnerabilities)
   • The process cannot manipulate the system clock nor request real time scheduling
   • The process cannot issue unexpected syscalls, such as those for a different architecture, concerning virtualization, power, filesystems, and a number of other functions.

   If the TUN module is compiled into the kernel or can otherwise be guaranteed to be loaded, then CAP_SYS_MODULE can be removed from the capability set and ProtectKernelModules set to yes.

4. Reload and restart your system configuration.

Alternative: Use userspace networking

Instead of using the suggested hardening configuration above, you can also run Tailscale using userspace networking, which does not utilize a TUN device nor need to run as a user with any privileges. As a result, userspace networking has poorer performance in CPU utilization, latency, and throughput, compared to running with TUN.

For instructions on running Tailscale in this mode, see Userspace networking mode.

Constrain and audit traffic in your LAN

If a node were to be compromised, you will also want to prevent movement within your network. Tailscale only manages traffic in your network that goes over Tailscale. In addition to constraining and monitoring traffic on your Tailscale network, you will want to constrain and monitor other traffic traversing other network interfaces.

Restrict, log, and inspect traffic flows

Restrict access to your private network at the boundary, for example, by implementing a firewall.

Log access flows in your network between devices.

Inspect traffic flows in your network at the entrypoint to the network, for example, by using an intrusion detection system. You can also inspect traffic flows between your devices, for example, using deep packet inspection — note that this will not work for Tailscale traffic, which is end-to-end encrypted.

Consider applying network enforcement points at a higher security boundary

If a node were to be compromised, it can no longer be trusted to apply any firewall rules configured for its interfaces, such as might be configured through the operating system. To ensure network containment despite such a compromise, consider having additional firewall enforcement points at critical places within your network.