Troubleshooting grants
Grants are Tailscale's unified approach to defining access controls for both network and application layers in your Tailscale network (known as a tailnet). This guide walks you through diagnosing and solving common grant-related problems, progressing from basic verification techniques to resolving complex authorization scenarios.
Diagnosing grant issues
This section covers the tools and methods available for inspecting your grant policies, verifying policy compilation, and examining how the Tailscale policy engine applies permissions to specific devices or users. Using these diagnostic techniques will help you identify the root cause of most grant-related problems.
Keep in mind that Tailscale validates the syntax of the tailnet policy file, including the grants section, when you save it. You won't be able to save it if it has a syntax error. This is true for our admin console editor, GitOps, Terraform, and our API. You also won't be able to save the tailnet policy file if it refers to a target (such as a group) that is not defined.
Verify your grants configuration
The first step in troubleshooting is to verify that your grants are correctly defined in your tailnet policy file. You can use the Preview rules tab from the Access controls page of the admin console to check how the rules apply to specific users or tags. This can help you ensure the policy configuration will work as you intend it to.
Inspect capabilities with whois
For diagnosing issues with application layer capabilities granted to specific devices or users, the tailscale whois
command is invaluable. This command provides detailed information about a node, including all capabilities granted to it through your tailnet policy.
To inspect the capabilities of a user or device, use its Tailscale IP address:
tailscale whois 100.101.102.103
The output includes:
- Basic device or user information.
- Network addresses associated with the device.
- A list of all capabilities granted to the device.
Pay special attention to the Capabilities section of the output, which details all application capabilities granted to the device. Each capability listing includes its full name and parameters, letting you verify that it is applying the correct permissions.
Resolve common issues
Many grant-related issues stem from basic configuration problems or misunderstandings about how grants work. This section addresses some common issues you might encounter when working with grants.
Selector matching issues
Selectors might fail to match the intended targets. This happens when the selectors in your grants don't correctly identify the devices or users you're trying to permit.
Common selector matching problems include:
- Misspelling email addresses or domain names.
- Using tag that have been created but not applied to any devices.
- Using group names that have been created but not populated with users.
- Forgetting to prefix group names with
group:
or tag names withtag:
. This type of syntax error will be caught automatically unless there's also a host with the same name as the group or tag.
To troubleshoot selector matching issues:
- Confirm that tags referenced in your grants have been applied to the appropriate devices.
- Check for typos in email addresses, especially if you're granting access to specific users.
- Use
tailscale status
to verify the tags applied to your devices.
You must run this command on each device you're checking.
If you're using autogroups (such as autogroup:member
), make sure you understand the devices within each autogroup. For example, autogroup:member
includes all users who are members of your tailnet, but not devices shared from other tailnets.
Less common and complex issues
Beyond basic configuration problems, you might encounter more complex issues with grants, especially when implementing sophisticated access control policies (such as posture checks, route filtering, or custom application capabilities). These issues often require a deeper understanding of how the Tailscale policy engine processes and applies grants.
Problems with via
field routing
The via
field in grants lets you segment resource access through different routers based on specific criteria. However, misconfiguration can lead to routing failures or unexpected traffic paths.
Common via
routing issues include:
- Specifying a
via
target that isn't reachable from the source. - Using a
via
target with subnet routers that don't have the appropriate subnet routes advertised. - Using a
via
target with app connectors that have not been appropriately configured. - Using a
via
target with exit nodes that have not been approved. - Missing
via
field when intending to route traffic to subnet routes, app connectors, or exit nodes.
To troubleshoot via
routing issues:
- Verify that the device specified in the
via
field exists and is online usingtailscale status
. - Confirm that the
via
device has the appropriate configuration. - For subnet routers, check that subnet routes are advertised using
tailscale status --routes
. - For app connectors, check that the app connector is properly configured.
- For exit nodes, check that the exit node has been approved in the admin console.
- Check that the
via
device is reachable from the source device usingtailscale ping
. - Verify that the destination is reachable through the
via
device by examining logs on thevia
device.
Example of a correct via
configuration:
"grants": [
{
"src": ["group:eng"],
"dst": ["192.0.2.0/24"],
"ip": ["*"],
"via": ["tag:subnet-router"]
}
]
In this example, make sure that a device with the tag:subnet-router
tag exists, is online, and has the 192.0.2.0/24
subnet route advertised.
Device posture verification failures
The srcPosture
field lets you restrict access based on device posture conditions.
Common device posture issues include:
- Devices running outdated Tailscale versions that don't support posture checking.
- Posture conditions that are too restrictive or impossible to meet.
- Misconfigured posture conditions with syntax errors.
- Devices missing required attributes for posture checking.
To troubleshoot device posture issues:
- Verify that all devices are running an up-to-date version of the Tailscale client. Versions prior to 1.52.0 don't support posture checking.
- Review the posture conditions in your tailnet policy file for correctness.
- Use
tailscale status --self
to get the device attributes that are checked for posture verification. - Test with more permissive posture conditions first, then gradually tighten restrictions.
Example of device posture verification:
"postures": {
"posture:latest": [
"node:tsVersion >= '1.42.0'",
"node:os == 'linux'"
]
},
"grants": [
{
"src": ["group:eng"],
"dst": ["tag:prod"],
"ip": ["*"]
"srcPosture": ["posture:latest"]
}
]
If devices are failing posture checks, temporarily remove the srcPosture
field to verify that the grant works without posture conditions, then add them back one by one to identify the problematic condition.
Application capability integration issues
Integrating application capabilities with grants can introduce issues when applications don't correctly interpret the capabilities provided by the Tailscale client.
Common application capability issues include:
- Applications not querying the local Tailscale client for capabilities.
- Incorrect capability names or parameter structures.
- Applications misinterpreting the capabilities returned by the Tailscale client.
To troubleshoot application capability issues:
- Verify that your application is correctly querying the Tailscale client for capabilities using the LocalAPI. This is only required if the application doesn't use the
tsnet
library. - Check that the capability name matches exactly what your application expects.
- Confirm that the parameters in your grant match the schema expected by your application.
- Test with simplified capability parameters first, then add complexity.
Example of a correct application capability grant:
"grants": [
{
"src": ["group:eng"],
"dst": ["tag:tailsql"],
"ip": ["tcp:443"],
"app": {
"tailscale.com/cap/tailsql": [
{
"dataSrc": ["prod", "staging"]
}
]
}
}
]
In this example, make sure that the application running on devices with the tag:tailsql
tag is correctly checking for the tailscale.com/cap/tailsql
capability and properly interpreting the dataSrc
parameter.
Prevention strategies
This section outlines best practices for designing, implementing, and maintaining grants in your tailnet policy. By following these recommendations, you can minimize issues and streamline troubleshooting when problems do arise.
Design principles for effective grants
- Least privilege: Grant only the minimum permissions needed for each source-destination pair. Be specific about ports and protocols in the
ip
field when appropriate. - Logical grouping: Organize related grants together and use comments to explain their purpose. This makes the policy file more readable and easier to maintain.
- Standardized naming: Use consistent naming conventions for groups, tags, and IP sets referenced in your grants. This reduces the risk of typos and makes troubleshooting easier.
- Progressive refinement: Start with broad grants and refine them over time as you better understand your access control requirements. This approach is less likely to cause unexpected denials.
- Documentation: Document your grants with comments in the policy file, explaining why each grant exists and what it's intended to permit.
Following these principles will result in a more maintainable policy file that's less prone to issues and easier to troubleshoot when problems occur.
Testing strategies
Consider using testing strategies before deploying changes to your grants:
- Document test cases: Create a set of tests in your tailnet policy file that cover all the permissions you need to verify.
- Use the access control rules preview: The admin console's policy file editor includes a preview feature that shows what changes your edits will make to the effective policy. Review this carefully before applying changes.
- Staged rollout: For significant changes, consider a staged rollout to a subset of devices or users before applying the changes to your entire tailnet.
By thoroughly testing changes before deployment and monitoring for issues afterward, you can identify and resolve problems before they affect your users.