Headscale Files

Minimum supported Tailscale client version: v1.64.0

Database integrity improvements

This release includes a significant database migration that addresses longstanding issues with the database schema and data integrity that has accumulated over the years. The migration introduces a schema.sql file as the source of truth for the expected database schema to ensure new migrations that will cause divergence does not occur again.

These issues arose from a combination of factors discovered over time: SQLite foreign keys not being enforced for many early versions, all migrations being run in one large function until version 0.23.0, and inconsistent use of GORM's AutoMigrate feature. Moving forward, all new migrations will be explicit SQL operations rather than relying on GORM AutoMigrate, and foreign keys will be enforced throughout the migration process.

We are only improving SQLite databases with this change - PostgreSQL databases are not affected.

Please read the PR description for more technical details about the issues and solutions.

SQLite Database Backup Example:

:::bash
# Stop headscale
systemctl stop headscale

# Backup sqlite database
cp /var/lib/headscale/db.sqlite /var/lib/headscale/db.sqlite.backup

# Backup sqlite WAL/SHM files (if they exist)
cp /var/lib/headscale/db.sqlite-wal /var/lib/headscale/db.sqlite-wal.backup
cp /var/lib/headscale/db.sqlite-shm /var/lib/headscale/db.sqlite-shm.backup

# Start headscale (migration will run automatically)
systemctl start headscale

DERPMap update frequency

The default DERPMap update frequency has been changed from 24 hours to 3 hours. If you set the derp.update_frequency configuration option, it is recommended to change it to 3h to ensure that the headscale instance gets the latest DERPMap updates when upstream is changed.

Autogroups

This release adds support for the three missing autogroups: self (experimental), member, and tagged. Please refer to the documentation for a detailed explanation.

autogroup:self is marked as experimental and should be used with caution, but we need help testing it. Experimental here means two things; first, generating the packet filter from policies that use autogroup:self is very expensive, and it might perform, or straight up not work on Headscale installations with a large number of nodes. Second, the implementation might have bugs or edge cases we are not aware of, meaning that nodes or users might gain more access than expected. Please report bugs.

Node store (in memory database)

Under the hood, we have added a new datastructure to store nodes in memory. This datastructure is called NodeStore and aims to reduce the reading and writing of nodes to the database layer. We have not benchmarked it, but expect it to improve performance for read heavy workloads. We think of it as, "worst case" we have moved the bottle neck somewhere else, and "best case" we should see a good improvement in compute resource usage at the expense of memory usage. We are quite excited for this change and think it will make it easier for us to improve the code base over time and make it more correct and efficient.

BREAKING

• Remove support for 32-bit binaries #2692
• Policy: Zero or empty destination port is no longer allowed #2606
• Stricter hostname validation #2383
• Hostnames must be valid DNS labels (2-63 characters, alphanumeric and hyphens only, cannot start/end with hyphen)
• Client Registration (New Nodes): Invalid hostnames are automatically renamed to invalid-XXXXXX format
  • my-laptop → accepted as-is
  • My-Laptop → my-laptop (lowercased)
  • my_laptop → invalid-a1b2c3 (underscore not allowed)
  • test@host → invalid-d4e5f6 (@ not allowed)
  • laptop-🚀 → invalid-j1k2l3 (emoji not allowed)
• Hostinfo Updates / CLI: Invalid hostnames are rejected with an error
  • Valid names are accepted or lowercased
  • Names with invalid characters, too short (<2), too long (>63), or starting/ending with hyphen are rejected

Changes

• Database schema migration improvements for SQLite #2617
• IMPORTANT: Backup your SQLite database before upgrading
• Introduces safer table renaming migration strategy
• Addresses longstanding database integrity issues
• Add flag to directly manipulate the policy in the database #2765
• DERPmap update frequency default changed from 24h to 3h #2741
• DERPmap update mechanism has been improved with retry, and is now failing conservatively, preserving the old map upon failure. #2741
• Add support for autogroup:member, autogroup:tagged #2572
• Fix bug where return routes were being removed by policy #2767
• Remove policy v1 code #2600
• Refactor Debian/Ubuntu packaging and drop support for Ubuntu 20.04. #2614
• Remove redundant check regarding noise config #2658
• Refactor OpenID Connect documentation #2625
• Don't crash if config file is missing #2656
• Adds /robots.txt endpoint to avoid crawlers #2643
• OIDC: Use group claim from UserInfo #2663
• OIDC: Update user with claims from UserInfo before comparing with allowed groups, email and domain #2663
• Policy will now reject invalid fields, making it easier to spot spelling errors #2764
• Add FAQ entry on how to recover from an invalid policy in the database #2776
• EXPERIMENTAL: Add support for autogroup:self #2789
• Add healthcheck command #2659

Upgrade

Please follow the steps outlined in the upgrade guide to update your existing Headscale installation.

It's best to update from one stable version to the next (e.g., 0.24.0 → 0.25.1 → 0.26.1) in case you are multiple releases behind. You should always pick the latest available patch release.

Be sure to check the changelog above for version-specific upgrade instructions and breaking changes.

Backup Your Database

Always backup your database before upgrading. Here's how to backup a SQLite database:

:::bash
# Stop headscale
systemctl stop headscale

# Backup sqlite database
cp /var/lib/headscale/db.sqlite /var/lib/headscale/db.sqlite.backup

# Backup sqlite WAL/SHM files (if they exist)
cp /var/lib/headscale/db.sqlite-wal /var/lib/headscale/db.sqlite-wal.backup
cp /var/lib/headscale/db.sqlite-shm /var/lib/headscale/db.sqlite-shm.backup

# Start headscale (migration will run automatically)
systemctl start headscale

Changelog

• [450a7b] [#2796]: Add creation_time and ko_data_creation_time to goreleaser.yml kos
• [64b714] .goreleaser: add upgrade section (#2820)
• [c80858] cli: do not show new pre-releases on stable (#2813)
• [e68e22] gen: test-integration (#2814)
• [52d27d] hscontrol: add /version HTTP endpoint (#2821)