Former tailscalar here and now someone managing tailscale at an org.
Having done my own review of the vulns and the fixes as they are landing in github, its pretty clear to me that this only affects situations where you use ACLs to permit ssh into a node (via tailscale ssh, not via opensshd), but dont give a path to sudoers/root.
If a user had no SSH access and/or connectivity to a vulnerable node, none of these vulns give them access.
However, if you have a vulnerable tailscaled and you give ssh access as some non-root account/privs, then yes, these vulns are relevant as theres now known privilege escalations that give access beyond some specific user account.
Not trying to play down the vuln (its important), but its not a free-for-all RCE that some ppl are acting like it is.
But a large portion of Tailscale users should be software engineering professionals, so a lot of the time, people use Tailscale just for SSH. I personally feel the impact is still quite big, especially since SSH leads directly to control of the server.
Do you use tailscale ssh but either dont have sudo, or don't permit login as root?
If all your tailscale-ssh ACLs allow you into an account with sudo, or allow you into root, theres no impact (you cant do anything with these vulns you couldnt do a month ago). Thats the nuance I'm trying to get across (admittedly my english is bad).
On the majority of nodes I provide Tailscale SSH access to, it's as root or a user with passwordless sudo. It's being used for admin access. So yeah, on those it doesn't matter as much.
There are a handful which restrict users to their own user accounts. But it's on dev server with friendly devs, so even for that use case it's not really the end of the world.
I'm sure our infosec folks are going to demand that we upgrade everywhere, however.
Engineers use vanilla SSH for SSH, Tailscale SSH is more focused on enterprise stuff where you can have an audit trail of all commands executed and enforce ACLs.
I’ve used tailscale in quite a few jobs now and not one have we used tailscale ssh. Call us old school or whatever but at my current place of employment we provision ssh keys on a box with ansible and just use regular old ssh over the tailnet, not tailscale ssh
The vast majority of tailnets are just hobbyists on the generous free tier and probably don't use tailscale ssh, but rather ssh over a tailnet. Usually it's either just one person or a family sharing access to the tailnet. But for corporate accounts where ACLs are used for SSH yeah this is big.
I use tailnet as a vpn and ssh into my server via the tailnet, but I dont use tailscale ssh the product. I think most people fall into this bucket.
Of course it should be taken seriously, but this topic is also piggy backing off of 7 days straight of non stop viral engagement farming on X after php2shell, where people we're farming likes by just posting stuff like
"YoU ThInK PhP2ShELl iS BaD JuST WaIt unTiL TaIlScAlE get HaCkED, ThIS JuSt ThE BeGiNNiNG oF a CybeR ReCKoNiNG"
To farm FUD from normies. Which of course led researchers who were working on tailscale vulns to take advantage of that algorithmic attention as well and talk about their research.
Like Im pretty sure we knew about this vuln for 3 days now, or atleast that it was responsibly disclosed (and not being used in the wild).
The attention this is getting is definately the product of some cringe viral snowball effect of the also rather benign p2pshell vuln.
It feels like there's a ton of AI influencer astroturfing going on, because its wildly easy to get engagement when you add the words "agent/ai" with "vulnerbility" and [insert popular service].
Calling it “tailworm” feels like alarmism to promote their find more.
Would only show worm behavior if machines within the tailnet had the right settings and also were members of other tailnets with those settings but then it would require someone in that tailnet to have another tailnet with the right settings…
I backup my different devices to my NAS. The user is a restricted backup user seeing only their own directory. I obviously don't want my torrent computer gets root on my NAS.
I personally use openssh, but someone else may use tailscale SSH.
Somehow I don't think it's a good idea to let Tailscale handle service authentication at all. It's supposed to be an additional security layer around existing infrastructure and services.
Letting Tailscale handle VPN and SSH at the same time is basically once again a single point of failure. Similar to exposing SSH directly to the internet.
I am fairly glad I have stuck with stock OpenSSH that ships with Linux. This incident proves I am glad with that choice.
Over the decades, I have maintained remote access through port forwarding on my router - at one point in 2008 I put SSH on port 80 so I could access it from the restricted wifi network at the hospital my grandmother was at. I have also used openvpn. I have also used cloudflare tunnels. But in the end, I always connect to hardened openssh with key only authentication.
It sounds unlikely you'd be affected by this anyway. The bug allows people you've explicitly authorized to ssh into an account on your machine, to access more than that account should. If you only use ssh yourself, or you only give ssh keys to accounts with sudo, or only to people you'd trust not to exploit this, or if other vulnerabilities allow local accounts to escalate, or if you only use tailscale for port forwarding, then you're unaffected. Arguably even less exposed than a public openssh instance, though I can't say I have any issue with that
I run WireGuard VPN on port 443 UDP. Many times I can connect to that from various public WiFi networks like coffee shops and public libraries. And then I ssh into my machines at home over my self-hosted VPN.
I’ve been hoping that HTTP3 adoption would make 443 UDP accessible from even more places. But maybe they already do the kind of thing you said for that too, looking at the traffic flow to determine that it is not HTTPS.
Weirdly sometimes there are WiFi networks where I am not able to connect to my WireGuard VPN on 443 UDP, but I can connect to servers over ssh on standard port 22 TCP. Perhaps they sometimes block all UDP traffic because they don’t want people to use BitTorrent etc, but one would think that with all the online meeting platforms that do video streaming there would be a lot of problems for people if all UDP traffic was blanket blocked.
I might eventually try out AmneziaWG instead of vanilla WireGuard, but at the moment I am fine not being able to Connect to my VPN sometimes. I only use it to connect to my other machines anyway, not as a VPN tunnel for my web browsing etc.
it's possible commercial routers such as the ones utilized by restaurants allow wireguard through these days? wireguard makes no effort to be stealthy when emitting.
some VPN apps bypass wireguard blocks automatically by using https tunnels and wahtnot.
There are too many features, and lines of code (probably more than OpenVPN). I'm pretty sure there are all kinds of zero days in it, with this attack surface and feature set.
I use it extensively, and I would rather pay them a fee if they could review thr code.
A possibility is a lock down mode, disabling risky features.
I just want a VPN for connrctiviy, no taildrop, taildrive, SSH, funnel, services, DNS, node sharing, ...They are disabled in my admin console, but the code runs on my devices and they can be enabled by mistake.
A question. Taildrop enables unauthenticated input and doesn't even obey ACLs. I'm not quite sure about this, but couldn't malware in a low trust device send a file to my Synology NAS, which will then automatically process it for indexing things like thumbnail, leading to potential code execution? Why doesn't it obey ACLs, and why isn't the recipient approval needed before the file exits the sender's device? I use tags, but I'm the owner not the tag.
This is also how I feel about Tailscale and its popular alternatives. They do too much.
Does anyone know of a Tailscale alternative that sticks to the basics?
Something that’s just WireGuard but without the manual key exchange, and with Tailscale-like NAT hole-punching that falls back to using an oblivious relay?
I've been toying around with nebula, self host your lighthouse on a cheap VPS (which even if it gets hacked, the most it can do is lie about traffic, not lie about node certs). Basically you're just paying for the punchthrough (you may even be able to do some duckdns shenanigans?) The main issue is that updating nodes is a pain in the butt, which is the main advantage of tailscale IMO - centralized control plane. Keeps it simple.
I was trying to do some kind of insane workflow where you as the operator have a privileged hardware yubikey that admins the nodes when you plug it in, and the point is the nodes would self-propogate the changes outwards. I immidiately ran into classic issues with distributed security in distributed systems (even with only one central operator trying to admin everything). Tailscale is also a bit more secure in its oauth layer - with this architecture, you need to set proper time limits for expiring so one node compromise doesn't get access to the whole fleet. Couldn't really come up with better ideas than forcing the operator to reuse some kind of hardware key (the idea trying to be avoiding maintaining a cloud control plane as much as possible)
Fun foray but creating such a system requires a lot of expertise that I unfortunately lack
Re: Your question. Taildrop only works between devices in the same account. So if you don't put low trust devices in the same account as your NAS this is mitigated.
> I can get away with exposing a port on my router
using wireguard is awesome, but I wouldn't say deploying and maintaining it lacks complexity exactly. and you didn't get rid of attack surface, you just made it your responsibility ;)
Attack surface almost entirely goes away if you just require a good port knock first! I do that with my protection SSH servers and it works just fine. Also prevents anyone from knowing there even is an SSH server there. Or any server if it doesn't respond to pings either, and only uses reverse tunnels to accept public connections
Dunno, in combination with having an up to date system following best practices, port knocking is probably going to save you when bots start scanning the internet with the next SSH 0day. Any amount of friction will cause the bots to move to the next IP address.
My knock for SSH was 20 ports within 2 seconds. knockd resets the sequence if you hit any of its ports out of order. I generated the sequence with a CSPRNG.
With AI, the complexity doesn't really matter anymore. Codex or Claude can easily configure complex site-to-site setups and peer configurations on mobile devices. I did this by hand before and it was hard and difficult to get right. Now I just tell AI and it does so perfectly, checks and verifies things. AI made life so easy.
No amount of claude can handle the key thing that Tailscale fixes about wireguard, all of the connection to connection logic is handled be it direct, via all sorts of weird hole punching or via relays. Wireguard itself is horrible to do this with, half the time you're just looking at `wg status` wondering why the packet counter isn't incrementing today. It can never achieve the magic that is a direct connection between plane wifi and a host that's behind NAT on the ground.
It makes me happy that I can automate some menial tasks or replace a search engine sometimes, but this is just because I already have knowledge of the field and the skill to apply it. You're not wrong that it'll probably get even better, but I don't see a path right now to unskilled people being able to effectively use an agent. Only time will tell if that can happen.
I do disagree with GP's assertion that Claude can never make something comparable to Tailscale, but that's just because I believe I (and probably many others here) could probably prompt that, not because I think just anyone could. LLMs can one-shot a fair few things based on unskilled prompts, but I don't believe this is one of them yet.
Right now it's not unskilled. It's managers, PMs, TPM's, or people with some tech knowledge but who previously couldn't do these types tasks (not that they weren't capable, but it takes a lot of effort and time to get skilled in these areas and they didn't devote the time). They know what they want at a high level and now can do it with AI.
This is mostly true, I think. I think this is a sweet spot for LLMs, too. Maybe with the right approach, a user could know or figure out how to articulate what they're missing, if the first attempt doesn't work well enough for them. I just worry that e.g. if they forget to review things like protocol details, they could silently end up with flaws and such, or jank. But yeah, there's potential where there wasn't before, and I'm all for that.
This is true, but I still wouldn't discount the possibility of mistakes. It's good to know how your home network works. Even Opus 5.5 can make some pretty dumb mistakes, particularly if it doesn't have the full picture.
Humans also make mistakes. At this point, I trust AI more than humans when it comes to casually configuring VPNs. I've read many who exposed things by mistake on reddit. AI has nearly solved this, probably is a solved issue at this point.
I would not personally expose my devices over a VPN that I self-hosted and set up purely through AI unless I felt like I understood how it worked completely. But that's probably just personal paranoia.
Just go read reddit threads on people who do this by hand following half baked guides. There's a huge population doing this, now everyone is switching to AI. It's a big win and safer. Not everyone is a trained network engineer, but now they are with AI
On the road I find WireGuard to be quirky when connecting to DNS names. When I'm on an IPv6 only network (like T-Mobile), connecting to a DNS name that offers A and AAAA, WireGuard on iOS prefers the A record, making me connect over one of T-Mobile's 464XLAT proxies which time out after x minutes. This then makes the connection look dead until I do something on my phone that re-enables it.
I wouldn't be too surprised if tailscale has become a critical vector of the internet like cloudflare over the years as most ai companies seem to use it
Tailscale is best used as an additional layer of security rather than your only layer. I host everything behind tailscale as if it was the public internet, with proper authentication, https, etc.
So a failure in tailscale is not an issue on it's own because you'd then also have to find an exploit in what I'm actually hosting as well. Both of those things happening at the same time is much less likely.
> Most of the Tailscale SSH issues we've identified concern privilege escalation. In some circumstances, someone already authorized to connect over Tailscale SSH may be able to perform actions beyond the permissions of their configured local user, including potentially gaining elevated privileges.
"Most"? "In some circumstances"?
Makes me think that they could also be investigating a full on unauthorized connection issue, or am I reading too much into it?
I think they're just writing cautiously because they're midway through the investigation and don't want to give false confidence. Doesn't seem worth reading too much into it until they provide a more thorough write up.
Obviously tailscale has some great devs, security is complex.
There is some weaselly language.
In some circumstances default config? with password? this is a condition leading to the next
someone already authorized to connect over Tailscale SSH they have access, but it may be limited or restricted, although scope could be beyond the intended
may be able to perform actions beyond the permissions If there is a full moon and your brother in law's second cousin is wearing a hat, he may be able to use your upstair's thermostat provided you have a second story.
Given what just happened with PHP a few days so, it's not surprising to me that they're holding their cards close to their chest. People will already be doing their damnedest to reverse engineer the exploit with this meager info.
> Obviously tailscale has some great devs, security is complex.
Quick reminder that in recent months Tailscale has had issues with age-old classic vulnerabilities such as input validation.
The problem with Tailscale is twofold.
First their attack surface is too big. They continuously shoehorn new features into the product.
Second they don't appear to do regular deep-dive code audits and instead they appear to rely mostly on the goodwill of third-parties to poke at select areas of their code as and when those third-parties feel like it. Hence simple vulnerabilities such as input validation keep on popping up.
Fancy SOC2 certificates are not the same thing. That's just dogfood for the corporate auditors.
At $work we are busy working on removing Tailscale. The bells and whistles of Tailscale sure are attractive candy, but we simply can't overlook the security risk of what has become a bloated product.
A security researcher colleague of mine was/is exploring an unauthenticated avenue along this line. I don’t have any update, she told me she didn’t feel comfortable exposing information to anyone other than Tailscale staff until she’s confident in what she found or didn’t.
Everyone is twitchy over security right now, but at first glance this seems not that bad. If the attacker has to be manually added to your tailnet that makes this not very exploitable. You should be more worried about all the iot slop on your local network.
Good idea to set up automatic updates for the tailscale client though.
Recently NetBird also had a huge privesc through their SSH component too...
I'm using both at $DAYJOB and my personal infra, damn you guys ought to be more careful with such a sensible and important component...
With the success of Tailscale I would like to be sure that their repositories and scenarios are continuously reviewed. I understand that there is always a next hop in cybersecurity though.
I'll point out that Tailscale has been repeatedly praised here as a replacement "because SSH was too old" by "security professionals". Those very same security professional happen to be the exact same closing their eyes when there's the state doing wrongdoings.
My question to those who kept praising Tailscale and making fun of good old plain SSH: how do you like your state backdoor --excuse me, I mean bug-- in Tailscale now?
This comment doesn't make any sense. The products are mostly different categories. One is a VPN replacement and the other is encrypted shell access. Most people use ssh over tailscale. Tailscale does have a "Tailscale SSH" product but it's not that widely used compared to regular ssh over Tailscale.
Having done my own review of the vulns and the fixes as they are landing in github, its pretty clear to me that this only affects situations where you use ACLs to permit ssh into a node (via tailscale ssh, not via opensshd), but dont give a path to sudoers/root.
If a user had no SSH access and/or connectivity to a vulnerable node, none of these vulns give them access.
However, if you have a vulnerable tailscaled and you give ssh access as some non-root account/privs, then yes, these vulns are relevant as theres now known privilege escalations that give access beyond some specific user account.
Not trying to play down the vuln (its important), but its not a free-for-all RCE that some ppl are acting like it is.
If all your tailscale-ssh ACLs allow you into an account with sudo, or allow you into root, theres no impact (you cant do anything with these vulns you couldnt do a month ago). Thats the nuance I'm trying to get across (admittedly my english is bad).
There are a handful which restrict users to their own user accounts. But it's on dev server with friendly devs, so even for that use case it's not really the end of the world.
I'm sure our infosec folks are going to demand that we upgrade everywhere, however.
Of course it should be taken seriously, but this topic is also piggy backing off of 7 days straight of non stop viral engagement farming on X after php2shell, where people we're farming likes by just posting stuff like
"YoU ThInK PhP2ShELl iS BaD JuST WaIt unTiL TaIlScAlE get HaCkED, ThIS JuSt ThE BeGiNNiNG oF a CybeR ReCKoNiNG"
To farm FUD from normies. Which of course led researchers who were working on tailscale vulns to take advantage of that algorithmic attention as well and talk about their research.
Like Im pretty sure we knew about this vuln for 3 days now, or atleast that it was responsibly disclosed (and not being used in the wild).
The attention this is getting is definately the product of some cringe viral snowball effect of the also rather benign p2pshell vuln.
It feels like there's a ton of AI influencer astroturfing going on, because its wildly easy to get engagement when you add the words "agent/ai" with "vulnerbility" and [insert popular service].
Would only show worm behavior if machines within the tailnet had the right settings and also were members of other tailnets with those settings but then it would require someone in that tailnet to have another tailnet with the right settings…
It’s far fetched.
I personally use openssh, but someone else may use tailscale SSH.
Letting Tailscale handle VPN and SSH at the same time is basically once again a single point of failure. Similar to exposing SSH directly to the internet.
Over the decades, I have maintained remote access through port forwarding on my router - at one point in 2008 I put SSH on port 80 so I could access it from the restricted wifi network at the hospital my grandmother was at. I have also used openvpn. I have also used cloudflare tunnels. But in the end, I always connect to hardened openssh with key only authentication.
I understand port 443 is always opened but nowadays you can’t just run ssh traffic on 443 without it being detected as not being https
If their firewalls can detect it not being typical http/s traffic, they can block it.
I’ve been hoping that HTTP3 adoption would make 443 UDP accessible from even more places. But maybe they already do the kind of thing you said for that too, looking at the traffic flow to determine that it is not HTTPS.
Weirdly sometimes there are WiFi networks where I am not able to connect to my WireGuard VPN on 443 UDP, but I can connect to servers over ssh on standard port 22 TCP. Perhaps they sometimes block all UDP traffic because they don’t want people to use BitTorrent etc, but one would think that with all the online meeting platforms that do video streaming there would be a lot of problems for people if all UDP traffic was blanket blocked.
I might eventually try out AmneziaWG instead of vanilla WireGuard, but at the moment I am fine not being able to Connect to my VPN sometimes. I only use it to connect to my other machines anyway, not as a VPN tunnel for my web browsing etc.
some VPN apps bypass wireguard blocks automatically by using https tunnels and wahtnot.
I use it extensively, and I would rather pay them a fee if they could review thr code.
A possibility is a lock down mode, disabling risky features.
I just want a VPN for connrctiviy, no taildrop, taildrive, SSH, funnel, services, DNS, node sharing, ...They are disabled in my admin console, but the code runs on my devices and they can be enabled by mistake.
A question. Taildrop enables unauthenticated input and doesn't even obey ACLs. I'm not quite sure about this, but couldn't malware in a low trust device send a file to my Synology NAS, which will then automatically process it for indexing things like thumbnail, leading to potential code execution? Why doesn't it obey ACLs, and why isn't the recipient approval needed before the file exits the sender's device? I use tags, but I'm the owner not the tag.
Does anyone know of a Tailscale alternative that sticks to the basics?
Something that’s just WireGuard but without the manual key exchange, and with Tailscale-like NAT hole-punching that falls back to using an oblivious relay?
I was trying to do some kind of insane workflow where you as the operator have a privileged hardware yubikey that admins the nodes when you plug it in, and the point is the nodes would self-propogate the changes outwards. I immidiately ran into classic issues with distributed security in distributed systems (even with only one central operator trying to admin everything). Tailscale is also a bit more secure in its oauth layer - with this architecture, you need to set proper time limits for expiring so one node compromise doesn't get access to the whole fleet. Couldn't really come up with better ideas than forcing the operator to reuse some kind of hardware key (the idea trying to be avoiding maintaining a cloud control plane as much as possible)
Fun foray but creating such a system requires a lot of expertise that I unfortunately lack
I really liked it. I like the idea of embedding firewall rules in a certificate.
But , like you observed, it does not scale as well as tailscale.
{tail,head}scale is the best there is for wireguard at more than 3 nodes.
I wasn't using any of the additional features, I can get away with exposing a port on my router and I am changing devices rarely.
News like this make me happy I removed a lot of complexity and attack surface
using wireguard is awesome, but I wouldn't say deploying and maintaining it lacks complexity exactly. and you didn't get rid of attack surface, you just made it your responsibility ;)
It would be idiotic to use the knock as the only authentication, which is why I don't.
To be quite honest, I just like the aesthetics of it.
I do disagree with GP's assertion that Claude can never make something comparable to Tailscale, but that's just because I believe I (and probably many others here) could probably prompt that, not because I think just anyone could. LLMs can one-shot a fair few things based on unskilled prompts, but I don't believe this is one of them yet.
Wireguard is the building block for higher level tools like Tailscale to build on.
Tailscale doesn't have this issue.
Connect from your home to a public VPS and connect your public devices to that VPS.
So a failure in tailscale is not an issue on it's own because you'd then also have to find an exploit in what I'm actually hosting as well. Both of those things happening at the same time is much less likely.
> Organizations using headscale v0.29.5 have no mechanism to disable the forwarding behavior at the center of the finding.
https://techjacksolutions.com/scc-vendor-rollup/tailscale-he...
There is some weaselly language.
In some circumstances default config? with password? this is a condition leading to the next
someone already authorized to connect over Tailscale SSH they have access, but it may be limited or restricted, although scope could be beyond the intended
may be able to perform actions beyond the permissions If there is a full moon and your brother in law's second cousin is wearing a hat, he may be able to use your upstair's thermostat provided you have a second story.
I fully understand being carefully worded when you need serious brain power just to figure out every claim in the logorrhea
Quick reminder that in recent months Tailscale has had issues with age-old classic vulnerabilities such as input validation.
The problem with Tailscale is twofold.
First their attack surface is too big. They continuously shoehorn new features into the product.
Second they don't appear to do regular deep-dive code audits and instead they appear to rely mostly on the goodwill of third-parties to poke at select areas of their code as and when those third-parties feel like it. Hence simple vulnerabilities such as input validation keep on popping up.
Fancy SOC2 certificates are not the same thing. That's just dogfood for the corporate auditors.
At $work we are busy working on removing Tailscale. The bells and whistles of Tailscale sure are attractive candy, but we simply can't overlook the security risk of what has become a bloated product.
Good idea to set up automatic updates for the tailscale client though.
My question to those who kept praising Tailscale and making fun of good old plain SSH: how do you like your state backdoor --excuse me, I mean bug-- in Tailscale now?
What is the advantage? Why would I want to swap the well tested auth in ssh for auth managed by tailscale?