YHK - yet another hacker news clone

Tailscale SSH: Investigation and remediation update

103 points figmert 1 hour ago 53 comments

wslh 1 hour ago | parent

With the success of Tailscale I would like to be sure that their repositories and scenarios are continuously reviewed. I understand that there is always a next hop in cybersecurity though.

_def 1 hour ago | parent

I think it's so invasive that they ship this per default. But thats the business model i guess.

femiagbabiaka 1 hour ago | parent

Do they ship this by default? I thought it had to be enabled: https://tailscale.com/docs/features/tailscale-ssh#configure-...

forbiddenlake 1 hour ago | parent

How do you mean? You have to run a command to enable tailscale ssh, it isn't on by default.

seemaze 1 hour ago | parent

I've always had to manually enable this feature. I've never experienced it on by default across multiple platforms and deployments.

RobMurray 4 minutes ago | parent

I've never used it once. don't even see the point. normal ssh works fine over Tailscale.

What is the advantage? Why would I want to swap the well tested auth in ssh for auth managed by tailscale?

LeoPanthera 1 hour ago | parent

Tailscale SSH is not enabled by default.

zer0zzz 34 minutes ago | parent

It’s also not even available on some platforms afaik

6thbit 1 hour ago | parent

  > Most of the Tailscale SSH issues we've identified concern privilege escalation. In some circumstances, someone already authorized to connect over Tailscale SSH may be able to perform actions beyond the permissions of their configured local user, including potentially gaining elevated privileges.
"Most"? "In some circumstances"? Makes me think that they could also be investigating a full on unauthorized connection issue, or am I reading too much into it?

awildfivreld 1 hour ago | parent

The underlying writeup can be found at https://tailworm.com/

drum55 45 minutes ago | parent

10,000 words of claudish, it's not worth reading.

remus 1 hour ago | parent

I think they're just writing cautiously because they're midway through the investigation and don't want to give false confidence. Doesn't seem worth reading too much into it until they provide a more thorough write up.

devmor 1 hour ago | parent

A security researcher colleague of mine was/is exploring an unauthenticated avenue along this line. I don’t have any update, she told me she didn’t feel comfortable exposing information to anyone other than Tailscale staff until she’s confident in what she found or didn’t.

raggi 4 minutes ago | parent

we'd love to hear about it at security@tailscale.com, we're happy to work from & with and credit early ideas as well.

pizzafeelsright 1 hour ago | parent

Obviously tailscale has some great devs, security is complex.

There is some weaselly language.

In some circumstances default config? with password? this is a condition leading to the next

someone already authorized to connect over Tailscale SSH they have access, but it may be limited or restricted, although scope could be beyond the intended

may be able to perform actions beyond the permissions If there is a full moon and your brother in law's second cousin is wearing a hat, he may be able to use your upstair's thermostat provided you have a second story.

fooqux 31 minutes ago | parent

Given what just happened with PHP a few days so, it's not surprising to me that they're holding their cards close to their chest. People will already be doing their damnedest to reverse engineer the exploit with this meager info.

traceroute66 19 minutes ago | parent

> Obviously tailscale has some great devs, security is complex.

Quick reminder that in recent months Tailscale has had issues with age-old classic vulnerabilities such as input validation.

The problem with Tailscale is twofold.

First their attack surface is too big. They continuously shoehorn new features into the product.

Second they don't appear to do regular deep-dive code audits and instead they appear to rely mostly on the goodwill of third-parties to poke at select areas of their code as and when those third-parties feel like it. Hence simple vulnerabilities such as input validation keep on popping up.

Fancy SOC2 certificates are not the same thing. That's just dogfood for the corporate auditors.

At $work we are busy working on removing Tailscale. The bells and whistles of Tailscale sure are attractive candy, but we simply can't overlook the security risk of what has become a bloated product.

slowin 1 hour ago | parent

I self-host a tailnet with Headscale and could not be happier: https://headscale.net/stable/

dist-epoch 1 hour ago | parent

Sure, that doesn't mean it's not vulnerable.

> Organizations using headscale v0.29.5 have no mechanism to disable the forwarding behavior at the center of the finding.

https://techjacksolutions.com/scc-vendor-rollup/tailscale-he...

minimaltom 12 minutes ago | parent

If you use Tailscale SSH with headscale you are also affected. The vuln is in tailscaled, not the control plane / netmap.

wrxd 1 hour ago | parent

I like Tailascale as a product but I decided to move my homelab to plain WireGuard.

I wasn't using any of the additional features, I can get away with exposing a port on my router and I am changing devices rarely.

News like this make me happy I removed a lot of complexity and attack surface

fooqux 1 hour ago | parent

From the sounds of what info they felt was safe to provide, this wouldn't have impacted you anyway.

giwook 52 minutes ago | parent

Don’t think that refutes their point.

femiagbabiaka 51 minutes ago | parent

> I can get away with exposing a port on my router

using wireguard is awesome, but I wouldn't say deploying and maintaining it lacks complexity exactly. and you didn't get rid of attack surface, you just made it your responsibility ;)

swingandamiss 45 minutes ago | parent

With AI, the complexity doesn't really matter anymore. Codex or Claude can easily configure complex site-to-site setups and peer configurations on mobile devices. I did this by hand before and it was hard and difficult to get right. Now I just tell AI and it does so perfectly, checks and verifies things. AI made life so easy.

femiagbabiaka 41 minutes ago | parent

I would not personally expose my devices over a VPN that I self-hosted and set up purely through AI unless I felt like I understood how it worked completely. But that's probably just personal paranoia.

swingandamiss 28 minutes ago | parent

Just go read reddit threads on people who do this by hand following half baked guides. There's a huge population doing this, now everyone is switching to AI. It's a big win and safer. Not everyone is a trained network engineer, but now they are with AI

arcanemachiner 40 minutes ago | parent

This is true, but I still wouldn't discount the possibility of mistakes. It's good to know how your home network works. Even Opus 5.5 can make some pretty dumb mistakes, particularly if it doesn't have the full picture.

swingandamiss 29 minutes ago | parent

Humans also make mistakes. At this point, I trust AI more than humans when it comes to casually configuring VPNs. I've read many who exposed things by mistake on reddit. AI has nearly solved this, probably is a solved issue at this point.

drum55 38 minutes ago | parent

No amount of claude can handle the key thing that Tailscale fixes about wireguard, all of the connection to connection logic is handled be it direct, via all sorts of weird hole punching or via relays. Wireguard itself is horrible to do this with, half the time you're just looking at `wg status` wondering why the packet counter isn't incrementing today. It can never achieve the magic that is a direct connection between plane wifi and a host that's behind NAT on the ground.

LoganDark 42 minutes ago | parent

Attack surface almost entirely goes away if you just require a good port knock first! I do that with my protection SSH servers and it works just fine. Also prevents anyone from knowing there even is an SSH server there. Or any server if it doesn't respond to pings either, and only uses reverse tunnels to accept public connections

mbreese 37 minutes ago | parent

Is there a way to automate port knocks in your .ssh/config? I think that’s the main reason why I haven’t added it to my home server.

LoganDark 23 minutes ago | parent

Technically you could use a ProxyCommand that does the knock and then runs nc to your target.

lakeviewterrace 36 minutes ago | parent

Port knocking is idiotic and amounts to Biff Tannen's "only I know how to start this car".

Gigachad 23 minutes ago | parent

Dunno, in combination with having an up to date system following best practices, port knocking is probably going to save you when bots start scanning the internet with the next SSH 0day. Any amount of friction will cause the bots to move to the next IP address.

esseph 19 minutes ago | parent

LLMs port knock for redteam scans. Have for a long time.

LoganDark 6 minutes ago | parent

My knock for SSH was 20 ports within 2 seconds. knockd resets the sequence if you hit any of its ports out of order.

icedchai 17 minutes ago | parent

Just moving SSH off a default port gets you 98% there.

LoganDark 15 minutes ago | parent

An attacker only has to get lucky once, so while I do put SSH on a non-default port too, it's not enough to avoid attacks on its own.

LoganDark 16 minutes ago | parent

It can also save you if, theoretically, there's ever a public exploit going around that can achieve RCE on nearly all versions of SSH, and some moron decides that nearly a day of embargo on the patch is more important. And said moron seemingly wasn't even trying to make sure Linux distros had security updates ready either, they were just sitting on it while the exploit was at large. This is apparently a risk now.

neilalexander 20 minutes ago | parent

Wireguard doesn’t respond to malformed packets or handshakes from unknown devices by design.

LoganDark 8 minutes ago | parent

Oh, awesome!

ectospheno 32 minutes ago | parent

Always is your responsibility even with tailscale. Now it is his responsibility without tailscale.

F_r_k 28 minutes ago | parent

BTW, netbird is a cool alternative also

aborsy 1 hour ago | parent

There are too many features, and lines of code (probably more than OpenVPN). I'm pretty sure there are all kinds of zero days, with this attack surface and feature set.

I use it extensively, and I would rather pay them a fee if they could review thr code.

Another possibility is as lock down mode, disabling fancy and risky features.

drdexebtjl 19 minutes ago | parent

This is also how I feel about Tailscale and its popular alternatives. They do too much.

Does anyone know of a Tailscale alternative that sticks to the basics?

Something that’s just WireGuard but without the manual key exchange, and with Tailscale-like NAT hole-punching that falls back to using an oblivious relay?

MisterTea 43 minutes ago | parent

Not a tailscale user so I am a bit lost here: where does SSH fit into a Wireguard based VPN service?

jubilanti 41 minutes ago | parent

As added candy to their daemon to get you into their ecosystem

Gigachad 18 minutes ago | parent

Everyone is twitchy over security right now, but at first glance this seems not that bad. If the attacker has to be manually added to your tailnet that makes this not very exploitable. You should be more worried about all the iot slop on your local network.

Good idea to set up automatic updates for the tailscale client though.

rietta 14 minutes ago | parent

I am fairly glad I have stuck with stock OpenSSH that ships with Linux. This incident proves I am glad with that choice.

Over the decades, I have maintained remote access through port forwarding on my router - at one point in 2008 I put SSH on port 80 so I could access it from the restricted wifi network at the hospital my grandmother was at. I have also used openvpn. I have also used cloudflare tunnels. But in the end, I always connect to hardened openssh with key only authentication.

minimaltom 13 minutes ago | parent

Former tailscalar here and now someone managing tailscale at an org.

Having done my own review of the vulns and the fixes as they are landing in github, its pretty clear to me that this only affects situations where you use ACLs to permit ssh into a node (via tailscale ssh, not via opensshd), but dont give a path to sudoers/root.

If a user had no SSH access and/or connectivity to a vulnerable node, none of these vulns give them access.

However, if you have a vulnerable tailscaled and you give ssh access as some non-root account/privs, then yes, these vulns are relevant as theres now known privilege escalations that give access beyond some specific user account.

Not trying to play down the vuln (its important), but its not a free-for-all RCE that some ppl are acting like it is.

hollow-moe 9 minutes ago | parent

Recently NetBird also had a huge privesc through their SSH component too... I'm using both at $DAYJOB and my personal infra, damn you guys ought to be more careful with such a sensible and important component...