198 points aarushyadav 1 hour ago 146 comments
Here's a video of Dave from YC trying it out: https://www.youtube.com/watch?v=9QdX1eObVYI
(Edit: you don't actually have to move your lips or even open your mouth for this to work! Since the RF can see inside your mouth the system can be fine tuned on "internal speech". This isn't the automatic tongue movements that occur when "thinking" words to yourself, but more like mouthing with your lips closed. It's a bit weird to do, but with practice, we have been able to get it down pretty well.)
Arnav and I were studying CS at Georgia Tech at the beginning of the year, with no experience in silent speech or sensing. We were both AI-native builders and power users of Wispr Flow. What bugged us was not being able to use these tools in the library (or an office) or in public. At Stanford’s TreeHacks in February, we decided to build the most technically ridiculous thing we could think of. We’d read about AlterEgo, a silent speech system out of MIT’s Media Lab, and wanted to recreate it in 36 hours. We built a crude prototype, sticking EMG electrodes onto my face:
https://drive.google.com/file/d/1mwhv5HZMz3UTevdrgwS5O7g2jWV...
We got it to kind of work at a letter level in a very janky way. It wasn’t good enough to win the hackathon, but convinced us to go all in. The first time I typed a sentence without anyone noticing, I felt the magic of the technology.
We quickly realized EMG (reading muscle electrical signals) wouldn’t work:
- Signal changes due to sweat
- Electrical interference
- Placement variability
- Most importantly, no one wants to stick electrodes onto their face!!
So we switched to RF and built the glasses you see now. They emit radio waves, similar to Bluetooth, that pass through your face. As they cross the tongue, lips and soft palate, they’re modulated by silent mouthing. We train foundation models on this data to decode these perturbations to text.
Despite prior academic research, getting this to work was a huge engineering challenge. First we built an extremely low noise RF sensing stack. Then we had to place antennas so their fields overlapped maximally with the articulators. Our first attempt on glasses failed completely, so we reverted to old habits and taped antennas on our face:
https://drive.google.com/file/d/1ks0dS0eZiTV9EqSFsyB_w5BBjpc...
Since taping antennas consistently in the same spot and angle was hard, we created this weird but effective head rig to hold them in place:
https://drive.google.com/file/d/1rhzNFpdDzr7oHk0isFY70dkoV4C...
This worked, but obviously wasn’t a product yet. So we worked deep in simulations to design glasses that could hold the antennas.
That evolved into our design now, with antennas embedded within the frames. Once we established there was a signal, the next hurdle was cross-user generalization. Everyone’s facial anatomy, and hence RF signal patterns, are vastly different. We tried algorithmic workarounds but realized the only lever was to massively scale pretraining data.
So that’s what we did. We set up booths in our office and paid people to read prompts wearing our glasses, to pretrain our models on. This produced a beautiful scaling curve, with errors falling quickly as we scaled our dataset and cross-user performance becoming genuinely compelling (as you see in the video above; Dave only had a short 20-sentence calibration).
We always heard about building things from first principles but never understood it before this. We tried tackling it top down but completely failed. Only when we broke each component (the antenna, electronics, ML) down to its fundamentals to understand and optimize it did we get it to work really well.
Over the next few months we’re continuing R&D to turn our prototype glasses into the slick product on our website and ship in Q1 2027. We’re also scaling up our dataset to improve performance on edge cases.
For anyone who loves voice dictation but hates using it around people, we’ve opened preorders for our first batch of 500 units, with a $200 discount off the final price. Allocation is running out quickly, so get yours at https://subvocal.co/preorder!
hmokiguess 1 hour ago | parent
That said, I think the idea of having a private walkie-talkie that connects on its own frequency for large crowded spaces to be very cool
dang 1 hour ago | parent
"Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something."
https://news.ycombinator.com/newsguidelines.html
p.s. Especially please don't do this when a thread is brand new. Such comments are the easiest to post, and therefore the quickest to appear, but by posting them, you steer the discussion straight away from the curious, thoughtful conversation we want here. Threads are extremely sensitive to initial conditions, so this is important.
theturtletalks 1 hour ago | parent
https://github.com/joeblau/subvox
There was this really interesting one where it reads your lips:
https://github.com/amywork777/lipflow
I’m using these to build my own so I can work in co-working spaces, planes, and libraries. Though moving my mouth with no words might draw some eyes.
willio58 1 hour ago | parent
This is what I thought the original post was about. The glasses you have to wear is where that lost me.
If we could get lip-reading via your computer/phone camera down perfectly, that's the real win. Making people wear your goofy frames is where you lose a massive portion of people. Nobody wants to look like Zuckerberg lol
aarushyadav 1 hour ago | parent
zie 53 minutes ago | parent
Good luck on the project!
theturtletalks 42 minutes ago | parent
Lip reading AI would be pretty big since privacy would probably go out the window.
altcognito 16 minutes ago | parent
aarushyadav 1 hour ago | parent
s3graham 1 hour ago | parent
I wonder if we could eventually end up with a "throat muscle" version where people learn to talk a new way without having to move the visible mouth muscles too.
mcbishop 1 hour ago | parent
melvinmelih 1 hour ago | parent
paisawalla 1 hour ago | parent
kennywinker 56 minutes ago | parent
aarushyadav 31 minutes ago | parent
fowlie 1 hour ago | parent
rootusrootus 53 minutes ago | parent
thisisauserid 1 hour ago | parent
40four 29 minutes ago | parent
Good reference! That series of books Is one of my all time favorites.
frenchie4111 26 minutes ago | parent
Second, I agree with exactly this. Something shaped like an AirPod
PebblesRox 1 hour ago | parent
PebblesRox 1 hour ago | parent
OhMeadhbh 1 hour ago | parent
LatencyKills 54 minutes ago | parent
The app essentially manages a bunch of TODO items written in markdown.
derefr 44 minutes ago | parent
I think it's because most projects don't have any settled place (or at least, any place that lives inside the project's source-code repo) to record PRD-like information.
But because AI coding agents really do benefit from having PRD-like info (i.e. a hierarchical breakdown of the "intended what" and "intended how" of the system that "should" exist in the repo, kept updated to reflect changing ideas about those intentions), they really want to record that type of info somewhere, so they can then reference it.
AI coding agents don't seem to have any "taste" (or maybe "confidence"?) when it comes to asserting that even if you don't already have a separate PRD-like file in the repo, you should have one, if you want the agent to do its job well, and so it would request you to let it create one (or just automatically do it.) Instead, they just shrug and cram the PRD stuff into whatever structure you do already have. Which, if you're starting fresh with a greenfield project, is usually just a README.
(I hypothesize that if you tell an AI coding agent to use Behavior-Driven Development [BDD] on a greenfield project, it'll probably lean toward maintaining its PRD-like info in the form of BDD test cases. Haven't tried this experiment yet; would be interested to know the outcome!)
LatencyKills 36 minutes ago | parent
Throughout the document Claude kept claiming that issues were "mitigated but not fixed". They were essentially bandaid fixes for things it couldn't figure out. When I pointed this out to the author he said, "Well Claude thinks it works."
I wonder what happens to software when enough people try to ship projects like this.
aarushyadav 53 minutes ago | parent
jasonlotito 48 minutes ago | parent
I understood from just this. But the demo video impressed even further.
> Spent 30 seconds scrolling down to try to figure out what this is.
Did you try reading the site? Or maybe doing something other than just scrolling?
Does it need some form of subway surfer to get you to pay attention?
> Eventually clicked away from the site because it was taking too long to get to the point.
So yeah... subway surfer. This generation is cooked.
jstummbillig 1 hour ago | parent
tomhow 1 hour ago | parent
aarushyadav 1 hour ago | parent
That's why we weren't able to make something that can mount to your laptop. We're trying to make a necklace form factor at some point in the future though, but can't give a date on this!
dist-epoch 59 minutes ago | parent
adamsmark 1 hour ago | parent
Throat mics don't work at all - see reviews online.
This is a different approach, so I am cautiously optimistic.
TechSquidTV 1 hour ago | parent
aarushyadav 1 hour ago | parent
dang 53 minutes ago | parent
Edit: the more that I think about it the cooler and more important this feature seems!
hk__2 49 minutes ago | parent
aarushyadav 41 minutes ago | parent
popap 1 hour ago | parent
aarushyadav 58 minutes ago | parent
They get modulated since your mouth is a complex dielectric environment. Tissue, water, and air have different dielectric constants (different permittivity to RF) and the motion of the muscles in this medium leads to interactions with the waves that fill it, changing their phase and amplitude as they propagate.
hippycruncher22 1 hour ago | parent
doctorhandshake 57 minutes ago | parent
temp123477 55 minutes ago | parent
ShinTakuya 54 minutes ago | parent
augment_me 53 minutes ago | parent
qwertygnu 44 minutes ago | parent
While this tech doesn't feel quite as dystopian as most recent AI innovations, I do think it is a further step in the direction of human disconnection.
hk__2 41 minutes ago | parent
233mhz 34 minutes ago | parent
internet_points 20 minutes ago | parent
barbazoo 25 minutes ago | parent
ideasphere 57 minutes ago | parent
johnhamlin 56 minutes ago | parent
bluegatty 55 minutes ago | parent
And after some time I could not figure out what this is?
Put this part: " building smart glasses that let you talk to your computer without talking out loud. You can mouth words silently and discreetly and the glasses decode them to text at the speed of speech."
right at the top and show someone doing that instead of hiding behind cryptic marketing.
Also, the product looks great.
Get your words out of the way 'say what it is'.
Very amazing tech though
aarushyadav 50 minutes ago | parent
avree 54 minutes ago | parent
aarushyadav 49 minutes ago | parent
achandra03 38 minutes ago | parent
I've found the opposite. Typing feels much more "deliberate", and of course you can hit backspace but you can't un-speak a word. With prompting, I've found that, in general, speaking leads to lower-quality results than typing.
cogythea 36 minutes ago | parent
> with many people
Not everyone. I, at least, definitely overthink what I type, and while I end up with nice-ish sentences in my prompts it takes a lot longer than a train of thought that the AI parses correctly most of the time anyway.
chicagobuss 26 minutes ago | parent
I suspect this has to do with the nature of the work you're doing; I dunno, just count me firmly in the camp of people who only rarely feel like I can express a thought sufficiently for coding agents with their voice.
LorenPechtel 21 minutes ago | parent
briandw 15 minutes ago | parent
kqp 5 minutes ago | parent
xbar 47 minutes ago | parent
mrngld 30 minutes ago | parent
But happy to see people get more options.
This reminds me of Eternity's End by Jeffrey Carver and the implants they had. Released in 2000 but the more time goes on the more the neural implant technology is coming about. To be fair I think Enders Game had similar tech to detect subvocal speech like this product.
1dom 24 minutes ago | parent
Same. I'm often told I talk too fast, but I feel I communicate slowly because a lot of my fast speech involves halting and backtracking and such.
I can easily type 100WPM+, but I trivially blast out a few paragraphs of perfectly adequate text in a few seconds and then spend a few minutes re-editing it.
lstodd 29 minutes ago | parent
Zambyte 44 minutes ago | parent
egeozcan 44 minutes ago | parent
sspiff 41 minutes ago | parent
But I guess thinking is going out of style anyway.
That said, I know a lot of people prefer talking to their computer, especially when their computer is a phone. So there is definitely a place for stuff like this.
rickydroll 39 minutes ago | parent
jbs789 33 minutes ago | parent
lstodd 31 minutes ago | parent
1dom 28 minutes ago | parent
wongarsu 35 minutes ago | parent
ronyeh 18 minutes ago | parent
mirmor23 50 minutes ago | parent
Regarding the website, you should lead with product, its details, rather than founders shown in 360 degrees from various vantage points. It is distracting and frankly irritating.
sourpanda 50 minutes ago | parent
ElijahLynn 8 minutes ago | parent
This is just really groundbreaking and I know the research has been out there for a little while but to actually implement is a whole nother ball game.
I think the use cases for this will expand as the product ships and gets out there!
_jholland 4 minutes ago | parent
"In the long, windowless hall, with its double row of cubicles and its endless rustle of papers and hum of voices murmuring into speakwrites, there were quite a dozen people whom Winston did not even know by name"
gh5000 50 minutes ago | parent
80fj0348i 32 minutes ago | parent
There is a reason secure activity happens within secured rooms built to a specific standard https://en.wikipedia.org/wiki/Sensitive_compartmented_inform...
proee 49 minutes ago | parent
aarushyadav 11 minutes ago | parent
joshstrange 46 minutes ago | parent
sixtyj 45 minutes ago | parent
All of them are visionary and we would think that ppl gonna like them…
Well… ordinary people want boring tech.
This will be similar with whispering to glasses… Visionaries will like it… and return to missionary (boring) s*x (I hope that I don’t get a ban.)
Myself, I would buy these glasses immediately. But I would hate them as well as I would think I am insane (who wants to talk to yourself?).
Edit: Dang replied. I wish all the best to the team behind “whisper glasses”. It is amazing. Connect it with ElevenLabs and you rule the world. (Previous: It wasn’t meant a hate speech.)
dang 40 minutes ago | parent
A tasteless sex reference is one thing, but shallow-curmudgeonly dismissal of new work is the real sin here!
sixtyj 23 minutes ago | parent
But I think (and see) that people prefer boring tech. Whisper glasses is amazing. But most people will probably find it creepy. (Pareto applied.). I hope I am wrong. These guys are like from sci-fi.
sschueller 43 minutes ago | parent
smokel 42 minutes ago | parent
Edit: mostly concerned about RF near my eyes for 8 hours a day.
hirako2000 38 minutes ago | parent
HIP_HOP 37 minutes ago | parent
aarushyadav 15 minutes ago | parent
Also, the RF transmission only activates when you're actively dictating.
replete 41 minutes ago | parent
polartx 39 minutes ago | parent
mrtesthah 37 minutes ago | parent
https://www.cnet.com/tech/services-and-software/apple-qai-ac...
Hope you guys are well-positioned patent-wise.
aarushyadav 10 minutes ago | parent
They use speckle patterns of a laser shone onto the skin, which is sensitive down to tens of nanometers. Pretty cool tech
kenferry 34 minutes ago | parent
aarushyadav 18 minutes ago | parent
collabs 33 minutes ago | parent
So this could take "noise canceling" to a whole new level if you could somehow "speak" across a loud bar or across the world, for that matter. Being integrated into something like Signal or similar end to end encrypted platform would be interesting as well.
aarushyadav 20 minutes ago | parent
clkao 32 minutes ago | parent
json_hero 32 minutes ago | parent
philsnow 14 minutes ago | parent
monkeydust 32 minutes ago | parent
MPSimmons 24 minutes ago | parent
intrasight 20 minutes ago | parent
lukko 23 minutes ago | parent
Really like seeing the early prototypes here, and would love to see more on how the RF tech works on the website.
Are you able to patent it?
dogscatstrees 19 minutes ago | parent
aarushyadav 15 minutes ago | parent
jdironman 14 minutes ago | parent
mmaths 14 minutes ago | parent
popupeyecare 11 minutes ago | parent
Can we use RF for other medical applications? As an anesthesiologist, we do nerve blocks before surgeries or for pain. One rough indicator that it's working is muscle paralysis. Also similar for botox.
Very interested.
howunfortunate 10 minutes ago | parent
1. Insanely cool
2. Gives me serious "whispering earring" vibes. Imagine having a full on conversation with an AI via subvocalization and listening with ear bud to guide you throughout the day. https://gwern.net/doc/fiction/science-fiction/2012-10-03-yva...
contingencies 9 minutes ago | parent
airbreather 8 minutes ago | parent
vessenes 7 minutes ago | parent
Second, how tough is this signal processing? Will it be done on-glasses, or through an app? Third, what's the equivalent of a wake word for these? I'm reminded of the Apple Vision Pro radar to see if your fingers are clicking. I could imagine a lot of modalities where I want to just start talking and have things happen, and a lot where I don't want the glasses to pick it up. Fourth, I think a cool voice change demo would be rad -- not sure what latency is like, but if you could get it end to end under like 250ms you could probably have someone exaggerate the mouth movements and pipe out audio of them talking like someone else. Which would be another really cool demo.
Super cool. You're going to be working 80 hour weeks for the next year, I'm excited to see what you put out!
0x4e 5 minutes ago | parent
I don't think it is an intuitive action at all and I became self-conscious about not making a sound rather than talking like I normally would.
I suppose it would require some "getting used to."
Regardless, I think it is a cool idea and a real problem. Sometimes I don't want to type nor talk out loud so I find myself whispering at my phone like I'm telling it a dirty secret.
--
As I'm writing this, I think this kind of product might not hot the market for consumers, but I'm seeing this used by Flock like companies to spy on people's speech, so kudos on making machine lip reading a thing!