YHK - yet another hacker news clone

To the author: 100% need this kind of thinking. It’s a balancing act. Historically though, dedicated labs haven’t had the best track record at commercializing their tech. The skillset differs vastly from running a product-focused business.

That said, even within this narrow scope - there are areas ripe for research.

Some of my particular interests include:

(1) Latent thinking/reasoning, which could arguably be more efficient but also opaque is one

(2) Separation of “intelligence” from knowledge - ie. keep the ability to reason like a frontier model and make tool calls (calling the ‘oracle’ or searching the web) but remove all parameters associated with niche knowledge/facts.

These are a couple off the top of my head that are very much aligned to scaling.

That statement is broad enough to be unfalsifiable, but if you narrow it down just a bit to scaling LLMs, maybe there is an end in sight. At least financial markets are skeptical of there being no end in sight.

If scaling LLMs really was the road to God in a box, then compute demand would be infinite, and nobody building to meet that demand would have a junk debt rating. Did those money people just not get it?

Almost all statements Yann LeCun made back then (LLMs won't be able to do this or that) turned out to be wrong. They were right to fire him.

Already happening: https://news.ycombinator.com/item?id=50397994

Obviously we'll see how it pans out, but other approaches feel a bit like trying to ignore the bitter lesson.

Is that observed scaling part of some equation or law? Is there a limit to how far that scaling goes before a ceiling or diminishing returns hit? Or is it simply based on more GPUs/parameters/compute plus how much human knowledge is in existence and will scale predictably forever?

Where's the money coming for those 10X and 100X ventures?

P.S. Getting downvotes (multiple) for basic economic facts is ...something. Someone flunked their economy 101.

Remember Bell labs, XEROX parc? Ask yourself why that existed and why research came out of them.

Monopoly.

So - many people don’t want to admit this but monopolies can be good from an R&D perspective, but without strong oversight by regulators - money will instead spent on R&D projects that specifically raise entry barriers.

https://en.wikipedia.org/wiki/Digg#2025%E2%80%93present:_The...

Seems like it quickly pivoted quickly to cash in on "AI" hype.

The working environment in academia is often precarious :-( , so such people often have no alternatives to going into industry, even though they still deeply appreciate and are influenced by the values of academia.

Perhaps it is more free, but my understanding has always been that professional academia still restricts freedom of research. You have to apply for funding, convince the board that your direction will be fruitul.

An archaeologist can't just dig wherever. They compete with their peers for a limitted funding pool. If the school's board is more interested in the bronze age than the paleolithic, you're probably not going to get your dino dig approved.

The same is true for CS, I assume. You can't just "i would like spend university resources developing brainfuck 2.0" because the other teams are more likely to get their funding over you getting yours.

Both corporations and university's have finite budgets, both with bottom lines and margins. The uni wants their research done in a popular field that will attract students because students generate income that funds research that attracts students that generate income that funds research that...

They just think it makes sense to explore things, rather than just rinse and repeat with more cash.

The good news for John is that when 99.99% of the competitors are doing all the same, it’s when the rewards for looking around can be the biggest. Those are the real moments for disruption, and companies disregard them at their peril.

Some are, some aren't. Some try, but botch it.

That's just patently not true. History is littered with products that were very innovative and useful (and often far superior to competing products), but were canned, bought by competetors, or shelved in other way - often because they threatened the profits being made by some entity (though that's not the only reason).