27 points gafferongames 4 hours ago 15 comments

27183 4 hours ago | parent

> It’s informed by trend extrapolations, wargames, expert feedback, experience at OpenAI, and previous forecasting successes.

In other words: bias. Tons and tons of self-congratulatory, glue sniffing bias.

Mond_ 21 minutes ago | parent

> Hacker News Guidelines

> Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something.

sigalor 4 hours ago | parent

I wonder when the people will get that intelligence is not only directed at the external, but only really starts when you look at the internal (joy, pleasure, traumas, taboos, awkwardness, abuse etc.). Look up the word "interoception".

glenstein 4 hours ago | parent

>Agent-2, more so than previous models, is effectively “online learning,” in that it’s built to never really finish training. Every day, the weights get updated to the latest version, trained on more data generated by the previous version the previous day.

That seems to me to be a natural progression, from discrete models to models that are just continuously improved. Maybe we'll end up with different models with different rates of improvement rather than static differences in performance, and methodologies for that improvement will be the thing we care about. Maybe over time, even benchmark tests will be primarily concerned with that kind of efficiency.

I think a huge debate right now is the relative value of the "frontier" models from Western companies at the cutting edge, vs distilled versions of those models that are good enough and exponentially cheaper coming from China. But a paradigm of 'always training' means an always active, always advancing frontier, which is a stronger moat than a one-off model that's more advanced for a few months.

Topology1 1 hour ago | parent

Mom said it was my turn to post this

Ydarbleoj 1 hour ago | parent

Sorry, but the notion that creative writing and robotaxi's exist as proof of anything is like saying my child can drive and write; and while true, the measure of that ability is not at the level of the best humans. It's average at best.

weezing 1 hour ago | parent

At which point do we get to fight in the anti clanker uprising?

firemelt 1 hour ago | parent

https://isaiprofitable.com/

I like this one better

ealready_value 54 minutes ago | parent

Oh look, the answer is the same when I looked a couple months ago. Interesting

reducesuffering 15 minutes ago | parent

This is exactly how Cloud Computing looked in 2012-2018. Dumping huge $ into computing buildout that wasn't profitable yet. All those co's: Amazon, GCP, Azure paid off immensely and are ridiculously profitable.

HPsquared 13 minutes ago | parent

Hey now, some of them reach -50% profit margins!

api 8 minutes ago | parent

On the other hand, people said this about Amazon, Google, Meta/Facebook, and loads of other huge tech things that are now wildly profitable.

It hasn't been a reliable predictive metric so far. All runaway growth tech sectors and businesses tend to look economically insane until they're not.

AlexErrant 14 minutes ago | parent

My main issue with this timeline is that AI still has trouble transitioning to the real world. It predicts for 2029:

> There are swarms of insect-sized drones that can poison human infantry before they are even noticed; flocks of bird-sized drones to hunt the insects; new ICBM interceptors, and new, harder-to-intercept ICBMs. The rest of the world watches the buildup in horror, but it seems to have a momentum of its own.

Does anyone really predict insect drones _in production_ 2 years from now, to the degree that we need bird drones to hunt the insect drones? How the hell are these things powered?

Lean/math/millenienium prizes are "grindable" [0]. Wake me up when AI is making order-of-magnitude improvements in ungrindable real world tasks like batteries, hypersonic engine manufacturing, and stealth/silent motors that you can't hear.

[0]: https://www.dwarkesh.com/p/the-next-paradigm

api 7 minutes ago | parent

Interesting but it degenerates into sci-fi tropes if you look at the extrapolations. Reminds me of 90s writing about what the Internet was going to do.

The Internet ended up being both more incredible and more mundane than predicted.

waffletower 50 seconds ago | parent

Fun to read this again and see actual parallels. The 2030 Takeover section is such a ludicrous leap, however. None of the supply chain infrastructure, energy, or Moravec's Paradox realities are ever addressed. Turn the page and suddenly humanity is largely annihilated with a Corgi-esque human breed kept as pets. How did these robots emerge from utter rhetorical nothingness? Robocalypse impossible? Perhaps not. By 2030? an intellectually embarrassing farce worthy of a facepalm.