YHK - yet another hacker news clone

How do we know if/when that happens if it’s beyond us to comprehend. An open ended and broad claim like this can always become true one day, similar to flying cars.

As long as humans are in charge it's vital for us to understand what is going on - be it in science or coding, once you give up all knowledge (yes knowledge = power) not just work to AI, you can be manipulated/controlled/fooled with ease.

No need to start worrying about AI reaching a level of intelligence far beyond human capabilities until it actually reaches the level of humans and assuming that ever happens it could be a very very long way off. All of the intelligence "AI" has right now comes from the human data that it was trained on. It's been given enough power to crunch numbers better than we do, as any computer can, and more data to play with than any one human can retain, but none of that equals intelligence. Speculating on a sci-fi future however, if AI is ever able to create new and more advanced forms of AI, which can also create new and and more advanced forms of AI we'll be outpaced quickly.

Ive never heard such drivel in my life. Are you a shill? The current "AI" is not intelligent, it can beat humans in certain areas but weve always had that with things like Deep Blue. I dont believe llms will reach human intelligence and certainly not super intelligence.

I have doubts about that. You don't need AI (understood as the current LLMs) to make a powerful chess engine, or computer vision or anything that reacts faster and better than a human. We've had those forever.

A good word predictor happens to be good with closed systems with very defined rules for which a huge training corpus exists and we are suddendly freaking out about how it's gonna become so much better than us at everything that we are basically obsolete.

When in doubt about where AI stands, ask it to create a poem, a joke or a riddle. And see how little actual intelligence it has.

Practical impact has always been linked but ultimately independent of knowledge. You can have very sophisticated technology based on zero actual knowledge or understanding, like roman concrete or borosillicate glass. And you can have a very deep understanding without practical impact, like lasers for decades or galactic algorithms for multiplication.

The problem is trading off short term and long term progress. A solution alone can be useful in the short term, but if humans don't also build up understanding it will probably prevent long term progress from happening. It's much more difficult to build on top of something you don't understand.

> their practical impact can be independent of whether humans fully understand how they were reached

See: most of pharma

Indeed, if a machine had told Heisenberg in 1943 (or whatever?) that "the critical mass for an atomic bomb is much less than you think", we could very well have had a different world today, regardless of Heisenberg understanding it or not.

Meaning requires integration. But I assume AI can make meaning of it, now or in the future. I assume the same with people. However I don’t see the point of knowledge without meaning.

> we can surely have the verifiers explain those

This is not obvious to me at all. LLMs are capable of amazing feats of reverse engineering but are terrible at explaining and abstracting things well.

I think this is an artefact of how we train them, and they could be trained to be better at this if we focused on it, but at the same time, it shows its possible for these things to be able to do things they cant explain to us (granted, they can't explain them well - you can bash your way to understanding with them anyway but they aren't good at it)

I don't think we're there yet for science. Perhaps for math proofs, but that's a very different domain ultimately since we know the rules of the game in a way we don't with science.

I sucked at remembering goniometric identities, but I was superfast with testing out a few options on my calculator to make sure I was using the correct one. Without a calculator I was nowhere.

Even Peter Scholz is not always sure of correctness of extremely complicated mathematical statements and welcomes computer assistance in verification of his results.

See the Liquid tensor experiment:

https://xenaproject.wordpress.com/2020/12/05/liquid-tensor-e...

https://www.ma.imperial.ac.uk/~buzzard/xena/pdfs/liquid_tens...

" I spent much of 2019 obsessed with the proof of this theorem, almost getting crazy over it. In the end, we were able to get an argument pinned down on paper, but I think nobody else has dared to look at the details of this, and so I still have some small lingering doubts. "

In a sense, mathematics is not very different from software engineering.

More precisely: Curry–Howard correspondence

https://en.wikipedia.org/wiki/Curry%E2%80%93Howard_correspon...

I did not; thank you.

One bit stands out to me: to better understand, Scholz will work the problem backwards. Sometimes that strategy is just so spot on.

--

To my eternal shame, philosophy (like this OC) makes my head hurt. Pretzels my brain, ouroboros like (aka "head up my own butt").

Caveats covered, here's my cave man cold take on two aspects of this topic.

#1 As we're all trying to paraphrase here: answers are great; understanding is even better. I like the word "legibility" (h/t James Scott). The current models are not very legible.

Which directly leads to:

#2 Understanding enables predictions.

If LLMs are stochastic parrots, then pi.dev (GPT + reasoning + ralph loop) is a drunken sailors walk (thru a solution space). A kind of messy heuristic for optimization. With some kind of gate for declaring "done!" Pretty useful.

However, I don't expect Pi.dev to make predictions. That'd require something else. Maybe that something is being worked on; am noob, so I wouldn't know. (I'm still unclear about whatever Jeff Hawkins is doing, in practical terms.)

--

Wrapping up: Automated prediction would be cool. That's very different from finding solutions to problems. And the current cohort of AI can't even explain why, much less make predictions.

Assuming predictions requires understanding. Maybe there are other strategies to generate predictions. Like a meta drunken sailor's walk, with drunken sailors all the way down.

(Someone please correct me as needed. TIA.)

[Aside: That article has an ad for their podcast They Joy of ₍₎. Perfect. I have no idea if it lives up to its promise.]

Maybe but it might also be useless. Knowing a very large number is the product of two very large primes doesn’t at all help the reader to understand factorization, for example.

  Knowing that something is true or false

Assuming you can trust it