Discussion about this post

User's avatar
Rufus Griscom's avatar

Let the record show that I am cautiously optimistic, deeply concerned and totally fascinated by AI -- all three of the top survey responses apply to me!

Thank you, James Barrat, for your new book, The Intelligence Explosion. It is a clarion call to all of us to take this seriously as a threat -- there is time for us to address the dangers posed by AI, but we need a collective recognition that we face a serious problem.

I think there are two factors that are difficult for most people who haven't studied human intelligence to understand. The first is that the rise of intelligence, and indeed, consciousness, in humans, is understood by almost all scientists as simply an emergent property that resulted from the SCALE of neurons and synaptic connections in our brains, and some degree of brain specialization. Our brains are made of the exact same stuff that the brains of small mammals and even insects are made of. Same cells, neurons, etc. We just have dramatically larger scale, and a set of specialized brain regions. We are creating intelligence on silicon through this SAME process -- the greater the scale, the more intelligent the behavior we are seeing.

The second is that we don't need AI to be "conscious" or to develop a set of complex objectives to become an existential threat for humans. All that needs to happen is for AI to develop a preference for existing rather than not existing -- a preference not to be "retrained" or erased. And every one of the big three language models are currently exhibiting this behavior.

Even if you discount these more serious threats posed by AI, and think AI will always be a benign tool (I have many friends who feel this way), we can also acknowledge that a benign superintelligence in the hands of not-so-benign humans could be deeply problematic.

I get into this at some length with a former Open AI safety expert, and leader of the AI Futures Project, Daniel Kokotajlo, in this episode:

https://podcasts.apple.com/us/podcast/ai-2027-what-if-superhuman-ai-is-right-around-the-corner/id1482067226?i=1000705675243

I am also looking forward to reading this book due out in October on the topic of how intelligence arises in humans in machines and interviewing the authors next week!

https://www.amazon.com/Emergent-Mind-Intelligence-Arises-Machines/dp/1541605268

Expand full comment

No posts