Thanks, Steve. Yes, Hofstadter and Penrose are great suggestions. Anything I say about them would only be parroting Deutsch, so instead I’ll just reference a couple of examples as they related to those authors.

The Fabric of Reality includes a discussion of Penrose’s theory as entailing the discovery of an entirely new physics that would not support computational universality (i.e. he argues that the Turing principle is false). Under this theory, it wouldn’t be possible to construe some thinking as computations at all. Given that the prospects for AGI are tied to the Turing principle, and Penrose’s criticisms of it are both rare and credible, his ideas in this area are recommended reading.

Similarly with Hofstadter, another on Deutsch’s recommended reading list. In The Beginning of Infinity, Deutsch discusses Hofstadter’s insights into the reality of abstractions and (contra Hofstadter) the error of reductionism. Again, the abstraction of intelligence (and people) is foundational to the prospects for AGI.

Deutsch focuses on what universality entails (e.g. intelligence is not dependent on human models or forms), and why we can’t increment from today’s narrow AI to AGI (due to the creativity gap discussed in my article).

Thanks again for flagging these authors.