Artificial Intelligence (AI) is currently a hot topic that’s being talked about in many media outlets as it continues to roll out. It still remains to be seen whether this activity will begin to cool off, as often happens with new technologies, or truly start to transform society.

Even a brief look at the ways that AI is being used makes it clear that there are enormous possibilities for both good and evil. This is certainly true in the short term, because people will find both good and evil things to do with it like we do with virtually all new tools.

There are even greater uncertainties in the long term, however, when we will start to deal with things like Artificial General Intelligence (AGI), which may be able to solve problems that humans cannot, and machine agency and consciousness, which will raise the possibility of AIs acting on their own in ways that may not be good for humanity.

A common theme across all these possibilities is the great uncertainty about what will happen. One of the sources of this uncertainty is the level of mystery in our understanding of how both machine and human intelligence works. It’s common to hear how current AI developers are surprised at the things that their creations are able to do. This uncertainty is mirrored by the amount that we still don’t understand about the human mind, which is often the primary model for designing these systems.

Consciousness, for example, still remains a great mystery even though all humans experience it. Despite this universal experience, and many different investigations, it still isn’t clear whether consciousness can be fully explained with strict materialism. This has resulted in a growing number of researchers proposing additions beyond the current materialist models that enable consciousness. They do this in ways that avoid having to resort to the supernatural, yet still point to something beyond the physical.

In other words, there seems to be a persistent metaphysical aspect to consciousness that has not yet been explained. Our lack of understanding of consciousness extends to its most fundamental nature, illustrating our lack of understanding of the nature of reality itself.

While such immaterial viewpoints are actively rejected by many researchers, it’s also well-known that there is a deep human need for the metaphysical, for transcendence. So while there is still this uncertainty regarding the mind, we need a story that allows us to remain human in this way, that allows us to accept the personal experience of self as something real and not just a tricky behavior of matter.

We need to keep open the possibility of a fundamental difference between humans and machines.