Why Algorithmic Systems Possess No Understanding

Apr 09, 2018 - 4:45 pm to 6:00 pm

Campus, Oak Lounge at Tressider Union

Sir Roger Penrose Special Seminar

Video recording can be seen here.


Many examples of highly effective algorithmic systems, such as AI devices, have been constructed in recent years. We have computer-controlled machines like self-driving cars and algorithmic systems that play chess and GO at levels that can out-perform even the best of human players. But do such devices actually “understand” what they are doing, in any reasonable sense of that word? I argue that they do not, and as an illustrative example I present a recently composed chess position that a human chess player, after briefly examining it, would correctly conclude that it is an obviously drawn position. Nevertheless, when it is presented to the top-level chess-playing program Fritz, set at grandmaster level, Fritz incorrectly claims that it is a win for the black pieces and eventually Fritz blunders dreadfully (though “correctly” according to its algorithm) to be soon check-mated by white. This demonstrates Fritz’s remarkable lack of any actual understanding of the game of chess, despite its vast computational abilities.


More sophisticated examples come from mathematics, most particularly with human understanding of the infinite, and it can be shown that this quality cannot plausibly be encapsulated by any algorithm arising from the processes of natural selection. I argue that the quality of understanding is a feature of consciousness, and that consciousness can come about only through physical processes not yet properly understood, most likely at the boundary between quantum and classical processes, as argued for in the Orch-OR proposal.


Penrose is known for his work in mathematical physics, in particular for his contributions to general relativity and cosmology. He has received several prizes and awards, including the 1988 Wolf Prize for physics, which he shared with Stephen Hawking for the Penrose–Hawking singularity theorems.

More detailed biographical information at his Wikipedia page <https://en.wikipedia.org/wiki/Roger_Penrose>  and the Encyclopedia Brittanica <https://www.britannica.com/biography/Roger-Penrose> .