Beyond Turing Machines

Bey ond T uring Mac hines ∗ Kurt Ammon www.cst ruct.o rg Abstract This p ap er discusses ”c omputational” systems c ap able of ”c omput- ing” functions not c omputable by pr e define d T uring machines i f the systems ar e not isolate d fr om their env i r onment. R oughly sp e aking, these systems c an change their finite descriptions by inter acting with their envir onment. 1 In tro duction T uring [8] in t r o duced the concept of ”computing machine s” whic h subse- quen tly w ere called T uring mac hines. He pro v ed that Hilb ert’s decision prob- lem (E n tsche idungsproblem) is unsolv able, that is , there is no T uring machine determining whether or not a give n statemen t in first-order predicate calcu- lus (a mathematical prop osition in n um b er theory) can b e prov ed. W egner [11] writes that T uring’s precise c haracterization of what can b e computed established the resp ectabilit y of computer science as a discipline. He ar g ues that T uring machines cannot capture the in t uitive notion of what computers compute when computing is extended to include in teraction. His in teraction mac hines ha ve b een criticized as a n unnecessary Kuhnian para digm shift [12]. Prasse and Rittg en [7] write that W egner’s ”in teraction machine s cannot compute non-recursiv e functions, so Ch urc h’s thesis still holds”. This im- plies that interaction machines cannot ”compute” functions not computable b y T ur ing mac hines. ∗ This work is licensed under the Crea tive Commons Attribution-No Deriv ative W ork s 3.0 Unp orted License (s ee h ttp://creativecommons.org /licenses/by-nd/3.0/ ). 1 This pap er pro ves that there is no T uring mac hine pro ducing a seque nce of all computable functions o n the set of natural n um b ers. The pro of im- plies the existence of ”computat io nal” systems that cannot b e mo deled b y predefined T uring mac hines if the systems are not isolated from their env i- ronmen t. Roughly sp eaking, these systems c hange their finite descriptions b y in teracting with an en vironmen t whose dev elopmen t is not completely predictable for practical a nd theoretical reasons. I arg ue that the pro of ev en applies to existing systems suc h as t he In ternet. Finally , I in tro duce a new t ype o f systems b y requiring t hat they b e capable of ”computing” functions not computable b y T uring mac hines. 2 Incomplete ness Theorem A c om p utable function is a function t ha t can b e computed by a T uring ma- c hine, that is, it can b e represen ted by an ordinary computer program. Th us, a computable function on the set of na tural n um b ers N in to N can b e re- garded as a computer pro g ram pro ducing a natural n umber in its output from an y natura l n umber in its input. An example of suc h a function f is f ( n ) = n + 1 whic h pro duces the success or n + 1 of an y natura l n um b er n . The computable functions on the set of natural n um b ers can b e rega rded as mo dels of computer programs and systems . The restriction to computable functions on the set o f natural n um b ers is not relev an t b ecause inputs and outputs of computer programs a nd systems are represen ted a s binary digits. Useful programs and sys tems should w ork for defined sets of inputs which corresp ond to subsets o f the set of natura l n um b ers. Such a definition can b e extended to all natural n um b ers by assuming a default output for inputs for whic h the program or system is not defined. Ordinarily , a program or system is defined o n a decidable set o f inputs, that is, there is a program deciding whether or not an input is a dmissible. Thus, t he computable f unctions on the set of natural num b ers mo del an imp ortant class of computer programs and systems . Incompleteness Theorem: There is no T uring mac hine pro ducing a sequence of all computable functions f 1 , f 2 , ... on the set of natura l num b ers N in to N . Pro of . W e assume t hat there is suc h a T uring mac hine. W e define a new computable function g on the set of natural n umbers by g ( n ) = f n ( n ) + 1 for all natura l n umbers n . Because the sequence f 1 , f 2 , ... contains all 2 computable functions a ccording to our o riginal assumption, there is a natural n umber n suc h t hat g ( x ) = f n ( x ) for all natural n umbers x . This immediately yields the contradiction g ( n ) = f n ( n ) and g ( n ) = f n ( n ) + 1 b ecause of the definition of the f unction g . This means that our original assumption is false, that is, there is no T uring mac hine pro ducing all computable functions f 1 , f 2 , ... on the set of natural n um b ers. The construction of the computable f unction g in the pro of can b e rep- resen ted b y a T uring machine T . Of course, T can b e incorp orated in to an y T uring machine. The pro of implies t hat no T uring machine can pro duce all computable functions f 1 , f 2 , ... on t he set of natural nu m b ers no matter ho w T is incorp orated. In particular, T can b e incorp orated in t o the T uring mac hine whose existence is assumed in t he pro of so that the extended T ur- ing mach ine pro duces t he sequence f 1 , f 2 , ... and the function g . But the extended mach ine is differen t from the original mac hine and thus its applica- tion w ould contradict our assumption that there is a (single) T uring mac hine pro ducing all computable f unctions on the set of natural num b ers. Because an y f ormal system can simply b e defined as a theorem-proving T uring mac hine (see, for example, [4, p. 72]), the theorem also implies that an y formal theory is incomplete in the sense that it cannot ”capture” all computable functions on the set of natural n umbers. 3 Creativ e Systems Let C b e a system capable of p erforming the reasoning pro cesses required for pro ving m y simple incompleteness theorem. Th us, C is capable of construct- ing a computable function g on the set of natura l n umbers not pro duced by an y giv en T uring mac hine M . F ig ure 1 illustrat es the corresp onding pro of whic h shows that there is no predefine d T uring mac hine M pro ducing all computable functions f 1 , f 2 , ... on the set of natural n um b ers C c an con- struct. A difference b et w een C and a T uring mac hine is that C is not regarded as a static predefined ob ject isolated fro m its env ironmen t. Rather, C and t he T uring mac hine M in the pro of are regarded a s separate in teracting en tities in the sense t ha t C assumes the existence of a T uring ma chine M pro ducing all computable functions f 1 , f 2 , ... on the set of natural num b ers in order to construct a computable function g not pro duce d b y M . Roughly sp eaking, C can c hange itself b y in teracting with its en vironment, that is, the T uring 3 C ✲ ❄ M ✲ f 1 , f 2 , ... g 6 = f 1 , f 2 , ... uses c o n struct s pr o duc e s Figure 1: Pro of mac hine M . Because the capabilities of C t o construct computable functions cannot b e mo deled by any predefined T uring mac hine, anot her mo del of such systems seems to b e required. Computable functions can be represen t ed by T uring machin es or com- puter programs, that is, they hav e finite descriptions. The set of suc h de- scriptions can b e effectiv ely en umerated. This means that there is a T uring mac hine generating a sequence con taining all finite descriptions, for example, in a scending length. F or these reasons, m y simple incompletenes s theorem implies that the function deciding whether or not a giv en description repre- sen ts a computable function on the set of natural n um b ers is no t computable. If this function w ere computable, its a pplication to an effectiv e enumeration of descriptions w ould yield an effectiv e enume ration of all computable func- tions on the set of natural n umbers. This contradicts m y theorem. Th us, there is no T uring mac hine deciding whether or no t a give n description is a computable function, that is, this decision problem (En t sche idungsproblem) is unsolv able. F urthermore, m y simple incompletene ss theorem implies that system s capable of proving this theorem seem to b e capable of ”solving” the ab o v e decision problem in the sense that they can construct more and more p o w erful computable functions on the set of natural num b ers b eyond the limits of any predefined T uring mac hine. This suggests the f ollo wing definition of a new t ype o f systems : Definition: A system is called cr e ative if it is capable of ”com- puting” non-computable functions, that is, determining v alues of non-computable functions fo r giv en arg umen ts. 4 With regard to this definition, creativ e systems can con tain an y computable function. Th us, they may b e regarded as an extension of the concept of T uring ma chines in the sense tha t they can ”compute” computable and non- computable functions. An architecture of creativ e systems w a s dev elop ed on the basis of ex- p erimen ts with the SHUNY A T A progra m [1, 2]. It is the first step to w ards the implemen ta tion of a creative system. Roughly sp eaking, a creativ e sys- tem comprises a reflection base and a v arying n umber of evolving analytical spaces. Definition: The r efle c tion b ase con tains a univers al pro gram- ming language, elemen t a ry domain-sp ecific concepts, and kno wl- edge ab out this language a nd t hese concepts. Because all computable functions can b e represen ted in a univ ersal pro gram- ming language, m y simple incompletenes s theorem implies that the reflection base cannot b e formalized completely . Definition: Analyt ic al sp ac es con tain partial kno wledge whose domains of application are limited but o rdinarily expand in the course of the dev elopmen t of the analytical spaces. Roughly sp eaking, the dev elopmen t of new knowle dge in creative systems can b e summarized in the f ollo wing principles: Principles of Dev elopmen t: 1. The know le dge in analytic al s p ac es arises fr om the r efle ction b ase and pr e c e ding know le dge. 2. The development of know le dge involves the gener ation of new analytic al s p ac es and the unific ation of existing ones. 3. The e c onomic al variations of new know le dge tend to b e pr e- serve d and the une c onomic a l ones to b e destr oye d. F or example, the reflection base ma y contain elemen tary kno wledge ab out constan ts and functions of a progr a mming language. A ve ry simple programming t a sk is the construction of a progra m pro duc- ing the success or n +1 of an y natural n umber n in its input. This prog r a m can b e constructed o n the basis of the elemen tary kno wledge that 1 is a nat - ural n um b er a nd x + y is a natural num b er for any natural nu m b ers x and y . 5 Another simple ex ample is the construction of a Quic ksort program sort ( L ), whic h sorts the elemen ts of a list L according to a giv en order relation ” x < y ” ( x is less than y ) b et w een any elemen ts x and y o f L . The core of suc h a program can b e r epresen ted by the functional pseudo co de app end ( sort ( x ∈ L : x < first ( L )) , first ( L ) , sort ( x ∈ L : first ( L ) < x ))) , (1) where the app end function app ends lists and first ( L ) is the first elemen t of a list L . In order to sort a list L , the program (1) sorts the elemen ts x ∈ L that a re less than its first elemen t first ( L ) and the elemen t s x ∈ L that ar e greater than first ( L ). The recursiv e a pplication of this ”divide-and-conquer” strategy yields smaller and smaller partial lists or empt y lists whic h need not b e sorted. Finally , the progra m (1) generates a sorted list con taining all elemen ts of L b y success iv ely app ending all partial lists previously sorted. The Quick sort pro gram (1) can also b e constructed on the basis of ele- men tary knowle dge ab out the functions it contains. Roughly sp eaking, this kno wledge need merely give the domains and ranges o f the functions, that is, the sets on whic h the functions are defined and the sets of v alues the functions may tak e on. F or example, the prop osition x < first ( L ) in (1) can b e constructed on the basis o f the knowle dge that first ( L ) is an elemen t of L and x < y is a prop osition for an y elemen t s x ∈ L and y ∈ L . An efficien t selection o f the functions used in the Quic ksort prog ram (1) seems feasible b ecause they are v ery elemen tary suc h as the ap p end function or ev en explicitly con ta ined in the progr a mming task suc h as the order relation ” < ”. F or example, the SHUNY A T A program generated theorem-proving pro- grams by analyzing pro ofs of simple theorems on the basis of elemen tary kno wledge ab out the functions the progra ms are composed of [1]. The de- v elopmen t of these programs illustrates very simple asp ects of the principles giv en ab ov e, for example, the unification of analytical spaces and the gen- eration of economical v ariations. The theorem-prov ing programs dev elop ed b y SHUNY A T A generated pro ofs of further theorems, in particular, a pro of of SAM’s Lemma whic h is simpler t ha n an y pro of kno wn b efore. The com- plexit y of SAM’s Lemma more or less represen ted the state of the art in automated theorem proving [1, p. 561]. G¨ odel’s incompleteness theorem sa ys that ev ery formal num b er theory con ta ins an undecidable form ula, that is, neither the fo rm ula nor its negation 6 are pro v able in the theory . The main problem in the pro of of G¨ odel’s theorem is the construction of suc h a formula. Analogo usly to the construction of the Quic ksort program (1), an undecidable formula can b e constructed on the basis of elemen tary rules for the format ion of formu las, that is, on the basis of elemen tary kno wledge ab out the sym b ols the f o rm ula contains. Ammon [2] describ es a pro of of G¨ odel’s theorem and refers to further exp erimen ts with the SHUNY A T A pro gram. 4 Discuss ion The In ternet is a net w ork of a v arying num b er o f computer programs and sys- tems. My simple incompleteness theorem implies that no predefined formal system can completely mo del suc h a net w ork b ecause a program (computable function) not ”captured” b y the formal system can b e constructed and added to the net work. The pro of of m y theorem implies that the construction of this program can b e achiev ed a uto matically . Systems capable of communic ating and in teracting with humans more naturally than existing systems should b e capable of reconstructing com- putable functions implicitly used in h uman forms of comm unication. My theorem implies that there is no predefined T uring machine or formal system mo deling this comm unication. Computer programs m ust b e tested and debugged b efore they can b e used in practice. My theorem implies that there is no general predefined alg o rithm for the v erification of programs b ecause a pro gram (computable function) not ”captured” b y the algorithm can b e constructed from the algorithm itself. Rather, the v erification of progra ms is ac hiev ed on the basis of exp erience. The T uring mac hine pro ducing a sequ ence of computable functions f 1 , f 2 , ... in the pro of of m y incompletenes s theorem can b e regarded a s an analytical space. The construction of ano ther T uring machine pro ducin g f 1 , f 2 , ... and a computable function g not pro duced b y the original mac hine can b e regarded as the construction of a new a nalytical space. Th us, my simple theorem implies t ha t all ana lytical spaces cannot b e unified into a single analytical space. Ro ug hly speaking, the dev elopmen t of new kno wledge in analytical spaces cannot b e regarded as a completely describable ”closed b o x”. Ra t her, it is an op en pro cess whic h ma y transcend a ny frame sp ecified in adv ance. My w ork can also b e regarded as an inv estigation with the aid of com- 7 puters why Hilb ert’s decision pr o blem ( Entsc heidungsproblem) is unsolv able b ecause creativ e systems can determine b ey ond the limits of an y predefined algorithm whether or not a statemen t in pr edicate calculus can b e prov ed. Ch urch’s thesis states tha t ev ery effectiv ely calculable function is general recursiv e [5, pp. 317-323]. T uring’s thesis, whic h is equiv alen t to Ch urch’s thesis, states that ev ery function that would b e naturally regarded as com- putable is computable under his definition, that is, by a T uring mac hine [5, pp. 376-38 1 ]. My work should not b e r ega rded as a refutation of Ch urc h’s or T uring’s thesis. Rat her, it sheds new light on these theses and on Hilb ert’s de- cision problem (Entsc heidungsproblem). In particular, it pro v es the existence of systems ”computing” non-computable functions if ”computing” means de- termining v alues of functions f or giv en arguments . But these systems ha ve no complete finite descriptions that can b e give n in adv ance. Rather, they can transcend any predefined forma l description. 5 Related W ork T uring [9, 10] discusses whether ”it is p ossible fo r machine ry to show intel- ligen t b ehaviour”. R eferring to G¨ odel’s theorem and other, in some resp ects similar, results due to Ch urc h, Kleene, R osser, and himself, T uring [9, p. 445] writes ”that there are limitations to the p o w ers of an y particular machine”. T uring [10, p. 4] states: The argument fro m G¨ odel’s and other theorems ... rests essen- tially on the conditio n tha t the mac hine mu st not mak e errors. But this is not a requiremen t for intelligenc e. He argues that a ”man prov ided with pa p er, p encil, and rubb er, and sub ject to strict discipline” can ”pro duce the effect of a computing mac hine”, but ”discipline is certainly not enough in itself t o pro duce in t elligence” [10, p. 9 and p. 21]. My simple incompleteness theorem confirms that the p ow ers of an y particular T uring mac hine are limited. Th e theorem implies that sys- tems capable of pro ving the theorem and in tera cting with their en vironmen t cannot b e mo deled b y any predefined T uring mac hine. Th us, ”o rdinary com- putational systems” suitably equipp ed to prov e the theorem a nd to in t era ct with their en vironmen t can in principle t r anscend the p ow ers of an y mac hine completely sp ecified in adv ance. My theorem implies that suc h systems are necessarily empirical and fallible in the sense that a complete predefined for- malization of their truth judgmen ts, for example, whether a giv en computer 8 program represen ts a computable function on the set of natural n umbers, is imp ossible. My theorem and principles ab o ut creativ e systems a re compatible with P ost’s view [6, p. 417]: ... logic m ust ... in its v ery op eratio n b e informal. Better still, w e write The Logical Pro cess is Essen tia lly Creative. W egner [11] argues that ”interaction is more p ow erful than algor ithms”. My simple incompleteness theorem can b e regarded as a mathematical pro of of his thesis. Moreo ver, the theorem a nd its pro of imply t he existence of systems ”computing” non-computable functions, that is, determining v alues of no n- computable f unctions for g iv en a r g umen ts. In this sense, they confirm the view of W egner and Goldin [12] ”that neither logic nor algorithms can completely mo del computing and h uman thought.” W egner [11] writes that the incompleteness of in teraction mac hines follows from the fact that dynamically generated input streams are mathematically mo deled by infinite sequence s ov er a finite alphab et, whic h are not en umer- able. The ”incompleteness”, rather indeterminacy of creativ e systems follo ws from their definition, in particular from the f a ct that no fo rmal theory can ”capture” all computable functions o n the set of natural num b ers. My incompleteness theorem implies t hat there is no general algorithm or formal system for the v erification of programs. This result is compatible with W egner’s view ”that prov ing correctness is not merely hard but imp ossible” b ecause ” op en, empi ri c al, falsifiable, or inter active systems ar e ne c ess a rily inc omplete ” [11, p. 10]. It is a lso compatible with G ¨ odel’s statemen t [3, p. 84] that ”one has b een able to define them [demonstrabilit y and definabilit y] only relativ e to a giv en languag e”, tha t is, there is no general definition of formal pro ofs but suc h definitions can only b e g iven in particular formal systems . References [1] Ammon, K. The automatic acquisition of pro of metho ds. Pro ceedings of the Seve n th National Conference on Artificial In telligence, St. P aul, U.S.A, August 1988. Morgan Kaufmann, San Mateo, Calif., USA. 9 [2] Ammon, K. An automatic pro of of G¨ odel’s incompleteness theorem. Artificial In telligence 6 1, 1993, pp. 291– 306. Elsevier Science Publishers (North-Holland), Amsterdam. [3] G¨ odel, K. Remarks Before the Princeton Bicen tennial Conference on Problems in Mathematics. In M. Davis ( Ed.), The Undecidable, Rav en Press, New Y ork, 1965. [4] G¨ odel, K., On undecidable prop ositions of formal mathematical systems - POSTSCRIPTUM . In M. Davis, The Undecidable, Rav en, Press, New Y ork, 1965. [5] Kleene, S. C. Introduction to Metamathematics. W olters-No ordhoff, Groningen, and North-Holland, Amsterdam, 1952. [6] P o st, E. Absolutely Unsolv able Problems and Relativ ely Undecidable Prop ositions - Accoun t of an An ticipation. In: M. Da vis (Ed.), The Undecidable, Ra v en Press, New Y ork, 1965 , pp. 338–433 [7] Prasse, M., and Rittgen, P . Wh y Ch urc h’s Thesis Still Holds. Some Notes on P eter W egner’s T racts on Interaction and Computability . The Computer Journal, V ol 41, No. 6, 1998. [8] T uring, A. M. On computable n um b ers, with an application to the En tsc heidungsproblem. Pr o c e e dings of the L ondon Mathematic al S o ci- ety , Ser. 2, V ol. 42, 1936- 37, pp. 230–265. Correction, ibid. , V ol. 4 3, 1937, pp. 544–546. [9] T uring, A. M. Computing Machine ry and In telligence. Mind 59, No. 236, 1950, pp. 433–46 0 . [10] T uring, A. M. In telligen t Mac hinery . In: B. Meltzer and D. Mic hie (Eds.), Machine In telligence 5, Edinburgh Univ ersit y Press, Edin burgh, 1969, pp. 3–23. [11] W egner, P . Wh y In teraction is More P ow erful than Algorithms, Com- m unications of the AC M 40 (5), 1997 . [12] W egner, P ., and Goldin, D. Computation Bey ond T uring Machin es, Comm unications of the ACM 46 (4), 2003. 10