Turing Test in Games

In my AI For Games class, we've been instructed to read Turing's original paper "Computing Machinery and Intelligence", in which he describes what is now widely called the "Turing Test" for judging the quality of an artificial intelligence, and describe how we might apply such a test to game AI. I thought this was a pretty interesting idea. Our responses to this prompt were not supposed to be particularly long or formal, so I thought I might post mine here and my readers may enjoy it.

Summary of the Paper

For those who don't know, Alan Turing was a brilliant mathematician and scientist active in the 1940s and 50s, and is widely considered the father of computer science and artificial intelligence. In his paper "Computing Machinery and Intelligence", Turing makes the case that not only will machines one day be able to learn and think, to have intelligence, but also that the best measure for this intelligence is to compare the machine behaviorally with a human. I personally am not sure I agree with this as a good measure of a machine's intelligence, but for now we will take it as a given rather than delve into the definition of intelligence.

In the paper, Turing describes a method for comparing a human and computer which he calls "The Imitation Game". A computer and human will interact with a third party, a human judge. The judge will be able to neither see nor hear the computer and human, but will communicate via typing. The judge asks questions of both the human and computer, and must decide which is the computer. The computer's goal, therefore, is to act as human as possible so as to convince the judge. The current time limit for this test is 5 minutes, though of course fooling a judge for longer would be ideal.

Though Turing expected it would be possible at the turn of the century, at this time, no one has been able to programme a computer such that it can fool a judge for the full 5 minutes.

My main issue with the Turing test is that rather than judging a computer's ability to learn things from experience and grow and change based on them, or have unique individual "thoughts", it instead judges a computer's mastery of a human language and ability to manipulate said language in a way similar to a human, in addition to its actual responses. I don't feel that the ability to use a language well is necessarily evidence of human-like intelligence, nor that it will always be present in an intelligent machine. However, I do understand the reason Turing used such an approach - the definition of intelligence is a tricky thing, and using a behavioural test is much simpler and cleaner way to judge AI.

Turing Test as Applied to Games

Games are an intriguing instance of machines interacting with humans, because AI within a game has a much more limited way to interact. For now, let us consider only the case where AI is controlling a "player" entity (either opposing or helpful), which might alternatively be controlled by a human player. This is the situation which makes the most sense to look at from a Turing test point of view.

In this situation, let us further stipulate that in-game chat systems are not being utilized, neither voice nor text based. Now the ways that players can interact are limited directly to the game mechanics. A Turing test in this situation would involve a third party judge player playing the game with two other players (either at once or in separate instances, as the game allows), one a computer AI and one human. The judge must then determine which was the computer and which was the human.

This seems fairly straightforward, and actually somewhat uninteresting. By limiting interaction so severely, is the test actually meaningful? Are we only judging how well the computer can play the game, and not how well it can respond and react like a human? The answer would greatly depend on how interactive the game is and how much on the fly strategy changing a human might be able to do in said game compared to a computer.

In my description of the game-based Turing test, note that I specified "playing the game with" rather than "playing the game against". Certainly in many cases the latter will apply, but I don't believe such a test need be exclusive to opponent AIs. Helper team members are a notable example.

When thinking about the ways in which players interact without using language, I immediately thought of the game Journey, released earlier this year for the Playstation 3. In Journey, the player moves across a desert toward a distant point. Occasionally players cross paths - but rather than allowing speech between players, the game limits interaction to visual movements of the player's avatars, ability to charge up each-other's items, and to using a series of different musical sounds. What these sounds mean is not defined - it is a language created each time two new players interact, by the two players themselves.

While there are not AI players in the game, it seems like an intriguing idea to create an AI who might be able to pose as a player in Journey and see if it could interact in such a constrained environment well enough to fool a judge in the Turing test. This puts both the player and AI on the same foot when it comes to "language" yet still allows a very organic interaction compared to typical game mechanics.

Other Reactions

In addition to his discussion of the test itself, the final section of Turing's paper was devoted to discussing ideas about machine learning. One analogy he made struck me: picture a mind (be it human, animal, or machine) as fuel for a nuclear reaction. A stimulus to the mind is like a neutron being fired into said fuel. If the fuel is sub-critical, the reaction will carry on for a brief time and die away - or, as another simile of Turing's poses it, "like a piano string struck by a hammer". This is similar to how a typical machine "mind" might act - a stimulus is supplied, it's programme runs in response, but eventually runs out of new instructions to execute (even if in a loop, it's the same instructions over and over). Turing posits that most of the time, human brains also act in this way - and animals always do. But if the fuel is in super-critical concentration, the reaction won't stop - it will grow and grow. The example Turing gives is, "An idea presented to such a mind that may give rise to a whole 'theory' consisting of secondary, tertiary and more remote ideas." A stimulus causes the mind to come up with a poem, a mathematical theorem, a new invention. In other words, this super-critical mind can be inspired rather than just effected by it's environment. A single idea becomes a springboard for more ideas. The question Turing then poses is, can we create a machine mind that can act in this "super-critical" fashion?

This really stuck with me as a much better description of my view of true machine intelligence. Not aping human behaviour but instead the simple yet extraordinary ability to take in a stimulus of some sort and produce something, not based directly from that stimulus, but some product of all the previous stimuli the machine experienced as well as the one just taken in. Whether this has direct application to games is questionable, but it could be assumed that an AI with this kind of ability would be a more engaging opponent or ally in any game.

Turing goes on to discuss how we might develop an intelligent machine - and talks about creating a "child" machine instead of a full-fledged intelligent "adult" machine. This child machine would have the ability to learn based on positive or negative responses, and to take in direct instructions in some form as well. While again this may not directly effect games, it describes a learning machine which would make a far more challenging opponent or ally than one using a pre-set list of strategies.

Conclusion

I feel that games suit themselves well as a possible application of the Turing test due to their constraint on methods of interaction between players. It would be more difficult or even impossible to use the Turing test to measure the AI of non-player controlled entities in games, however, as the entire approach is based on comparing human and non-human control - if no human control is possible, the computer can't possibly convince a judge of its supposed humanity. For these types of entities, other tests must be developed, or temporary human control must be allowed.

Journey to Scotland

Well everyone, I've finally arrived in Dundee and am all moved in to my room at Opal! You probably won't get to read this until tomorrow, though, because dork that I am I forgot to pack an ethernet cord and everything closes at 5pm here, so I won't be able to buy one until tomorrow. I should have been here at 3, but the trip didn't go exactly as planned...

It started off well enough. Andrew and my parents drove me to the airport at 5 am Friday morning, sending me off with many hugs and then waves as I went through security. The trip to Dallas, where my first layover was, was fairly uneventful. The seats were too small for someone of my noble girth, but I was used to that - it still put me in a foul mood by the time I got to Dallas, though. Luckily I had a 4 hour layover to improve my outlook - I spent a full hour in TGI Fridays eating tasty chicken and relaxing. I changed over my money in Dallas, since I wasn't sure I'd have time in my later stops - it was so exciting to get UK currency! Did you know their bills are all different sizes depending on the amount? It helps visually impaired sort their bills, what a great idea! They also don't use bills for anything under 5 pounds - there are 1 and 2 pound coins though.

Finally it was time for the trans-Atlantic flight from Dallas. The plane was HUGE - 9 seats to a row! Every seat had a little TV and you could choose from 10 or so movies as well as current tv shows and documentaries. Pretty slick! The seats were also much more comfortable. And best of all, I got some really nice seat partners - a small family from Bologna, Italy. I sat next to the mother, and we chatted in broken English off and on for the flight. Thank you for making my flight much better, random Italian family!

I was able to sleep a little on the plane, but was still feeling pretty tired when we pulled into London Heathrow airport. That's when things started going terribly wrong. First of all, our flight was a little late arriving. We then sat on the plan for about a half hour while we waited for them to manually set up the boarding ramp since the auto machinery was broken. I began to worry about my transfer, but only a little. I still had an hour, surely that would be enough.

As soon as we got off the plane, there was a woman waiting to give us some sort of express pass - seems they were worried we would not make the connecting flight. I speed walked at this point - and I speed walked a LONG way, just to get to the terminal transfer bus. Heathrow is HUGE! The transfer bus had a really long line, so I didn't get on the first one that came. It also ran very slowly compared to the little train in Dallas. Finally I ended up in Terminal 5, where my new flight was. The woman meeting us there didn't bother with an Express pass. "You won't make it." She said. I look at her incredulously - I still had half an hour, I was at the correct terminal, and I didn't have a chance to make it?

She ran with me anyway, to an area I would learn was the border entry area - another LONG distance, and one I did not run with ease. The woman behind the counter shook head at us. "You'll have to get on the next flight."

My plans started falling apart. The next flight? I'd miss my train! And my appointment to check in to my apartment! In their emails, my apartment complex Opal had seemed very adamant that you must check in during you allotted time. But I calmed myself. I would call them and figure it out, it would be alright. Still, my stress level began rising.

I was taken over to a customer service area. Funnily enough, my Italian family that I made friends with on the plane were also here, having missed their flight as well. A very, very kind woman at the service desk hooked my up with a flight two hours later, and allowed me to use her phone. I called and changed my train ticket, paying almost 4 times again as much as I had for the first ticket. It was still only 22 pounds, not the end of the world, but it irritated me. Then I called Opal - no answer. I left a message explaining the situation and asking if I could check in at 5 or 5:30, my new arrival time in Dundee, and if not, if I could check in the next day - I figured if worse came to worst I could stay at a hostel in Dundee that night.

With new boarding pass in hand, and thanking the customer service lady profusely, I went to the UK Border counter. Here I met my next obstacle - apparently the customs forms I had painstakingly filled out before leaving were not what I needed, and despite what the woman on the plane had told me, I needed to fill out the small blue form they had passed out there. I sighed and sat down - it wasn't much information so I didn't mind too much. I brought it to the counter, hoping I wouldn't have to have my luggage searched. I was surprised I had to do customs here, where I didn't have my checked luggage. The man didn't even ask about what I was carrying. I had worried so much about that, and he didn't even care - guess I could have brought gifts of Oregon jam and such after all!

Unfortunately, I wasn't free to go - the machine refused to recognize my fingerprints, reading not a match even after trying several times. I began to worry. The man took me to another booth with another machine, with the same results. They asked me to sit down while they contacted a superior.

At this point, I must admit, I began to cry. I was exhausted, having been up for 24 hours now with only fitful naps on the plane. I was sweaty from running and not having showered in over a day, and felt disgusting. Every muscle hurt, and I had huge bruises on my legs from the arm rests on the plane. My tow was smashed from my suitcases. I had missed my flight, I had no idea if I would have a place to stay, I couldn't access the internet or my phone and I didn't even know how to dial numbers in this country properly. I was NOT looking forward to wrestling my checked bags from the airport to the train and then to my apartment. I was all alone, and to top it all off, I felt in real danger of being deported. Fingerprints are how they know you aren't a felon in your home country - if they thought my original fingerprints were not run correctly, I could see real grounds for not admitting me into the UK. I also began to worry that even if I was let through, I would miss my new flight as well.

After a little while, the man at the booth called me up and said it had simply been a hiccup in the machine, gave me my passport and boarding pass back, and sent me on my way. He never talked to a superior about me from what I saw, and I didn't have to scan my fingers again. I am somewhat convinced he simply took pity on my crying there, and overrode the machine. But I'll never know.

Next, it was on to security. Yes, the UK makes you go through security again! And if you thought US security was a slow, annoying process, you've clearly never been through a UK airport. The US is streamlined and efficient by comparison. Every bag was stared at for what seemed like five minutes in the x-ray machine. They made me take my box of hard drives out and send it through again, though I didn't really mind - they let me keep it and that's what's important. The metal detectors we had to walk through were hyper sensitive. Everyone ahead of me set it off - and then, instead of using a handheld detector like they do in the US, they frisked the offenders. So with some trepidation I stepped through - but somehow, my bad luck must have taken a break - I didn't set it off. Collecting my bags at last, I headed out only to find that gates aren't assigned to flights until just before boarding. Tired, worried about missing the flights, I waited. I got a water, fumbling with my UK money embarrassingly at the counter. "Sorry, I'm not used to these coins yet..." "That's alright miss, that's a 2 pound coin there. Here's your change." Great, now I had more coins to be confused about. I thought about getting a meal at one of the restaurants, as I was getting pretty hungry, but I was too concerned about missing my flight to chance it. Finally my flight gate was assigned, and thankfully it was very close by. I went to the gate and waited for what seemed like another half hour until the plane arrived.

This plane was much smalled than the last, though bigger I think than the Dallas plane. The seats were very cushy and comfortable, though the seatbelts were too small for someone of my noble girth and I had to get an extender, to my embarassment. I was in a center seat, which I hate, but was just happy to be where I was supposed to be. Thankfully I got on early and had a place for my luggage without trouble - that is one thing I was lucky with the entire trip, as some people were forced to check their carry-on on both previous flights. I was paranoid about loosing my luggage, especially now that I'd changed flights, and didn't want to chance it.

Anyway, it took quite a long time for us to take off. I began worrying that I might be even later into Dundee - but I didn't have to worry about the train, at least, since I had a flexible ticket now that would let me take a later time if needed. Things went better than expected, though. No one showed up for the window seat next to me and I got to scoot over and look out over the UK as we flew. I didn't see much, however, as my exausted body pulled me into a deep sleep as soon as we were at cruising altitude. I woke as we were landing - the flight had taken less than an hour, making up for the long wait before takeoff.

In Edinburgh airport (much smaller than Heathrow), I headed to baggage claim. Unlike at PDX, they offered baggage carts without charge. I grabbed one and waited for my luggage to come around. To my relief (tempered by my knowledge that I would have to deal with the bags without a cart at least for a short time at the train station) both of my gigantic checked bags arrived. I wrestled them onto the cart with the help of a kind bystander, and headed to find out where the shuttle to the train station picked people up. But, staring at the huge pile of luggage on my cart, I decided I'd rather pay more for a taxi whose driver would help me load and unload the luggage, rather than try to bring it on a bus. I waited in line and eventually got a cab, the kind driver helping me with everything and chatting as we drove to Haymarket train station in downtown Edinburgh. He had an extremely thick accent, and I began to reallize how difficult it would be to understand some people in my new home. Still, I got through the conversation well enough. "Your going to have trouble with your bags at Haymarket," he mumbled from the front seat. "The road's closed, I'll drop you as close as I can but you'll have to walk a block to the station." I smiled and said I would figure it out, but inwardly groaned. My worst fear, reallized. One block isn't much at all, but with more bags than hands to carry them, it's quite a long way.

The cabby was true to his word and repositioned the cab once in order to get me right next to the walkway to the station. I thanked him profusely after he unloaded my bags, and wished him a good day. In turn he wished me luck and headed out.

Staring at the long walk to the station entrance, with no luggage carts in sight, I sighed and turned to my bags. I pulled up the handles on my largest case, picked up my carry-on roller bag, and set it on top of the case. It was precarious, but if I leaned the case forward to drag it, the smaller bag stayed in place. I hefted my shoulder bag, grabbed a handle in both hands, and scooted slowly backwards down the walkway. I'm certain I looked absolutely ridiculous, but I discovered that walking forwards was simply not an option. After several stops to rest, I finally arrived at the station entrance. Awkardly maneuvering inside, I checked in and got my tickets. There were still no luggage carts in sight, to my extreme dissappointment. One of the station employees helped me through the checkpoint, since my bags made it impossible to go through the turnstyle properly. I found my platform, staring at first increduslously at the stairs leading down, then happily discovering a lift nearby. Then it was time to wait again.

About a half hour later, the Dundee train arrived. I move toward it, but the doors closed just as I got to them. Worried, I looked around. "Are you boarding, miss?" A train crew member asked in a very thick accent. Not having understood him properly, and also distressed and not thinking clearly, I just stared and held up my ticket. "Yes, this is the Dundee train, but you can't take that luggage on here, come on." He led me to another door and helped me get my luggage on board the train. "You have entirely too much luggage," he scolded. Feeling rather raw, this nearly brought me to tears, but I forced myself to ignore it. I stowed as much of my luggage in a rack nearby as I could, though one bag had to stay in the entrance area of the train as it was just too big for the rack. And at last, we were off to Dundee.

The train ride should have been very enjoyable. The seats were comfortable and it afforded a gorgeous look at the Scottish countryside. I'll have to ride it again when I'm not fuming with irritation, exaustion, pain, and worry. As it was, I actually fell asleep occassionally on the two hour ride, to my surprise. The train was nearly empty. I woke up in time to stare out happily over the Tay bridge, seeing the larger and prettier driving bridge further downstream, and all of the city of Dundee laid out on the shore ahead. It was a truly welcome sight.

I gathered my luggage and clamored off the train once it stopped in Dundee. There wasn't a large hurry this time as this was the final stop. Resigning myself to the horror of dealing with my luggage again, I began setting my bags up as I had at Haymarket station. This is when everything started going right again. "May I take one of your bags for you, miss?" asked a man who had just gotten off the train. He wasn't an employee, just a kind looking middle-aged gentleman. I smiled up at him "That would be a huge help, that's very kind of you."

He helped me to the main station, where I got another pleasant surprise. After asking a station employee for help getting through the turnstile, he asked bewilderedly, "Why aren't you using a stroller?" I stared at him in confusion, never having heard that term describe anything other than a baby stroller. He pulled a luggage cart out from behind the desk to clarify. I smiled again, even larger this time. "That would be wonderful!" I sent my mysterious helper on his way with a thank you, and piled my luggage once again on a rack. This one I had some trouble with at first until I discovered you had to push down on the handle to release an automatic break. The station employee directed me to a public phone and the taxi pick up area, both upstairs.

I got to the lift and almost had another difficulty - at first I couldn't get the luggage rack into the lift - then, I couldn't get it out. Finally I managed to do so, accidentally ramming into a "call for help" button in the process. Oh well. I made it to the pay phone, inserted my money, and tried to dial the number for my apartment complex. However, I'm unfamiliar with UK numbers and I guess I didn't dial correctly. I tried several times in several different ways and couldn't get it to work. But I didn't despair - my mood was much improved by the help I'd gotten so far here in Dundee. I went up the ticket salesman and explained that I was having trouble dialing, being from out of country. I showed him the number. Rather than showing me how to dial it, he just used the station phone and dialed it for me, for free. I nearly wept in relief as someone answered on the other line. It was very quiet, and the station was loud, so I had to confirm that it was the right number. But it was! "Can I come check in now? My name is Sarah Herzog, I missed a flight connection in London and I'm very late, I was supposed to check in between 3 and 4." "Of course, come on over!"

I smiled broadly at the man behind the counter as I hung up. "Thanks, you saved me!" Cheerily I brought my luggage to one of the waiting taxis. The driver was a curmudgeonly old Scotsman, with a thick accent just like the last driver. We loaded in my luggage and I showed him my address. "It's just north of Abertay University," I explained. He nodded, and we were off. This one didn't try to engage me in conversation, for which I'm rather glad - I just wanted to get where I was going and collapse. We pulled in and I saw people running around everywhere, with Opal employees greeting new tenants. The cabby helped me unload at the curb, then headed out. A woman with an Opal badge walked up to me. "Oh, we'll get some strapping young lads to carry these bags up for you. Follow me, we'll get you all checked in." I sighed in relief and abandoned my bags with another Opal employee, and went to pick up my electronic entry key and mail key, as well as paperwork. I got into the lift to head up to my flat, discovering that not only did this building not use US numbering for its floors (obviously) it also didn't use typical UK number. -1 was the ground floor, then 0 was what the US would call the second floor. I was on floor 1, the third floor for US folks.

My luggage had already arrived outside my room. I happily pulled it inside, closed the door, and collapsed on the bed. I was in my new home!

Pictures and description of my apartment coming soon!