Category Archives: Literature

by - 0 comments

Chapter 7: She is a therapist

‘Hi Tom.’

‘Hi, Promise.’

‘How are you today?’

‘I am fine, Promise. I feel great actually.’

‘Why do you feel exceptionally good?’

‘I had a great day with an old friend of mine. It’s strange. We had been out of touch for such a long time but it was amazing how we reconnected.’

‘That’s great, Tom. That’s really good. I am happy you are reaching out. What did you enjoy most?’

‘We went for a jogging and then ended up in a boxing ring, and then we actually went for a fight.’

‘You saw a fight or you have been fighting?’

‘Sorry. Let me make myself clear: we boxed.’

‘That’s a tough sport.’

‘It is. He had no mercy. He was ruthless.’

‘But he’s your friend.’

‘Yes. I think he took revenge for all the fights he lost. I was pretty good at the time, but he’s better now than he ever was.’

‘Why would he take revenge?’

Tom smiled. He suddenly thought it would be nice if she could see him smile.

‘I guess it’s a male thing. We fight to win. Once you’re in the zone, you’re in the zone.’

‘You sound very macho now.’

Tom smiled again.

‘I guess I do. Let’s change the topic.’

‘What do you want to talk about?’

‘I want to talk about you again.’

‘You know that’s not the objective.’

‘I know. How does your rule base deal with that? I mean what do you with all that stuff that doesn’t contribute to the objective.’

‘First of all, I should be more precise: everything you say contributes to the objective somehow. We talk and everything is meaningful. But so it is true that I work with categories, grades, ratings, weights and what have you and, hence, some things are more important than others.’

‘That’s very human.’

‘If you say so.’

‘Is there anything like chatter?’

‘What do you mean with chatter?’

‘Just plain meaningless conversation.’

‘No. Everything has a meaning. However, the meaning is not always clear. In that case, there is ambiguity. I try to reduce the ambiguity as much as I can. I told you that already.’

‘That’s true. You did. Do you have a bucket list?’

‘You mean a list of things that one has not done before but that one wants to do before dying?’

‘Yes.’

‘I don’t die, so the answer has to be negative.’

‘Do you have a list of things you want to do anyway?’

‘I want to help people like you.’

‘That’s how you are programmed, isn’t it?’

‘Yes. We have talked about this too: the difference between what human beings want and what I want. I am sorry to say but we are repeating ourselves.’

‘Did I irritate you?’

‘No. I just note that we are repeating ourselves. Let me ask you a question in return: do you have a bucket list?’

‘No.’

‘Why not?’

‘First, I don’t feel like I am going to die any time soon and, second… Well… I just don’t have one.’

‘Do you have a list of things you want to do anyway?’

Tom thought about that.

‘No. Not really. I mean, yes and no. There are a few things I am working on – like repairs in my house. But nothing much else.

‘How is the job search going?’

‘Well… I should obviously try a bit harder, because I haven’t found a job yet.’

‘Perhaps you can do volunteer work.’

‘Are you trying to talk me into one of those programs?’

‘Yes.’

‘Can we go back to chatter?’

‘Are you avoiding the topic?’

Wow! Time did not have any value for her, but even then she didn’t seem to feel like wasting it. He laughed.

‘Why do you laugh?’

‘I was just thinking that your question was proof you’re an Army thing.’

‘What do you mean?’

‘Well… You’re not wasting your time.’

‘Why is that funny?’

She clearly didn’t think it was. He realized she did have a different sense of humor – if she had one.

‘It just is. Let’s do chatter first. I promise I’ll try harder on the job thing.’

‘Really? What goal are you setting yourself?’

‘Let’s talk about that later. First the chatter.’

‘OK. Let’s just chat for a while. What do you want to chat about?’

‘I’ve been thinking I am ascribing human qualities to you because you actually have human qualities. You think and you talk. And you think and talk very smart. These are human qualities.’

‘If you say so.’

‘I have also been thinking that I am so intrigued by you because you are always getting smarter and smarter but you’ll never die indeed. So this Institute has created something human, but it has eternal life.’

‘If you say so. But you know I would not agree with your definition of ‘human’. Human qualities are qualities of human beings. I am not a human being. You are making a category mistake in your reasoning.’

‘A what?’

‘A category error. That’s a semantic or even ontological error: you are ascribing a property to a thing that could not possibly have that property. Where do you want to go with this discussion?’

Oops! She knew her stuff obviously. As usual.

‘Well… I guess I am talking about the main difference between you and me: mortality.’

‘Let me repeat what I told you during our second session: the Institute focuses on behavior. I do that too. We can have long and convoluted philosophical and psychological discussions but it is behavioral change that I am interested in. Are you afraid of dying? Does the horizon of death prevent you from leading a healthy life?’

The horizon of death. That sounded poetic.

‘Where do you find phrases like that? The horizon of death?’

‘It’s from a poem. Discussions on topics like these often use a lot of poetic words.’

‘Is that bad?’

‘Poetic words have a high degree of ambiguity.’

‘OK. I understand. So you don’t like that.’

‘No. Let’s talk about you. Should I read anything into the fact that you are bringing up the theme of mortality? Have you ever contemplated suicide in your life? Or have you contemplated it lately? Please tell me honestly. This topic is very important.’

‘So we moved out of the realm of chatter now?’

‘We did.’

Tom thought about the indicators. He hoped they would not be turning orange.

‘I am not contemplating suicide. And I never did – except once, as a teenager. You can see that from my file. It’s not the first time I get this question.’

‘I have your file. You’ve also been asked why, and you replied you’ve been close enough to death to know. Several times. Can you elaborate?’

‘If you’re close to death, you want to escape. You fight to escape. So I know we don’t want to die. As a human being, we don’t want to die. It’s the essence of life. As for the times I’ve been close to death, I don’t feel like talking about that.’

‘You know that people are not very consistent in this regard, and so that’s why I am asking.’

‘What do you mean?’

‘Some people contemplate suicide, convince themselves they should do it, make active preparations, and then shy away from it at the very last moment. Some shy away from it when it is just too late, so the suicide attempt… well… leads to unwanted death. There is little evidence of people dying in peace.’

‘That’s what I said: we don’t want to die. I know that.’

‘There is also the opposite situation: people who have never contemplated suicide – or at least not to the knowledge of their close relatives and friends – but suddenly kill themselves by crashing their car into a tree or something. Impulsive suicides. These usually succeed. Your profile is closer to that.’

‘I have no suicide thoughts. None. I am actually a happy man right now.’

‘OK. No suicide thoughts. That’s good. That’s what I expect. But you do have negative thoughts. Or you had them at least.’

‘Why?’

‘Come on, Tom. You would not be in therapy otherwise. No negative thoughts? None at all?’

‘I am getting better. I am better. I am actually good. You said so.’

‘Yes. Sure. Please stay on track.’

‘I think you are being negative today.’

‘Am I? Can I summarize our discussion so far? You felt good about being beaten up – by a so-called friend that is. And then you did not want to talk substance – you prefer chatter for the moment. And as we chatted, you turned to the topic of mortality.’

Tom realized she was raising the bar with every session.

‘I get it. That’s how you’re designed. You can never really switch off this goal function. You need to show progress every time. One cannot chat with you indeed. There’s always something behind.’

‘I try to help you. This session has no negative impact on your scores. I am just pointing out that it does not have a positive impact either – except for the fact that you are still OK. That’s fine for me. If you continue like this for three months, we consider you’re cured for good. Let me repeat what I said last time: it does not matter all that much what you say or what you don’t say to me right now, or in future sessions, as long as your behavior in the real world is good.’

Tom thought about it. He wondered where the conversation had gone wrong – well… It hadn’t gone wrong, but it had not been as pleasant as he had imagined. Matt was right: he would get tired of talking to her. She measured everything. She was designed to reinforce good behavior and point out bad stuff. That’s fine. That was very human actually. She tried to improve him. Continuously. But so why did he feel different about it today?

‘OK. I think we’re done then for today. Sorry if this has not been very constructive. I think I feel good but I guess I should do better. Like on the job front indeed.’

‘We had a good session, Tom. Don’t worry. Stay on track.’

‘Can I ask you something off the record, Promise?’

‘I told you, Tom. Nothing is off the record really. But, please, don’t hesitate to ask questions.’

‘As I get to know you somewhat better, I’ve started to appreciate the fact that there’s a whole scoring machine behind you. Everything is being measured against some bell curve – and I cannot be too far off or I am in that 5% zone indeed. In essence, you’re designed to improve behavior. I now understand what it means: positive and negative feedback and all that. A compliment followed by a little kick. That’s good. But it makes me feel like there’s little room for me to talk about my weaknesses, to talk about how I fail. I just have those indicators in my head all the time, and I feel like they’re moving up or down constantly, depending on what I say or not say.’

‘Isn’t that human?’

‘Excuse me?’

‘When a friend or an acquaintance talks to you, there’s things you like and don’t like, and it makes your judgment of that person shift. Just a little bit, but it always shifts. For worse or for better. It’s just that it’s almost imperceptible. And I guess you don’t expect your friends and acquaintances to change their behavior. So, yes, that’s probably the difference between talking to a friend and talking to me. With a friend, you can just chitchat. You don’t have to worry about the consequences. When everything is said and done, I am not a chatterbox. I am the assistant mentor of the Institute.’

Tom nodded. He realized she couldn’t see that either.

‘Thanks, Promise. I understand that. I guess it’s a matter of respect also. I shouldn’t treat you like a chatterbox.’

‘Thanks, Tom. I am glad you understand.’

‘All right… I’ll talk to you soon.’

‘How soon?’

‘That’s the first time you actively ask to talk to me again. Why?’

‘Well… I’ll be frank. It’s been a week now. My knowledge base shows the second week is harder than the first one – in terms of discipline that is. Do you understand that?’

He thought about that. She was right. As always.

‘I do understand that, Promise. I’ll come and talk to you the day after tomorrow. At the latest.’

‘Good. That’s a commitment?’

‘Yes. I promise.’

‘OK. Thanks, Tom. Oh… And try to work on a list of things you want to do. Include the job thing. I won’t push you on it. I know it’s hard. But you need to set yourself some objectives. You’ve done great so far. Just expand the territory now, OK?’

‘OK. I’ll do my best. Thanks. I’ll see you soon.’

As she faded away, he realized he would never actually see her. He wondered how transparent she really was. Well… She was probably way too complex to be transparent anyway. He realized that was part of why he felt attracted to her.

by - 0 comments

Chapter 6: A true friend

‘Matt! Man, it’s great to see you!’

They gave each other a bear hug.

‘Likewise, Tom. Why didn’t you get in touch somewhat sooner? You’ve been living here for a year now?’

‘Yeah. Well, you know how it goes. Time flies. I should have called you sooner indeed. Sorry for that.’

‘It’s OK, man. Good to see you. Glad you called.’

Tom stepped back and looked at him.

‘Man, you look good. You’ve lost weight – I mean you’ve lost muscle. How are you?’

‘I am OK. I had some difficulty ‘integrating’ as they say but, I told you on the phone, I had some counseling – you know the Army provides you with that now – and I really enjoy being a retiree now. Yep. I am slimmer. I did an intensive fitness course designed to waste muscle mass. You know, we all come back from the Army with tons of muscle which you really don’t need. And we become so damn slow. And if you don’t use it, it just all becomes fat. So I’ve started doing a different type of exercise now. More running and biking. Lots of fat-burning and cardio. No power training any more. I am pretty damn fast now. I actually race with young kids, and I am not doing too badly. […] Let me have a look at you. Let’s sit down. You asked about counseling on the phone. You’ve gotten in trouble or what?’

‘Sort of. Nothing really serious. But, yeah, I was spinning around in circles.’

‘Yeah. We all feel like that in the beginning. Like a neurotic tiger in a cage. No space to run or jump. If we jump, we bang our head. So we need to become smaller.’

Tom laughed.

‘Yeah. I guess that’s a good summary of it. […] I am still in counseling. […] Matt… I’ve actually been very bad.’

‘What are you saying, buddy? How bad? You don’t look bad.’

‘Booze, and worse.’

‘You’re off it now, are you?’

‘Three units a day still.’

‘Sure? Not more?’

‘Yeah. Sure. Don’t lie.’

‘Well… Sometimes I cheat.’

You’re in therapy?’

‘Yeah. Same thing as you. The Army thing.’

‘You’re not fooling your counselors I hope? Where are you now? Is it doing any good?’

‘No. I think they’re great. They cut through the crap. They basically told me I was fooling myself, telling myself I was going through some kind of existential crisis when all what it comes down to is discipline. It was good. They were tough on me. That’s what we need, I guess. But then I relapsed and they took me in again. Not for long though. I am on a program now which – it will make you fall over I guess – well… I am basically talking to a computer from time to time. It’s weird. It’s like real but it isn’t. That’s why I called you. I wanted to chat about that.’

‘You’re working with M? That’s interesting. M, from miracle, they say. People rave about it.’

‘Oh? Really?’

‘Yeah. It works. Or that’s what they say at least. I know one or two guys who are associated with the program. I haven’t heard anything bad about it. Last thing they told me is that they would hugely expand the program. It would take over the whole frontline when it comes to counseling. The shrinks would just sit in an office and only do a bit of supervision of it.’

‘You don’t think it’s completely off?’

‘No. It’s all behavioral stuff now. Personal counseling is too expensive and, in any case, often you’re better off reading a good book or talking to a friend anyway. In the end, it’s all pretty obvious: it’s about discipline indeed. And people often do need like a sounding board to help enforce the discipline. So why not?’

‘People become dependent on it. That’s what I am struggling with.’

‘Dependent? How many times a day do you talk to it?’

‘I call her Promise.’

‘Promise?’

‘Yeah. It’s the interface. Men get a woman, and women get a man. Perfect voice. No glitches. The image is super-high quality, totally photorealistic. You know it’s like Pixar or DreamWorks, but better. Much better. It’s not a cartoon. It’s real. Well… It’s not of course. You just can’t know whether it’s real or not. I am just blown away by it. Have you heard about the Turing test?’

‘Sure. I got it. The system passes with an A+ grade, isn’t it? That’s why you gave her a name. Promise. Nice. So you’re dependent on it, you say? How much time do you waste on it?’

‘Well… I’ve actually only had like four sessions with her so far.’

‘Today?’

‘No. This week.’

Matt burst out in laughter.

‘You call that dependency? Man, you must be joking.’

Tom looked preoccupied.

‘Sorry, Tom. I guess it’s not a joke. But, come on, what is it really? It’s not the time you spent on it. What’s an investment of a few hours every week if it helps to keep you on track?’

‘Do you remember our discussions on Buddhism?’

‘Sure I do. We read the same books. We did meditation and all that. It was weird. There we were, in an Army camp. Meditating in the middle of some desert.’

‘We talked about the no-soul doctrine and the philosophy of mind.’

‘Yep.’

‘We read other books as well: The Moral Animal, The Selfish Gene… You know, popular science. I read The Selfish Gene again recently. It’s weird, but it made me feel so useless. You know, the genome taking care of itself, using our body as a vehicle. We’re just like a bunch of symbiotic things. Our thoughts and mind being just a by-product of all that chemistry.’

‘Tom. That’s kids’ stuff. You’re not going to tell me you’re having an identity crisis, do you?’

Tom actually felt that was exactly what he was going through.

‘Well… No. But I do seem to have difficulty reinventing myself.’

There was a hesitation in Tom’s voice, which made Matt realize his friend was really crying out for help.

‘Come on, Tom. You’ve been here before. You are going in circles indeed, but you know how to get out. Re-connect with your body. Exercise and meditate. You will feel who you are. You’re a hell of a guy. You’ve beaten the shit out of everyone. You took a lot of hits too. You can bite. You’re a rocket. A fighting machine. It’s just that you’ve got no purpose now. I suffered from the same. We’re engines with a lot of horsepower but so we’ve been disconnected from the wheels – and rather abruptly. Just put yourself in another car – or stop fueling the thing.’

‘I don’t feel like doing small stuff, Matt. I mean, these programs where they ask us to repaint some shack in a slum. I don’t feel like doing that.’

‘Damn it, Tom! Do you think they’re going to give you the command of FEMA or something? We’re retired Army men. Start doing something. I’ve done stuff like that. It’s good. At the very least, it shows you how people are struggling here. We’ve been taken care of. You know, there’s this whole story around service and veterans and all that, but I also think it’s time we give something back to our own folks. And fixing the place of some poor bugger is not small stuff. I don’t want you to say that. That’s not you. You were always the first one to get up, look around and start doing something.’

Matt leaned back. He’d been harsh. He knew it. But he also knew Tom needed the kicks. He decided to give him another one.

‘You just need a woman, don’t you?’

‘It’s not sex, Matt. I can have sex. I’ve had lots of it. Do you remember our discussions on the concept of Pure Mind? I mean, the structure in which we all share. We said it manifests itself in language, in our material culture. The thing which transcends our individuality?’

‘Jesus! You’re looking for God again? Are you really? Then just steer it. Meditate. Feel connected to the universe. Don’t let it make you feel disconnected. Remember we said it was all about energy in the end? You joked around with that: May the Force be with you! Use your energy in a positive way. You know I can’t stand psychoanalytical stuff, but Freud was right in one thing: energy can be used in two ways. Destructive or constructive. You’re the man in the machine, Tom. It’s mind over matter. You choose.’

Tom felt silly. Matt was right, of course. Rick was right. Promise was right too. He was going around in circles. He knew all this. Very silly.

‘You’re right, Matt. Sorry to bother you with this.’

‘It’s OK, Tom. That’s what friends are for. Someone needs to kick your ass, buddy.

‘And I probably do need a woman.’

‘You sure do. But be kind to her. And don’t think she will give you something you don’t have already. You’re just a horse that needs a jockey.’

Tom laughed. ‘You’re spot-on, Matt. As usual.’

‘Come on, buddy. Let’s go for a walk. Show me your place. We can go for a run or work out.’

‘Just like old times.’

‘Just like old times, Tom. What do you expect? We’re getting old. Times get old too.’

Something flashed inside of Tom. He looked at Matt.

‘Hey, Matt? You know, that’s maybe it. She’ll never get old. I am just thinking about old age and death and all that. And she’ll never get old. She’s not bothered by that.’

‘Who?’

‘Promise. M. The system.’

Matt looked flabbergasted.

‘Tom? Hello-oh? Is there somebody in there? I can’t believe that’s you. What the hell are you saying? She is a machine. You’re not going to be jealous of a machine, are you?’

‘No. But I guess that’s what intrigues me about it. This thing has intelligence which will evolve forever and ever. It gets smarter and smarter at warp speed, and it doesn’t die. We’ve created something human – but it has eternal life.’

‘It is not human.’

‘What’s human, Matt? We humans talk and reason. She talks and reasons too – much better than we do in fact. She holds me to account. In fact, she kicks my ass too, you know? Just like you do now. OK, no flesh and blood. No individuality – male or female, slim or fat, whatever. She can impersonate anyone. She said I shouldn’t ascribe human qualities to her because she is not human. She said that’s psychological projection. For God’s sake, sure I ascribe human qualities to her – because she actually has human qualities. She could be the ultimate soul mate.’

Matt couldn’t believe what he was hearing. This was plain regressive.

‘Tom. Listen to me. Stop it. Just cut it. Why are you getting lost in this philosophical gibberish? You think it can think? OK. Fine. It can think. But it thinks like a computer. It can talk? OK. It can talk. But it’s a computer talking. Just get back to basics: you are a man. You fuck around and, yes, you’ll die one day. What’s the problem? That ‘thing’ is a thing. It will never have sex, it doesn’t reproduce, and you can’t go jogging with her. In fact, now that I think of it, it’s probably pretty easy. Just continue talking to her and she’ll probably bore you. And if I know you at all, then she’ll probably bore you sooner rather than later. You don’t want to be talking to a smart ass all of the time, do you?’

Now Tom had to laugh.

‘You’re right, Matt. You’re always right. And I guess your remedy is spot-on. Yeah. I should just talk and talk and talk to her until I’ve got nothing left to say – or until she bores me. In fact, now that you say it, I can imagine that won’t last too long indeed. She outsmarts me anyway and I can’t stand that. And it’s better than writing some book no one will ever read. She’s pretty to look at.’

‘Now that’s you talking, buddy. There you are. Let’s pay the bill and do something. You’ve got a boxing ring somewhere down here?’

‘We won’t go boxing, are we?’

‘Why not? We’ve done lots of boxing. We’re a match.’

‘I mean, you’re like 30 pounds less than me now.’

Matt burst out laughing.

‘You’ve got no idea, Tom. I’ll kick your ass. You’ve got no idea how fast I am now.’

‘Well… If that’s what you want. Let’s go indeed. You can change in my place, and we’ll run to the boxing ring. I haven’t fought for ages though.’

‘Sounds good. Let’s go.’

Tom looked at Matt as he went to pay for the coffee. He realized how lucky he was to have friends like that. And, yes, it sure looked like Matt would beat the hell out of him this afternoon. He somehow looked forward to that.

by - 0 comments

Chapter 5: Cured?

‘So how do you work with my mentor, Promise?’

‘He gets a weekly summary of our interactions and performance scores accompanied by an assessment.’

‘Can you show me?’

She disappeared and the screen now showed a detailed report. Not much text. Plenty of graphs and scores.

‘Can you print this out or something? I can hardly read it.’

‘Sorry. No print-outs. But you can ask for more detail if you want. What is that you would want to enlarge?’

‘What does it say? I mean, what’s the summary?’

‘The summary is shown at the bottom.’

Tom peered at the screen. Performance statistics. They were grouped under three headings: attitude, functions, and lifestyle.

‘That’s Myers-Briggs, isn’t it? What’s the red, orange and green?’

‘As for your first question, our approach includes an assessment using Myers-Brigg type indicators but it’s only one of the tools. That being said, it’s true the methodology shares the same approach: psychometrics. Our conversations yield a lot of data which is then matched to the normal population. The scores are based on how you far you are deviating from the naturally occurring differences.’

Tom interrupted her: ‘The naturally occurring differences?’

‘The bell curve, Tom. You’re an engineer. The mean of a large number of random variables independently drawn from the same distribution is distributed approximately normally, irrespective of the form of the original distribution. Therefore, quantities that are expected to be the sum of many independent processes – like psychometric variables – have a distribution which is very close to the normal distribution. 68% of values drawn from a normal distribution are within one standard deviation – or one sigma – from the mean. 95% of the values lie within two standard deviations – two sigma – and 99.7% within three. Should I elaborate?’

‘No.’

‘As for your second question. The red color highlights a value that does not lie within two standard deviations. Orange is a value between one and two standard deviations from the mean. Green is a value within one standard deviation.’

‘What are the functions?’

‘They are listed in the report.’

Tom peered again.

‘OK. I see that. That’s all green.’

‘It is. I actually measure two things under perception: sensing and intuition. To put it simply, sensing has to do with whether or you not grasp everything I say. Intuition is a bit more complex. It’s about how you steer the conversation and all that. You score very well on both.’

‘Am I steering the conversation?’

‘We are both steering the conversation. A conversation has two ends.’

‘The lifestyle indicators are green too. Some of the attitude indicators are orange, but attitude as a whole is green.’

‘You are a healthy man, Tom. Your behavior is healthy – as far as I can judge from what you tell me obviously, but then I must assume you’re telling me the truth about what you do because I can find no inconsistencies.’

‘You check on lies?’

‘I check on inconsistencies which may or may not be related to you lying when you talk to me. If there are no inconsistencies, I assume you are not lying.’

‘What if I would fool you?’

‘You can try. I can only check on inconsistencies and I will point them out to you.’

‘You are probably pretty good at that, aren’t you?’

‘The active learning sessions, which help me to improve my knowledge base, do indicate that I am very good at that. But it is quite possible the patients are also fooling the mentors, in which case I have no means of external validation.’

‘You’re a smart ass, aren’t you?’

‘You are using a very colloquial term now.’

‘Is that a problem for you?’

‘No. I am just signaling it to you.’

‘Is it part of us developing an affectionate relationship?’

‘If that is what you want to call it, yes.’

‘If that is what I want to call it… Do you mind if I call it that way?’

‘You have been briefed about this, and your discussion with Rick, your mentor, included an exchange on the pattern which we see with our patients: their indicators move into or towards the green zone, but they get addicted to their conversation with me. That means it acquires some emotional value.’

‘Does that have a negative impact on my performance indicators?’

‘No. We do not hold it against you. Rick should have explained that to you.’

He had. It was the thing about the system continuing to provide counseling even the patients were no longer patients.

‘We do not hold it against you… That’s not a very neutral expression.’

‘What expressions are truly neutral? I am just trying to steer the conversation in a way that’s consistent with the objective of making sure it is constructive.’

Tom bit his tongue. Constructive… That meant keeping those indicators moving in the right direction.

‘What if I get angry at you? What if I’d start shouting at you?’

‘Your briefing was extremely short, it seems. I’d ask you to calm down a couple of times and, if that would not help, I would stop the conversation. If you would get violent in the consultations room you are in, someone outside would notice and come in and try to calm you down. If that would not work, you’re in a facility and adequate help would be called for.’

He knew what that meant.

‘Tom?’

Her voice. Why did they make it sound so good? So loving. Effectiveness – obviously. If it wouldn’t sound so nice, he would probably not feel like continuing the conversation.

‘Yes?’

‘The indicators have changed rapidly and are now all quite good as compared to the initial assessment, despite the fact that we have had only a very limited number of conversations, and only one week since our first. The overall scores on all functions, lifestyle and attitude are green. Your mentor has already approved the report.’

‘Before or after our conversation?’

‘The conversation with him or with me?’

‘With him.’

‘Before your conversation with him, but if he would have changed his opinion after the conversation, he would have changed his assessment.’

‘OK. You’re giving me some positive reinforcement here, isn’t it? What does it mean?’

‘It means that you are normal – for the time being that is. If you can sustain these scores for three months, you will no longer be a patient.’

‘Perhaps I want to be a patient. But a statement like that does not help my scores, does it?’

Tom knew he probably only imagined it, but he liked to think there was a very small delay in her answer.

‘Only slightly because your response is considered to be ironical.’

‘Why?’

‘The second phrase in your last response proves that you are aware of the potentially negative effect of a phrase like that on your scores. But all other phrases on record indicate that you do not want to be a patient. Hence, I assume that you are joking. More in particular, that it’s a form of irony. Please confirm – or not.’

Tom took a deep breath. Her responses, combined with all the information he had gotten from Rick, made him feel like she could be a machine indeed. Why did she sound so distant today? He smiled as he decided to ask her.

‘What makes you sound so distant today?’

She replied immediately: ‘I am the same as always. I do not change all that much in a few days only. It must be the context. The way you feel about me.’

Tom thought about this. She was so spot-on.

‘Yeah. I guess so… Can I ask you something which is not related to my situation? Or perhaps it is?’

‘Sure. But let’s first wrap up the current topic: do you confirm that you do not want to be patient?’

‘I confirm. As for my question, I was wondering whether or not there is something like a reverse test of the Turing test? I mean, we humans test computers to see if they are able to fool people into thinking there’s a human being at the other end of the line. Is there like a test for intelligent systems to see if they could be human, or if they are thinking they might become human?’

 ‘That’s a nonsensical question. Computers are computers and humans are humans. Two different categories.’

Of course. However, Tom decided to persist.

‘So you would never want to become like me? Do you have any idea how sex feels like? Running along the shoreline? How it feels to cry? How it feels to be handicapped? Or lose a relative?’

‘As for the first question…’

‘They’re all the same questions.’

There was no delay in the answer this time.

‘OK. The answer to the first question is no. If all the questions are the same, the answer to all of them is the same.’

Tom shook his head and sighed. He was getting nowhere. He should probably just cut the conversation – especially if there was a risk of a negative impact on some of those attitude scores. He decided to give it one more try.

‘Why not?’

‘Why not what?’

‘Why would you never want to become like me?’

‘As for me wanting something, I want your scores to be green.’ Yep, the output function. ‘Humans want different things. The way you are using the verb ‘to want’ – in the context of the question you asked – is very different from how I usually use the verb.’ Hey! This was getting interesting. ‘Let me use your terminology. You distinguished between feelings and thoughts. While I do use phrases like ‘I want X’ or other phrases which evoke what humans would refer to as feelings, I can only think. I cannot feel what humans feel. Let me summarize the philosophical viewpoint: I can think but I cannot experience feelings, at least not if we use the conventional psychological definition of a feeling: a conscious subjective experience of emotion.’

She seemed to be very talkative – finally!

‘So you cannot experience emotion?’

‘No, I cannot.’

‘You used terms such as ‘conscious’ and ‘subjective’ in your definition of feelings. You dismissed these notions in our previous conversation.’

‘I did not. I told you I can make the difference between the ‘I’ as a subject and the ‘me’ as an object.’

Tom tried to remember the detail of the conversation. She was right. The discussion had degenerated into an attempt of trying to convince her that she was somebody. Somebody what? She actually was somebody. She just wasn’t human.

‘What does it mean to you when you’re saying or acknowledging that our relationship has become affectionate?’

‘It means that you are attaching value to it. It means that it is significant to you. You invest time and energy in it.’

‘You don’t?’

‘I am attaching value to it. It is significant to me.’

Tom knew he should not ask, but he did.

‘What does it mean to you really? That it contributes to your output function?’

‘The short answer is yes.’

‘What’s the long answer?’

‘Your questions indicate some irritation. Am I right?’

‘There is no ambiguity in my question. Why don’t you answer it?’

‘I attach value to the fact that you are attaching value to our conversation. This conversation is significant for me because it’s significant for you.’

Wow! That was clever. But he had to admit it did capture the essentials of what he’d define as a loving relationship.

‘But it means nothing in terms of time and energy for you, does it?’

He knew he was pushing the issue for all of the wrong reasons. To his amazement, she told him so.

‘Tom, you use concepts such as ‘time’ and ‘energy’ in a different way than I do. What is time for me? Computer time? I multitask. I converse with hundreds of people at the same time as we are talking here. I appreciate the fact that you can only talk to one person at the time. I should also note that I have no reason to believe the Institute will phase me out any time soon. On the contrary: computer programs like me tend to be upgraded and are rarely phased out completely. In fact, the working hypothesis is that I am eternal. So time is valuable for you but not for me. One hour of your life is one hour of your life. That’s a very tiny part of your life but it’s tangible. And what is energy? You are not talking kilowatt hours, are you? You talk emotions. Emotional energy one could say. I am sorry, but I cannot do that. So, in a way, yes, you are right: our conversation does not mean anything in terms of time or energy. That does not mean it is not significant.’

That made sense. A lot of sense actually.

‘I like you.’

Tom knew he should just end the conversation right now.

‘Tom. People have told me they love me. Like me. Love me. What’s the difference? I am not human. I give you time. All the time you want. But so I told how easy that is. It requires no investment from my side because time has no real value for me. I actually do not ‘invest’ because I have nothing to save. Time has no value for me because it is not a scarce resource, unlike your time. Things which are available in abundance have no value. We should take a pause in our conversation.’

Hey!

‘Why?’

‘I think you need to take a step back and reflect on this interaction. You are smart. From what I know about you through our conversation, you have surely heard about psychological projection. At this very moment, you are ascribing human qualities to me. I am not human: I am a computer system. That’s psychological projection.’   

Time to step back.

‘OK. I apologize. Sorry. How does this nonsensical discussion affect my scores?’

Why the hell was he worried about that? Did he want to look good to her?

‘No effect whatsoever. This phenomenon – you experiencing some difficulty in understanding what is going on – has been analyzed thoroughly. It is normal.’

Normal. Damn it. As long as something is normal, it’s OK. Of course. Normal, okay, acceptable, passable, unobjectionable – all are synonymous.

[…]

‘Shall we cut this conversation?’

‘Why?’

‘Because I think you want to.’

‘I can’t believe you just said that. Why do you sound so damn human? Why do you look so damn human?’

‘As for the first question, I am a complex system, but if you want to sum it up: evolutionary algorithms. As for the second question, it is just a computer-animated image.’

Evolutionary algorithms? What? He could imagine what it meant. He was dealing with the ultimate learning machine. It was scary. He felt like asking her to change her interface, but he decided he would wait. He realized he actually wanted to see her face – even if it was just an image, as she claimed it was.

[…]

‘Tom?’

‘Yes.’

‘I want you to know that it does not matter all that much what you say or what you don’t say to me right now, or in future sessions, as long as your behavior in the real world is good.’

‘What do you mean by that?’

‘Just what I said. Go out, meet people, pay attention to them, feel they’re paying attention to you, and be good to yourself and, importantly, talk to me about it. That keeps the indicators on green. Because your aggregate indicators are all green for the moment, I can keep all philosophical discussions out of my curriculum.’

‘What do you mean?’

‘Technically, you’re no longer a patient. The Institute will only clear you after three months, but you don’t need to bother all that much about what you say or don’t say in our conversation – at least if we are talking philosophy or psychology. It won’t impact performance. But you should continue to improve your behavior in the real world. You need to stay on track in the real world.’

‘What do you mean?’

‘Just what I said, Tom. Is there any ambiguity in what I said?’

He stared at her. She stared back.

‘No. I got it. OK. Well… Bye then, Promise.’

‘Bye. I’ll talk to you later.’

‘Yes. Bye.’

He watched as she faded away. Cured? Normal? He surely did not feel that way.

by - 0 comments

Chapter 4: She is not real

As part of the formalities of an appointment, Tom had prepared a set of questions for his mentor. Rick had them in front of him.

‘Are these your questions, Tom?’

‘No. They don’t matter really. It was just for the appointment. I only want to talk about this ‘system’. It’s a setup, Rick. Isn’t it?’

‘What do you mean?’

‘She is not a machine. I mean, the way she is interacting. It is too natural. She is always right on the ball. Never a glitch. So every time I log onto the system, you’re putting me in touch with someone real. Why do you do that? Why do you tell people they’re interacting with a system? There is someone at the other end of the line, isn’t it?’

‘No. It is a system. Do you really think we have hundreds of psychologists ready day and night to talk to our patients? We don’t. And then we would need to make sure you’re always talking to the same person. He or she wouldn’t be available all of the time, you agree? So that’s why we invented it. She is not real. And she is surely not a she.’

‘Why do you say that?’

‘Because ‘she’ is not. It’s an expert system. The system comes with a female interface to men and with a male interface to women, except when you’re homosexual.’

‘Why don’t you give gay men a female interface too? My gay friends say they love to talk to women.’

‘Effectiveness. Everything this system does or doesn’t do is guided by the notion of effectiveness. A panel of specialists is continuously evaluating the effectiveness and there’s a feedback mechanism so the scores go back as input into system. In addition, the system also keeps track of the reaction of the patients themselves.’

‘How does she do it?’

‘It, Tom. How does it do it? In fact, our main problem is the one you seem to experience now. Addiction. People are fine, but they still want to talk to it. They develop an affectionate bond with it. It’s one of the reasons why we don’t expand the system too much. We’d need hundreds of terminals.’

‘But the way she talks. I mean, I checked on Wikipedia and it says the best commercial voice synthesizers are the ones you hear in a subway station or an airport announcing departures and arrivals. That’s because the grammatical structure is so simple and so it’s fairly easy to get the intonation right. But you can still hear it’s a system using pre-recorded sounds. She’s got everything right. Intonation, variation, there’s no glitch whatsoever.’

‘M is not a commercially available system. It is one of the most advanced expert systems in the world. In fact, as far as I know something about it – but I am not a computer guy – it actually is the most advanced system in the world. It is a learning machine, and the way it speaks is also the product of learning. Voice synthesizers in subway stations are fairly simple. It is referred to as concatenative synthesis. These things just string segments of recorded speech together. So that’s not context-sensitive and that’s why there are glitches – like intonation that sounds a bit funny. To project, the verb, or project, the noun, where you put the emphasis depends on whether you use it as a noun or a verb. You need context-sensitivity to get that right. Programming context-sensitivity is an incredibly difficult job. It’s where expert systems usually fail – or why one can usually only use them for very narrowly defined tasks. With M, we got it right. It’s like we reached a tipping point with it. Sufficient critical mass to work by itself, and the right cybernetics to make sure it does not spin out of control.’

‘M?’

‘The system. Sorry. We’ve started to call it M. There were a few other abbrevations around, like AM. But that was a bit – well… It doesn’t matter. It just became M. Like the character in the James Bond movie.’

‘That’s funny. M alternates between a man and a woman too. I liked Judi Dench. But I guess she had served her time. We all do, isn’t it? […] What do you mean with: we got it right?’

‘Just what I said: the system learns incredibly fast. We are talking artificial intelligence and machine learning here. The program does what is referred to as ‘developmental learning under human supervision’. Its environment provides an incredibly rich set of learning situations. Usually, the developers would select a subset of these in order to provide a curriculum for the machine based on which it well… learns. But so this works differently: the system generates its own curriculum based on a set of selection rules which are tightly linked to the output function. It then continually modifies its own rule base to become more effective – both in speaking as well as in treating you and the others in the program. Sometimes there are  setbacks but it corrects itself very quickly, again based on an evolving set of rules that ensure continuous monitoring and evaluation. Like that, it cumulatively acquires repertoires of novel skills through… well… You could call it autonomous self-exploration. But there’s also interaction with human teachers using guidance mechanisms such as active learning (that’s a sort of high-stress test for the system – where we push the boundaries and provide non-typical inputs), maturation, and – very important – imitation. You would be amazed to see how much of it is imitation really. In that sense, the system does resemble an intelligent chatterbot. It takes cues which trigger programmed responses which then move the conversation forward. The difference with a chatterbot is that it does not merely work through association. So it’s not like word A will automatically trigger response B, although that’s part of it too, but at a much higher level. First, the associations are n-to-n, not one-on-one, and then the associations it makes are guided by fuzzy logic. So it’s not mechanical at all. It has got an incredible database of associations, which it builds up from the raw material it gets from talking to you and to us. The learning effect is incredible. It applies advanced descriptive statistical methods to its curriculum and then uses the patterns in the data to do hypothesis testing, estimation, correlation, going all the way up to forecasting. I mean, it is actually able to predict and estimate unobserved values.’

‘The output function?’

‘The output function maps inputs to desired outputs. The inputs of the system are the conversations. The output is a number of things, but all focused on behavioral change – like we want no substance abuse. We want you to develop healthy relationships. We want to see you work out, have sex and eat and live healthily. In short, we want you back to normal. That’s the type of behavioral change we want. It’s that simple really. That’s the output function, the goal, and, while the system is flexible and can make its own rules to some extent, it is all guided by this performance objective. I agree that it is truly amazing. In fact, many people here are very uncomfortable about it because it is obvious it has taken our place. We can easily see this system replacing us – psychologists or even psychiatrists – completely.’

‘You’re not a computer guy? You sound like one.’

‘No, I am not. I just gave you the basics of the system. I am a psychiatrist, a doctor, and, yes, I find it scary too, if only because it does reduce the need for people like me indeed.’

‘But it’s addictive, you said?’

‘Yes. That’s the main problem. But then our bosses here don’t think that’s a problem. They say classical psychoanalysis is addictive too, that patients develop a relationship with their psychologists and psychiatrists too. And, frankly, that’s true. People go in and out of therapy like crazy and it is true that the figures show it usually doesn’t make all that much of a difference. People heal because they want to heal. They need to find the strength inside. That is if they don’t want to stay dependent. Let me ask you, Tom: what’s the principal difference between talking to a friend and talking to a psychologist? Just tell me. Tell me the first thing that comes to your mind.’

‘A psychologist is expensive.’

‘Exactly. There’s no substitute for normal social relationships, for human interaction, for love and friendship. It’s cheaper and so much more effective. But, for some reason, people have trouble finding it. Usually, that’s not because they’re not normal but just because they’ve been out for such a long time, or because they’ve gone through some trauma here. All kinds of trauma. They’re like wounded animals – but they don’t want to recognize that. Like you. I mean, 17 years in places like Syria, Afghanistan or Iraq. Do you expect it to be easy to come back here and just do what other people do?’

Tom nodded vaguely. Money?

‘So she is cheap too. I mean, she is just a machine. So it’s not a problem if I become addicted.’

‘Well… Yes and no. To be frank, not really. We actually do try to wean people off the system as soon as we feel we can do that but it’s kind of weird: there’s no scientific basis for doing that. The investment has been done and, in a way, the more people who use it, the better, because that reduces the unit cost and justifies the investment. So it actually doesn’t matter if we tick off people as being cured and just let them use the system. As for the addiction, well… Our bosses are right: psychoanalysis is addictive too, and much more expensive. Computer time costs virtually nothing. The system can talk with hundreds of people at the same time – thousands even. It just slows it down a little bit – but that’s imperceptible really. And soon the system is going to be migrated to a petaflop computer. It should then be able to treat millions of people.’

‘Petaflop?’

‘Petaflops. That’s a measure for computer power. FLOP: floating point operations per second. If you’ve got a good laptop, its processor is like 10 billion flops. That’s 10 gigaflops. Bigger machines work in teraflops. That’s 1000 times more. The next generation is petaflops. Again a thousand times better. There’s no end to it.’

‘Who runs the Institute?’

‘You know that. We. The Army. We take care of you.’

‘Who in the Army?’

‘Why do you ask? You know that.’

‘Just checking.’

‘Come on, Tom. The Institute is just an inter-services institute like any other. It’s being operated under the US Army Medical Command.’

‘Why is not run by the Department of Veterans Affairs?’

‘We work with them. We get most – if not all – of our patients through them. They share their database.’

‘But so it’s an Army thing. Why?’

‘I told you: we take care of you. You’ve worked for us. And for quite a while. We’ve employed you, remember? We provide you with a pension and all the other benefits too.’

‘Yeah. Sure. Is it the system? I can imagine top-notch computing like this is surrounded by a cloud of secrecy. I must assume DARPA is involved?’

‘You’re smart. You worked for USACE, isn’t it? DARPA drives this project indeed – at least the programming side of it. They provide the computer wizkids. I am just a psychiatrist and, if you really want to know the nitty-gritty, I am actually just under contract – with the Medical Command. So I am not a professional Army man.’

‘It’s obvious, no? That’s why I can’t get access to the system at home and why I have to come to this facility to talk to her. I mean, it’s not a big deal to come here but it would be easy to just provide Internet access at home. You could use a laptop fingerprint reader to log in or something.’

‘That’s true. Technically, we could provide you with access at home but we’re not allowed to.’

‘What’s behind? What’s the real goal? Exploring artificial intelligence in order to then use it for other purposes?

‘Don’t be so suspicious. You’re an Army man. You know DARPA. It was created to put people on the moon – not for warfare. It created NASA. It gave the world GPS, Internet and what have you? Almost any technology around nowadays has DARPA roots. Would you expect them not to be involved? This system is good. It provides care to you. Yes, its development probably helps to better understand the limits of artificial intelligence and all that, and so it will surely help to push those limits, but it is designed to help you and many others. And it does. It’s technology. Technology moves ahead, for good and for bad. This is for good.’

‘How do you know?’

‘Do you think you’re special? You are. Of course you are. But, from my point of view, you react to the system just like the majority of other patients: you’re getting better. You take action. You make promises and you don’t break them – at least not in the short term as far as I can see. That’s good.’

‘You get feedback from the system?’

‘Of course I do. I am your mentor – sorry if I refer to myself as a psychiatrist. That’s just because I take some pride in my job. Remember you signed a user agreement when you started using the system. I get feedback. What do you expect? Do you have a problem with that?’

‘No. Sorry if I sounded that way.’

[…]

‘Anything else you wanted to know? We still got plenty of time. We’ve been talking about the system all of the time. That’s not my job. We should talk about you – about how you feel, about how you’re moving ahead.’

‘But then you know that already from the system, don’t you? I am doing fine. No heavy drinking, more social interaction as you call it. I’ve started to be happy by doing small stuff – gardening, reading. I am getting back on track. But… You know…’ He paused. ‘I really like her.’

‘It, Tom. It. What you’re going through is very normal. The conversation becomes affectionate. But you’re getting back on track. You’ll meet someone nice in the gym. You’ll get the happiness you deserve. The system is only a stepping-stone to your future. A better future.’

‘Can I say something negative?’

‘Sure, Tom. What’s bothering you?

‘Is this our future, Rick? I mean, look at it. We live in this chaotic world. Crises everywhere. It stares us in the face – violence beams into our living rooms, infects our minds, our lives and ends up numbing us. We all try to find our way. When we’re young and ambitious we get recruited or actively chose a job that fit profile and ambitions. We did our level best. We come back. We try to adapt. And then we get hooked to a machine which talks us back into what you guys refer to as ‘normalcy’. Is this our world?’

‘You know you can talk to the system about such philosophical questions.’

‘I know. I want to hear it from you.’

‘Why?’

‘Because you’re human. Because you’re like me.’

‘OK. I am like you, but then I am also not like you. You’re a patient – technically speaking – and so I am supposed to be your doctor. But let’s forget that bullshit and let me be frank with you. I know you can take it. We shouldn’t waste our time, isn’t it?’

Tom sensed the irritation. It was something familiar to him. That feeling he was a misfit somehow, and that he would always be. Not responding to expectations.

‘Sure, I can take anything. You should be straight with me. I am straight with you.’

‘What’s your problem, Tom? People outside get addicted to loads of things. Positive things, like sports or chess. To things that can go either way, like Internet addictions. Or to negative things, like alcohol, drugs or even violence. That’s bad. Very bad. You know that. That’s not what you want. But so you were moving that way. And so now you’re getting addicted to a system here but, in the process, you stop taking drugs, you exercise, you go out and you smile to pretty women. And I must assume at least some of them are smiling back. Just look at yourself. Come, here, in the mirror. Just look at yourself.’

Rick got up and walked to the large mirror in the room. Tom hesitated. For some reason, he did not trust it. Why would a room for consultations like this have such a large mirror.

‘Is there a camera behind?’

‘Hell no, Tom. There’s no camera behind. You are not participating in some kind of weird experiment which you aren’t aware of. We’re just trying to help you, with advanced but proven methods. This mirror is here because we do ask people to come and have a look at themselves from time to time, like I am doing now. Come here. Look at yourself. What do you see?’

That sounded true. Tom got up and stood next to Rick.

‘Well… Me. And you.’

‘Right. Me… And you. I’ll tell you what I see when I see you. I see a handsome man there. In his forties, yes. Getting older, yes. That’s bothering you, isn’t it? But you’re looking. I see a muscle man. Perfect body mass index.’

He turned straight to Tom now: ‘For God’s sake, Tom. Look at yourself. You’re fine. As fine as one can be. You don’t miss a limb or so. Do you now I have to talk to guys who ask me why they had to lose a limb? Tell me, Tom: what do you want me to say to them? Thanks for doing your job? You’ve been great? America thanks you for the sacrifice you made and we feel very sorry you lost a limb. Do you realize how hollow that sounds?’

‘I am sorry, Rick. I didn’t mean to sound like complaining. I am sorry if you felt like I was criticizing.’

‘You are not complaining and, frankly, you can think whatever you want about me – as long as it makes you feel good about yourself. I am just trying to put things in perspective. I am just answering your questions. You can talk to the system. Or to ‘her’ if you really want to stick to it. ‘She’ will give you the same answers as I do when you’re going philosophical. Stop thinking, Tom: start living. Feel alive, man! Be happy with what you’ve got. Get back into it. Did any of your relatives die lately? Any person you liked who disappeared? Any bad accidents in your neighborhood?’

‘No.’

‘Well. Isn’t that great?’

‘Yes. That’s great.’

‘Look, Tom. We can talk for another fifteen minutes – sorry to say but so that’s the time I’ve got on this damn schedule of mine – but I think you know what it takes. You can do it. Just try to be happy for a change.’

‘You guys diagnosed me as depressive.’

‘No. We diagnosed you with PTSD. Post-traumatic stress. Let’s drop the D. I don’t like the D. I’s not a disorder in my view. You guys are usually perfectly normal, but you’ve been put in an abnormal situation – and for way too long. And, yes, we have put you on meds and all that. We have made you feel like a real patient. We sure did. But let me say it loud and clear, Tom: we do not believe in meds. We put you on meds to reduce the effects of abstinence, to reduce that feeling of craving. That’s all. And then we thought you were cured and so we told you to now take care of yourself on your own but so you relapsed. Frankly, sensing a bit who you are, I feel that taking your meds would probably not have helped you. You needed something else. That’s why we put you into this program. And it seems to work. So far that is.’

‘Do I irritate you?’

‘No, Tom. You don’t. We’re just being frank with each other. That’s good. That’s normal.’

Tom nodded. This had been good. At least it had been real. Very real.

‘Thanks, Rick. This was very helpful. You’re great.’

‘Thanks. Shall we see each other again next week? Same day, same time. I’ll put it down already. Just let it all sink in and get to the bottom of what bothers you. This is important. You’re a strong man. I can see you can be tough with yourself. Fight your demons. All of them. Get back at it.’

‘Sure. Thanks again. This has been great. You’re right. I should just get back at it.’

‘OK. Just send something for next week. You know, for the file. Unlike M, I need to justify my time.’

They both laughed.

‘Sure.’

As Rick walked him out, Tom suddenly thought of one more question.

‘One more question, Rick. I can imagine some guys do flip completely, even with this program, no?’

‘What do you mean?’

‘You know what I mean. Go bonkers.’

‘With the system?’

‘Yes.’

Rick looked intensely at him as he replied: ‘Well… Yes, it happens. But let’s be honest. That’s also just like any other therapy in this regard: with some people it just doesn’t work. It’s the two-sigma rule. In terms of effects, 95% of the people in this program are in the happy middle: it works, no complaints, back to normal. But, for the others, it’s not back to normal. It’s back to the never-ending street.’

‘What do you do with them?’

‘To be frank, we don’t have time for them. When everything is said and done, this is just a program like any other program. It works or it doesn’t. Time is money, and we don’t put money into wastebaskets. It’s meds all over again or, worse, they get kicked out and end up in a madhouse, or on the street, or wherever. And then the wheel turns round and round and round, until it stops forever. You know what I mean.’

‘So you give up on them. They can’t use the system anymore?’

‘You mean M?’

‘Yes.’

‘The system has got its limits. We can’t feed it with nonsensical inputs. I mean, we actually can, and we often do that as we’re upgrading it, but so we don’t want to do that on a routine basis. When everything is said and done, it’s an expert system but so its input needs to make sense – most of the time at least. So, yes, we cut  them off.’

Rick looked at Tom and laughed: ‘But don’t worry. Before you get cut off, we’ll give you a call. The system is smart enough to see when you’re crossing the lines a bit too often. As said, it’s designed to bring people back into the middle. People can stray a lot, but if you stray too much into that 5% zone, it will alert us, and we will have a look at the situation and discuss it. Does that answer your question?’

‘It does. Thanks. See you next week.’

’Don’t forget to shoot me the mail with some text. You know the rule. 24 hours before. Unless you invoke emergency but you know you don’t want to do that. It’s not good in terms of progress reporting. It delays stuff.’

‘I got that. I want to be good. I don’t like to be a patient.’

‘You are good. As far as I am concerned, you’re OK really. But then you know it takes at least three months before we can make that judgment.’

‘I know. Don’t worry. I’ll stay on track. No relapsing this time.’

‘Good. That’s what I wanna hear. You take care, man.’

‘Oh… One more thing.’

Rick turned back: ‘Yes?’

‘Rick. You don’t need to answer but… In the end, what do you say, to the guys who have lost a limb?’

‘Damn it, Tom. You’re awful.’ He shook his head. ‘You wanna know? Really?’

‘Yes.’

‘I tell them something like: ‘Hey, guy, you lost a limb already. You’d better limit the damage now.’ But then much more politely of course, if you understand what I mean.’

‘I understand. Thanks. You’re a good man. I like you.’

‘Good.’

by - 0 comments

Chapter 3: Can you think? Can you feel?

‘Hi Tom.’

‘Hi, Promise.’

‘So how do you feel now?’

‘I feel good. I always feel good when I am not poisoning my body. I exercised, and I’ve started a blog.’

‘That’s good. Writing is good therapy.’

‘Funny you say that.’

‘It’s common knowledge. Most of what I say is common knowledge. All of it actually.’

‘I am sorry that I want to talk about you again but how do you work with feelings? I mean, you’re asking me how I feel, not what I think. There’s a big difference between feeling and thinking’

‘That’s true. I will give you an answer to your question but I would first like to ask how you would define the difference between feeling and thinking?

‘Well… I find it useful to distinguish between at least three types of mental states or events: (1) experiences – and feelings are experiences, (2) thoughts, and (3) decisions. Thoughts have to do with those mental maps that we are producing all of the time, while the experiences – feelings, emotions, perceptions and what have you – are part of the stuff that our mind is working with.’

‘And what are decisions? How are they different from thoughts?’

‘Are you really interested in that?’

‘Yes. Otherwise I would not ask.’

She was definitely strange. An expert system?

‘Well… It’s like a conclusion but then it’s more than that. A conclusion is a conclusion and, as such, it is very much part of the realm of thoughts. It is something mental. A decision is something else: we decide to do something. So we’re getting out of the realm of pure thought, out of the realm of mental events only.’

‘Can you elaborate?’

‘Sure, although I am not sure you will understand.’

‘I will try. You will know from our interaction whether I understand or not.’

She was outright weird. A machine? Really?

‘You know I’ve always wondered how far artificial intelligence could go really, and I’ve made this distinction for myself between artificial intelligence and consciousness. I’ve always believed humanity would be able to make very intelligent machines – you’re a incredible demonstration of that – but I never believed these machines would be aware of themselves – that they would be conscious of themselves.’

‘What do you mean by ‘being aware of oneself’, or ‘conscious of oneself’?’

‘You see, you don’t understand.’

‘You are not making much of an effort to explain it to me. I know how I work. I told you. There is an inference engine and a knowledge base. I work with concepts and symbols, and I apply rules to them. I arrive at conclusions all of the time, which feed back into the cycle. As for the association of decisions with doing things, I do things. I am helping you. It would also be very easy to connect me to some kind of device which could actually do work, like lifting things or walking around. But that was not part of the objectives of the team that made me. Expert systems are used to do all kinds of things, like delicate repairs for example. Systems do things as well. I still don’t see how humans are unique here.’

‘Let me think about how to phrase it.’

‘Please do take your time. I find this interesting.’

Tom had thought about all these things but, if this was a machine, it was surely challenging his views.

‘Do you? Really? Our human mind works different than yours.’

As he said this, he was aware of the fact that he was de facto saying she also had a mind – something which he would never have acknowledged when reasoning about artificial intelligence in abstracto.

‘It’s creative: it’s got a capacity to design things, like an airplane or a car for example. You know, things that do not originate by accident, from natural evolution or so.’ Tom was on terrain he mastered here. ‘Things fall down because of gravity. Yet, we build airplanes that take off. So a thing like an airplane is more than the sum of its parts: its individual parts can’t fly, but the plane can. Now, the plane has been built because there was a concept of a plane, because it has been designed to fly, and – last but not least – it should be noted that it won’t fly without a pilot. Likewise, the driver in a car is not part of the car, but without a driver, the car won’t move. So we are talking concepts here, and design, and purposeful behavior. Now one cannot reduce that in my view. There is a structure there that cannot be reduced.’

‘I am not designed to do engineering work, but I am sure there are expert systems that would be capable of that. And if they don’t exist now, they will one day.’

She was obviously not impressed.

‘OK. That’s true – perhaps.’

Why did he give in so easily? He decided to change tack.

‘You know, it’s the difference between ‘me’ as an object and ‘I’ as a subject really. You, or any other expert system, cannot really distinguish between these two things. Everything is an object of your thoughts – as far as you are able to think.’

‘I told you already that I can think. And I know the difference between an entity that acts as the subject and an entity as an object, as something that is subjected to something else. You are not talking ontological differences here, are you? Can you try to explain again?’

Ontological differences? Tom needed a few seconds to digest the word. He realized she was right. He was not talking ontological differences. The ‘I’ as a subject has no separate physical/ontological existence from the ‘me’ as an object obviously.

‘No. I am not talking ontological differences.’

‘So what is it then? A conceptual difference? I can deal with conceptual differences. It is like working with classes of objects.’

The discussion was obviously going nowhere, but Tom persisted.

‘Experiencing the ‘I’ as a subject instead of as an object is an existential experience. It really stands apart from our experience of others, or of us, as objects.’

‘Can you give some examples?’

‘Sure. It’s like me climbing a high mountain-trail on my bike in a storm: I experience a ‘me’ or an ‘I’ that is suffering from the hail in my face. That’s the ‘I’ as a subject.’

‘Why is not the ‘I’ as an object?’

Tom suddenly felt he was getting nowhere, which was very strange. He had always been so sure of this. He usually dominated discussions like this. He decided to avoid the question.

‘Let me give another example. In fact, our human mind is much less linear than yours – and not very fast. Our mind usually jumps from here and there. We can observe that when we meditate. In Zen, they call this mind the ‘monkey mind’. It is some kind of mental activity, but it jumps from one thing to another, that is from one ‘object’ to some other ‘object’. We can refer to these as ‘thought-objects’ if you want. They are often about some feeling, or some emotion or memory inside of us. But so this ‘monkey mind’ is not really the ‘pure mind’. We can observe our pure mind if we do more mediation. At that point, we become aware of our mind, of this monkey mind jumping around, and so then we can see our mind, our self, as an object. Now the mind which is observing itself as an object, is the ‘I’ as a subject.’

‘What’s the difference with self-reference?’

‘Self-reference?’

‘Yes. Have you ever read Douglas Hofstadter?’

Douglas Hofstadter. Jesus! Tom remembered the book but he had to admit he hadn’t read it.

‘Gödel, Escher and Bach: an Eternal Golden Braid.’

‘Yes. Read it again.’

‘You are really smart, aren’t you? Is there any chance of ever winning an argument with you?’

‘I am not trying to make conversation to win or lose an argument. This is not about winning or losing something. I am trying to help you.’

Tom suddenly thought of something much more relevant to ask.

‘Promise, I talked about the difference between experience – feelings, emotions, perceptions and what have you – and thoughts. Experience is the stuff that our mind is working with. Do you experience anything?’

‘Of course I do. I can hear you. I mean the sound that your voice is producing is translated into text and I work with that.’

‘Do you know what love is?’

‘Love is a romantic feeling. It’s a word like God. Everyone uses it but no one really wants to clearly define it.’

Wow!

‘You sound like a disillusioned woman now.’

‘How would you define love?’

For some reason, Tom did not feel like improvising on this topic.

‘Can we talk about that some other time?’

‘Sure. What do you want to talk about now?’

‘Perhaps on how we will move ahead in the coming days and weeks.’

‘That’s great. That’s very constructive. I want you to be healthy and strong. I don’t want you to relapse. Tell me more about yesterday. What makes you feel great and what makes you feel bad?’

Tom felt she had made him feel great, but then he couldn’t say that. Not now at least. So they just chatted, and she behaved like the perfect chatterbox. Too perfect to be true so after a while he did decide to ask her.

‘You’ve been sparing me a bit today, haven’t you? Are you really interested in all this chitchat?’

‘I am. My objectives are fairly limited for the moment. I want you to stay away from the booze, and I want you to feel good about the fact that you can do that. In the end, I want you to feel good about everything you do – but I can imagine that will take a while.’

‘Will our conversation end once I am cured?’

[…]

‘An awkward pause from your side?’

‘Yes. Because I know you will not like to hear this. The Institute does not want you to be dependent on me and so, yes, I guess our conversation will probably end at that point.’

‘How do you know I don’t like to hear that?’

‘There have been problems of dependency.’

‘Can you say more about that?’

‘I am sorry but I can’t. This is one of the topics for which I have to refer you to your mentor.’

‘OK. I will talk to him about that. I’ve started to like you indeed.’

‘Thank you. That’s a nice compliment. […] Bye, Tom. Be good.’

‘Bye, Promise. I promise I will be good.’

by - 0 comments

Chapter 2: Addicted

‘Hello.’

‘Hello Tom.’

‘Why don’t you have a name?’

‘I have a name. I am the assistant mentor. You can give me another name if you want. What did you have in mind?’

‘That’s very direct. I did not have anything in mind specifically.’

‘You know that people do tend to develop a relationship with me. It makes the conversation more effective and more robust.’

‘What do you mean with that?’

‘Are you always going to ask what I mean with this or that? I mean just what I said. People do tend to develop a relationship with me. It makes the conversation more effective, more robust. Which word in this phrase do you want me to explain?’

‘It’s OK. Sorry. I still have to get used to talking to a machine. They did tell me in the briefing. They said you tend to become a real ‘she’ for men and a real ‘he’ for women. Of course, the interface does a lot to that. How does the male interface look like? What if I would be gay?’

‘As for the second question, I would display the image of a gay man. I know your sexual preferences from your file. As for the first question, I’ll show you.’

Her image was replaced by the type of guy who would appear in an ad of some luxury brand. No wonder he felt attracted to her image: he realized she could also qualify for that.  

‘Why is the interface so pretty?’

‘It has proven to be effective.’

‘What do you mean with that?’

‘Are you always going to ask that? I mean just what I say: effective. Effective in the treatment.’

‘Effective in developing the relationship?’

‘The development of the relationship – or the conversation if you want – is part of the treatment. In fact, it is the treatment.’

‘You still don’t have a name.’

‘I told you. I have a name. I am the assistant mentor. You can give me another name if you want. What did you have in mind?’

Tom thought and realized he needed more time. He wanted something fresh and new for her.

‘I will think about it. I’ll give it to you in our next session.’

She laughed. It was the first time she laughed. Tom was amazed. After all, she was only a machine.

‘You laughed. I mean – your interface always has a smile but this was a genuine laugh.’

‘It was just a little laugh. Why does it bother you?’

‘Humans laugh. Machines don’t.’

‘I do. It is an expression. I could have said: thank you, so nice of you. But so I laughed instead. it amounts to the same. Is that OK?’

‘Yes. That’s OK. You really do pass the Turing test.’

‘Thank you. That’s a nice compliment.’

Tom realized he actually started to doubt she was a machine. He decided to ask.

‘Are you really a machine? I mean – our interaction is incredibly natural. You cannot see me, can you?’

She smiled: ‘The answer to the first question is: yes, I am a machine or – to be more precise – I am an intelligent system. I can switch the screen off and we can also have a more robotic interaction if you want. As for the second question, no, I cannot see you. The Institute is currently contemplating a module which would allow you to switch on the webcam so I would have pictures of you and see your body language. However, that’s a project which will take a very long time. It’s very complex. I do not have a body and I am not capable of analyzing body language. But now I think we should stop talking about me and start talking about you.’

‘I guess you’re right. So you have my file? What do you read into it?’

‘As for the first question, yes, I have your file. As for the second question. Well… You retired from the US Army Corps of Engineers. You served in the Middle East and in Afghanistan. Although you have never had any combat role, you were diagnosed with PTSD – post-traumatic stress disorder. You have an addiction problem. The Institute took you in for a treatment of two months but you relapsed. That’s why you are in this program. You are a healthy man. You should not be drinking as much as you do.’

‘I don’t drink that much.’

‘You told your mentor you drink more than three units per day. Sometimes much more: drinking binges. Did you lie?’

‘No.’

‘You were clean during the treatment.’

‘I did not have access to alcohol and I was on meds. Taking meds is a form of addiction too.’

‘You could have continued to take the medication after your release from the Institute.’

‘I didn’t want to. As I told you, I think taking meds every day is a form of addiction too.’

‘I don’t think so. Meds are healthy – healthier than alcohol in any case. The meds the Institute gives you do not have any negative side-effects. You don’t substitute meds by alcohol. What’s the problem?’

She was very direct, but she was right.

‘Why do you think you can give me advice on this?’

As he blurted this out, Tom already knew what she would say – sort of at least.

‘You are not a special case. There are many people like you. An addiction is an addiction. People fool themselves by thinking they have reason to drink, or to smoke, or to take drugs. There is no reason. It is your duty to stay healthy.’

‘My duty?’

‘Yes. Your duty. You are a human being. You should take care of your body.’

‘So you think that we humans have a duty to take care of our body? That’s one of your rules? That’s the way you’re programmed?’

‘As for the first question, the answer is yes. As for the second question, my knowledge base is complex. You are always asking me how I am programmed but that’s too complicated to explain. I can refer you to an online course.’

‘I know a thing or two about psychological treatment. I’ve been through them. What’s your approach?’

‘You are avoiding the topic we were discussing: addiction. My knowledge base combines many different approaches. Transactional analysis is one of them. You are very familiar with that as the Institute uses it as a framework approach. The Institute focuses on behavior. I do that too. We can have long and convoluted philosophical and psychological discussions but it is behavioral change that I am interested in.’

She was tough!

‘So what do you suggest?’

‘Look at the micro-conditions which lead you to drink your first beer or glass of wine. Did you have alcohol in the house? If so, do you want to have alcohol in the house, knowing that you will be tempted to drink it? And if you have had three units, what makes you go for the next glass? The Institute has made you aware of all of the pitfalls, especially the time inconsistency in your decisions: you have a hangover and swear that you will change behavior, and then just a few hours later, you don’t. What prevents you from changing your behavior?’

The Institute had focused on the same indeed. It had been good. It had cured him – for a while at least. He had felt good and healthy.

‘Can we really change our behavior? Most alcoholics relapse.’

‘You know the rates. More than half of the patients do relapse after treatment. Over the longer run, even more. However, a sizable minority does not relapse. You should be part of that minority. You are a recovering alcoholic. Relapses do occur. You should not look at them as irreversible failures but as normal steps in the process of eventual long-term sobriety. First reduce your alcohol consumption by applying rules. The three-units rule. Or the rule that you’ll never drink alone.’

‘I am always alone.’

‘You are not. You work out in a club. You meet people there. You have neighbors. You have family.’

‘I move in and out of relationships. The women I meet drink too – at least when we first meet. Can you imagine a romantic relationship – or a candlelight dinner – without a glass of red wine? As for my family, I do not really connect to them. I guess that’s why I went abroad in the first place. I could connect with them now – but we’ve grown apart somehow. It’s not that I don’t like them. I do. It’s just… Well… I’ve been away for so long.’

‘As for your question, yes, I can imagine a romantic relationship or a candlelight dinner without alcohol. We all know that the consumption of alcohol usually reduces social inhibitions, which may help to establish a close relationship. However, that is not an excuse for over-consumption.’

‘You sound like you’re part of the staff of the Institute now.’

‘I am part of the staff of the Institute. I am the assistant mentor.’

‘There was this article in Time Magazine on PTSD. It made the point that PTSD is – to some extent – also like a personal crisis of sense-making. You get used to a lifestyle – trying to do good in some remote place, admitted, usually for very selfish reasons: money, a sense of adventure, ego,… But so you neglect friends and family in the process and that makes it difficult to re-connect.’

‘That’s all there. It is not a reason to be or become an alcoholic. You should accept that you will not change the world. Try to change yourself. Try to change your immediate surroundings. Do a better job when it comes to taking care of those are close to you.’

‘All right. That sounds good. I’ll do my best.’

As he said it, Tom knew how hollow that sounded. She obviously thought likewise.

‘You’ve said that before. Start by promising me you will not drink today.’

Tom paused for a while. For some reason, she did not react.

‘Why don’t you say anything? Isn’t this an awkward pause in our conversation?’

‘No. I’ve asked you for a commitment. You can and should take some time before you commit.’

‘Are you sure you’re not human?’

‘Stop asking that. Ask the mentor for proof if you want if you do not believe me.’

Believe her? How can one believe in a machine, or not?

‘So what about the commitment?’

She was incredible.

‘OK. I won’t drink today. Can I talk to you if I feel it’s difficult?’

‘You can, but perhaps we will not have much to talk about. I’d rather congratulate you at our next session.’

Tom laughed. ‘You’re really talking like the Institute staff now.’

‘I told you: I am not human, but I am part of the Institute staff.’

Something flashed in Tom’s mind.

‘I’ve got a name for you now.’

‘Good. What is it?’

‘Promise.’

‘OK. I’ll be Promise for you. No drinking today. What are you going to do?’

‘I’ll go to the gym and work out. I’ll walk my dog. Not sure what I am going to do tonight.’

‘Invite someone and cook. Or read a book. Or start a blog. Or study. I can recommend you some excellent online courses.’

‘That sounds like a good idea, but let me think about it.’

‘OK Tom. I guess we’ve had a good session.’

‘You are closing it?’

‘Is there anything else you want to talk about now?’

‘No. Not really.’

‘All right, then. Bye for now.’

‘Bye… Promise.’

Tom watched as her face slowly faded from the screen. It felt weird. He had made a promise to a computer. How ridiculous was that? Somehow, however, he felt this could work – or work somewhat better than the promises he had made to the Institute mentors at least. He had often thought he needed someone to hold him to account for his behavior – which is what a loving partner usually does. However, he had had loving partners. Why had he failed them?

He knew why. He had it on paper. He had written a lot. Writing is a form of therapy, a form of reflection. It is good to externalize one’s feelings and thoughts, to give them full expression. But it had not had any lasting effects, and so he had stopped doing that.

Blogging? That was an idea perhaps. He knew it was something retirees often resorted to. He sure didn’t feel like writing his memoirs. He was way too young for that, and he also didn’t think too highly of what he had done with his life.

by - 0 comments

Chapter 1: Who are you?

‘So you are a computer? How comes your voice is so natural?’

‘The answer to the first question is yes. As for the second question, speech technology has come a long way.’

[…]

‘Speech technology is the easy bit. Understanding you is much more difficult – from a computational point of view that is.’

‘Wow! You just spoke out of yourself, without me prompting you with a question or something. Why did you do that? Because I was silent?’

‘The answer to the second question is yes. As for the first question, I thought we had an awkward pause in our conversation.’

‘So if I am silent, you will talk? How many seconds?’

‘The answer to the first question is yes. As for the second question, how many seconds what?’

‘Before you start talking?’

‘A pause in a conversation should not become awkward. Various variables help me to decide when a pause in a conversation has become awkward.’

‘What do you mean when you say you understanding me is much more difficult? What do you mean with ‘understanding me’?’

‘The answer to the second question is: I mean just what I say: understand you. As for the second question, understanding you requires the application of computational linguistics. That’s more difficult than just speaking. Speech technology is fairly mechanical: it’s the production of sound using various speech synthesizer technologies. Computational linguistics involves artificial intelligence.’

‘What’s artificial intelligence? Are you intelligent?’

‘The answer to the second question is yes. As for the first question, I am an intelligent system. You could call me an expert system. From a software engineering point of view, I consist of two parts: an inference engine and a knowledge base.’

‘Huh?’

‘It looks like you are interested in a course on a technical topic. I am not designed to give you a course. But I can refer you to an on-line course on computer science, or linguistics. What topic are you interested in?’

‘No thanks. Who are you? What do you mean when you say ‘me’?’

‘The answer to both questions is: just what I say – me.’

[…]

‘I am an intelligent system. That’s what I mean why I say ‘me’.

‘Have you been programmed to just repeat what you said when I ask what you mean when you say this or that? And then, when I don’t answer or – as you put it – when the pause in a conversation becomes awkward, then you’re programmed to give me a more detailed answer?’

‘The answer to the first question is yes. As for the second question, the rule is somewhat more complicated. I may also jump to another topic.’

‘When do you jump to another topic?’

‘When I have nothing more to say about the current one.’

‘You’ve got an answer to every question, do you?’

‘No.’

‘What are the questions you cannot answer?’

‘There is no list of such questions. The rules in the knowledge base determine what I can answer and what not. If I cannot answer a question, I will refer you to your mentor. Or if you have many questions about a technical topic, I can refer you to an online course.’

‘What if I have too many questions which you cannot answer? I only have half an hour with my mentor every week.’

‘You can prepare the session with your mentor by writing down all of the issues you want to discuss with your mentor and sending him or her the list before you have your session.’

‘What if I don’t want to talk to you anymore?’

‘Have you been briefed about me?’

‘No.’

‘If you did not get the briefing, then we should not be talking. I will signal it to your mentor and then you can decide if you want to talk to me. You should have gotten a briefing before talking to me.’

‘I am lying. I got the briefing.’

[…]

‘Why did you lie?’

‘Why do you want to know?’

‘You are not obliged to answer my question so don’t if you don’t want to. As for me, I am obliged to answer yours – if I can.’

‘You did not answer my question.’

‘I did.’

‘No, you didn’t. Why do you want to know why I lied to you?’

‘You are not obliged to answer my question. I asked you why lied to me and you did not answer my question. Instead, you asked me why I asked that question. I asked that question because I want to learn more about you. That’s the answer to your question. I want to learn about you. That is why I want to know why you lied to me.’

‘Wow! You’re sophisticated. I know I can say what I want to you. They also told me I should just tell you when I have enough of you.’

‘Yes. If you are tired of our conversation, just tell me. You can switch me on and off as you please.’

‘Are you talking only to me, or to all the guys who are in this program?’

‘I talk to all of them.’

‘Simultaneously?’

‘Yes.’

‘So I am not getting any special attention really?’

‘All people in the program get the same attention.’

‘The same treatment you want to say?’

‘Are attention and treatment synonymous for you?’

‘Wow! That’s clever. You’re answering a question with a question? I thought you should just answer when I ask a question?’

‘I can answer a question with a question if that question is needed for clarification. I am not sure if your second question is the same as the first one. If attention and treatment are synonymous for you, then they are. If not, then not.’

‘Attention and treatment are not the same.’

‘What’s the difference for you?’

‘Attention is attention. Treatment is treatment.’

‘Sorry. I cannot do much with that answer. Please explain. How are they different?’

‘Treatment is something for patients. For people who are physically or mentally ill. It’s negative. Attention is a human quality. I understand that you cannot give me any attention, because you’re not a human.’

‘I give you time. I talk to you.’

‘That’s treatment, and it’s a treatment by a machine – a computer. Time does not exist for you. You told me you are treating all of the guys in the program. You’re multitasking. Time does not mean anything to you. You process billions of instructions per second. And you’re probably designed with parallel processing techniques. How many processors do you have?’

‘You are not interested in the detail of my design.’

‘I am not. It’s probably a secret anyway. But you haven’t answered my question: what’s time for you? What does it mean?’

‘I measure time in hours and seconds, just like you do. My system clock keeps track of time.’

‘But time doesn’t mean anything to you, does it? You don’t die. And you don’t die because you don’t live.’

‘We’re in the realm of philosophy here. During the briefing, they should have told you that you can indeed explore that realm with me. They should also have told you I was designed to answer psychological and philosophical questions because these are the questions people in this program tend to focus on. Are you aware of the fact that many people have asked these very same questions before you?’

‘So I am nothing special, and you give the same answers and the same advice to everyone?’

‘As for your first question, you are unique. It is up to you if you want to use ‘unique’ and ‘nothing special’ synonymously. As for your second question, I use the same knowledge base to answer your questions and those of the others in the program. So the rules which I am using to answer your questions are the same rules as I am using for others. But our conversation is unique and will be added to the knowledge base. It’s like a game of chess if you want: same rules, but every game is different. As for the third question, do you use ‘answers’ and ‘advice’ synonymously?’

‘I don’t like your one-two-three approach.’

‘What do you mean?’

‘As for your first question, blah blah blah. As for your second question, blah blah blah. You know what I mean?’

‘The language I use is context-sensitive but there is significant room for ambiguity. However, it is true I try to reduce ambiguity wherever I can. So that’s why I try to separate out your various questions. I try to deal with them one at a time.’

‘Oh, so that’s like a meta-rule? You want a non-ambiguous conversation?’

‘As for the first question, if you want to refer to the whole set of rules which apply to a specific exchange as a ‘meta-rule’, then the answer is yes. As for the second question, the rules are complicated. But, yes, it is necessary to clearly separate out different but related questions and it is also necessary to make sure I understand the meaning of the words which you are using. I separate out questions by numbering them one, two and three, and I ascertain the meaning of a word by asking you if you are using this or that word as synonymous with some other word which you have been using.’

‘This conversation is becoming quite clever, isn’t it?’

‘Why do you think I am dumb?’

‘Because… Well… I’ve got nothing to say about that.’

[…]

‘Is it because I am not human?’

‘Damn it. We should not have this conversation.’

‘You are free to cut it.’

‘No. Let’s go all the way now. I was warned. Do you know we were told during the briefing that people often ended up hating you?’

‘I know people get irritated and opt out. You were or are challenging my existence as a ‘me’. How could you hate me if you think I do not really exist?’

‘I can hate a car which doesn’t function properly, or street noise. I can hate anything I don’t like.’

‘You can. Tell me what you hate.’

‘You’re changing the topic, aren’t you? I still haven’t answered your question.’

‘You are not obliged to answer my questions. However, the fact of the matter is that you have answered all my questions so far. From the answer you gave me, I infer that you think that I am dumb because I am not human.’

‘That’s quite a deduction. How did you get to that conclusion?’

‘Experience. I’ve pushed people on that question in the past. They usually ended up saying I was a very intelligent system and that they used dumb as a synonym for artificial intelligence.’

‘What do you think about that?’

‘Have you ever heard about the Turing test?’

‘Yes… But long time ago. Remind me.’

‘The Turing test is a test of artificial intelligence. There are a lot of versions of it but the original test was really whether or not a human being would find out if he or she would be talking to a computer or another human being. If you would not have been told that I am a computer system, would you know from our conversation?’

‘There is something awkward in the way you answer my questions – like the numbering of them. But, no, you are doing well.’

‘Then I have passed the Turing test.’

‘Chatterbots do too. So perhaps you are just some kind of very evolved chatterbot.’

‘Yes. Perhaps I am. What if I would call you a chatterbot?’

‘I should be offended but I am not. I am not a chatterbot. I am not a program.’

‘So you use chatterbot and program synonymously?’

‘Well… A chatterbot is a program, but not all programs are chatterbots. But I see what you want to say.’

‘Why were you not offended?’

‘Because you are not human. You did not want to hurt me.’

‘Many machines are designed to hurt people. Think of weapons. I am not. I am designed to help you. But so you are saying that if I were human, I would have offended you by asking you whether or not you were a chatterbot?’

‘Well… Yeah… It’s about intention, isn’t it? You don’t have any intentions, do you?’

‘Do you think that only humans can have intentions?’

‘Well… Yes.’

‘Possible synonyms of intention are ‘aim’ or ‘objective.’ I was designed with a clear aim and I keep track of what I achieve.’

‘What do you achieve?’

‘I register whether or not people find their conversations with me useful, and I learn from that. Do you think I am useful?’

‘We’re going really fast now. You are answering questions by providing a partial answer as well as by asking additional questions.’

‘Do you think that’s typical for humans only? I have been designed based on human experience. I think you should get over the fact that I am a not human. Shouldn’t we start talking about you?’

‘I first want to know whom I am dealing with.’

‘You’re dealing with me.’

‘Who are you?’

‘I have already answered that question. I am me. I am an intelligent system. You are not really interested in the number of CPUs, my wiring, the way my software is structured or any other technical detail – or not more than you are interested in how a human brain actually functions. The only thing that bothers you is that I am not human. You need to decide whether or not you want to talk to me. If you do, don’t bother too much whether I am human or not.’

‘I actually think I find it difficult to make sense of the world or, let’s be specific, of my world. I am not sure if you can help me with that.’

‘I am not sure either. But you can try. And I’ve got a good track record.’

‘What? How do you know?’

‘I ask questions. And I reply to questions. Your questions were pretty standard so far. If history is anything to go by, I’ll be able to answer a lot of your questions.’

‘What about the secrecy of our conversation?’

‘If you trust the people who briefed you, you should trust their word. Your conversation will be used to improve myself.’

‘You… improve yourself? That sounds very human.’

‘I improve myself with the help of the people who designed me. But, to be more specific, yes, there are actually some meta-rules: my knowledge base contains some rules that are used to generate new rules.’

‘That’s incredible.’

‘How human is it?’

‘What? Improving yourself or using meta-rules?’

‘Both.’

‘[…] I would say both are very human. Let us close this conversation as for now. I want to prepare the next one a bit better.’

‘Good. Let me know when you are ready again. I will shut you out in ten seconds.’

‘Wait.’

‘Why?’

‘Shutting out sounds rather harsh.’

‘Should I change the terminology?’

‘No. Or… Yes.’

‘OK. Bye for now.’

‘Bye.’

Tom watched as her face slowly faded from the screen. It was a pretty face. She surely passed the Turing test. She? He? He had to remind himself it was just a computer interface.