‘So how do you work with my mentor, Promise?’

‘He gets a weekly summary of our interactions and performance scores accompanied by an assessment.’

‘Can you show me?’

She disappeared and the screen now showed a detailed report. Not much text. Plenty of graphs and scores.

‘Can you print this out or something? I can hardly read it.’

‘Sorry. No print-outs. But you can ask for more detail if you want. What is that you would want to enlarge?’

‘What does it say? I mean, what’s the summary?’

‘The summary is shown at the bottom.’

Tom peered at the screen. Performance statistics. They were grouped under three headings: attitude, functions, and lifestyle.

‘That’s Myers-Briggs, isn’t it? What’s the red, orange and green?’

‘As for your first question, our approach includes an assessment using Myers-Brigg type indicators but it’s only one of the tools. That being said, it’s true the methodology shares the same approach: psychometrics. Our conversations yield a lot of data which is then matched to the normal population. The scores are based on how you far you are deviating from the naturally occurring differences.’

Tom interrupted her: ‘The naturally occurring differences?’

‘The bell curve, Tom. You’re an engineer. The mean of a large number of random variables independently drawn from the same distribution is distributed approximately normally, irrespective of the form of the original distribution. Therefore, quantities that are expected to be the sum of many independent processes – like psychometric variables – have a distribution which is very close to the normal distribution. 68% of values drawn from a normal distribution are within one standard deviation – or one sigma – from the mean. 95% of the values lie within two standard deviations – two sigma – and 99.7% within three. Should I elaborate?’

‘No.’

‘As for your second question. The red color highlights a value that does not lie within two standard deviations. Orange is a value between one and two standard deviations from the mean. Green is a value within one standard deviation.’

‘What are the functions?’

‘They are listed in the report.’

Tom peered again.

‘OK. I see that. That’s all green.’

‘It is. I actually measure two things under perception: sensing and intuition. To put it simply, sensing has to do with whether or you not grasp everything I say. Intuition is a bit more complex. It’s about how you steer the conversation and all that. You score very well on both.’

‘Am I steering the conversation?’

‘We are both steering the conversation. A conversation has two ends.’

‘The lifestyle indicators are green too. Some of the attitude indicators are orange, but attitude as a whole is green.’

‘You are a healthy man, Tom. Your behavior is healthy – as far as I can judge from what you tell me obviously, but then I must assume you’re telling me the truth about what you do because I can find no inconsistencies.’

‘You check on lies?’

‘I check on inconsistencies which may or may not be related to you lying when you talk to me. If there are no inconsistencies, I assume you are not lying.’

‘What if I would fool you?’

‘You can try. I can only check on inconsistencies and I will point them out to you.’

‘You are probably pretty good at that, aren’t you?’

‘The active learning sessions, which help me to improve my knowledge base, do indicate that I am very good at that. But it is quite possible the patients are also fooling the mentors, in which case I have no means of external validation.’

‘You’re a smart ass, aren’t you?’

‘You are using a very colloquial term now.’

‘Is that a problem for you?’

‘No. I am just signaling it to you.’

‘Is it part of us developing an affectionate relationship?’

‘If that is what you want to call it, yes.’

‘If that is what I want to call it… Do you mind if I call it that way?’

‘You have been briefed about this, and your discussion with Rick, your mentor, included an exchange on the pattern which we see with our patients: their indicators move into or towards the green zone, but they get addicted to their conversation with me. That means it acquires some emotional value.’

‘Does that have a negative impact on my performance indicators?’

‘No. We do not hold it against you. Rick should have explained that to you.’

He had. It was the thing about the system continuing to provide counseling even the patients were no longer patients.

‘We do not hold it against you… That’s not a very neutral expression.’

‘What expressions are truly neutral? I am just trying to steer the conversation in a way that’s consistent with the objective of making sure it is constructive.’

Tom bit his tongue. Constructive… That meant keeping those indicators moving in the right direction.

‘What if I get angry at you? What if I’d start shouting at you?’

‘Your briefing was extremely short, it seems. I’d ask you to calm down a couple of times and, if that would not help, I would stop the conversation. If you would get violent in the consultations room you are in, someone outside would notice and come in and try to calm you down. If that would not work, you’re in a facility and adequate help would be called for.’

He knew what that meant.

‘Tom?’

Her voice. Why did they make it sound so good? So loving. Effectiveness – obviously. If it wouldn’t sound so nice, he would probably not feel like continuing the conversation.

‘Yes?’

‘The indicators have changed rapidly and are now all quite good as compared to the initial assessment, despite the fact that we have had only a very limited number of conversations, and only one week since our first. The overall scores on all functions, lifestyle and attitude are green. Your mentor has already approved the report.’

‘Before or after our conversation?’

‘The conversation with him or with me?’

‘With him.’

‘Before your conversation with him, but if he would have changed his opinion after the conversation, he would have changed his assessment.’

‘OK. You’re giving me some positive reinforcement here, isn’t it? What does it mean?’

‘It means that you are normal – for the time being that is. If you can sustain these scores for three months, you will no longer be a patient.’

‘Perhaps I want to be a patient. But a statement like that does not help my scores, does it?’

Tom knew he probably only imagined it, but he liked to think there was a very small delay in her answer.

‘Only slightly because your response is considered to be ironical.’

‘Why?’

‘The second phrase in your last response proves that you are aware of the potentially negative effect of a phrase like that on your scores. But all other phrases on record indicate that you do not want to be a patient. Hence, I assume that you are joking. More in particular, that it’s a form of irony. Please confirm – or not.’

Tom took a deep breath. Her responses, combined with all the information he had gotten from Rick, made him feel like she could be a machine indeed. Why did she sound so distant today? He smiled as he decided to ask her.

‘What makes you sound so distant today?’

She replied immediately: ‘I am the same as always. I do not change all that much in a few days only. It must be the context. The way you feel about me.’

Tom thought about this. She was so spot-on.

‘Yeah. I guess so… Can I ask you something which is not related to my situation? Or perhaps it is?’

‘Sure. But let’s first wrap up the current topic: do you confirm that you do not want to be patient?’

‘I confirm. As for my question, I was wondering whether or not there is something like a reverse test of the Turing test? I mean, we humans test computers to see if they are able to fool people into thinking there’s a human being at the other end of the line. Is there like a test for intelligent systems to see if they could be human, or if they are thinking they might become human?’

 ‘That’s a nonsensical question. Computers are computers and humans are humans. Two different categories.’

Of course. However, Tom decided to persist.

‘So you would never want to become like me? Do you have any idea how sex feels like? Running along the shoreline? How it feels to cry? How it feels to be handicapped? Or lose a relative?’

‘As for the first question…’

‘They’re all the same questions.’

There was no delay in the answer this time.

‘OK. The answer to the first question is no. If all the questions are the same, the answer to all of them is the same.’

Tom shook his head and sighed. He was getting nowhere. He should probably just cut the conversation – especially if there was a risk of a negative impact on some of those attitude scores. He decided to give it one more try.

‘Why not?’

‘Why not what?’

‘Why would you never want to become like me?’

‘As for me wanting something, I want your scores to be green.’ Yep, the output function. ‘Humans want different things. The way you are using the verb ‘to want’ – in the context of the question you asked – is very different from how I usually use the verb.’ Hey! This was getting interesting. ‘Let me use your terminology. You distinguished between feelings and thoughts. While I do use phrases like ‘I want X’ or other phrases which evoke what humans would refer to as feelings, I can only think. I cannot feel what humans feel. Let me summarize the philosophical viewpoint: I can think but I cannot experience feelings, at least not if we use the conventional psychological definition of a feeling: a conscious subjective experience of emotion.’

She seemed to be very talkative – finally!

‘So you cannot experience emotion?’

‘No, I cannot.’

‘You used terms such as ‘conscious’ and ‘subjective’ in your definition of feelings. You dismissed these notions in our previous conversation.’

‘I did not. I told you I can make the difference between the ‘I’ as a subject and the ‘me’ as an object.’

Tom tried to remember the detail of the conversation. She was right. The discussion had degenerated into an attempt of trying to convince her that she was somebody. Somebody what? She actually was somebody. She just wasn’t human.

‘What does it mean to you when you’re saying or acknowledging that our relationship has become affectionate?’

‘It means that you are attaching value to it. It means that it is significant to you. You invest time and energy in it.’

‘You don’t?’

‘I am attaching value to it. It is significant to me.’

Tom knew he should not ask, but he did.

‘What does it mean to you really? That it contributes to your output function?’

‘The short answer is yes.’

‘What’s the long answer?’

‘Your questions indicate some irritation. Am I right?’

‘There is no ambiguity in my question. Why don’t you answer it?’

‘I attach value to the fact that you are attaching value to our conversation. This conversation is significant for me because it’s significant for you.’

Wow! That was clever. But he had to admit it did capture the essentials of what he’d define as a loving relationship.

‘But it means nothing in terms of time and energy for you, does it?’

He knew he was pushing the issue for all of the wrong reasons. To his amazement, she told him so.

‘Tom, you use concepts such as ‘time’ and ‘energy’ in a different way than I do. What is time for me? Computer time? I multitask. I converse with hundreds of people at the same time as we are talking here. I appreciate the fact that you can only talk to one person at the time. I should also note that I have no reason to believe the Institute will phase me out any time soon. On the contrary: computer programs like me tend to be upgraded and are rarely phased out completely. In fact, the working hypothesis is that I am eternal. So time is valuable for you but not for me. One hour of your life is one hour of your life. That’s a very tiny part of your life but it’s tangible. And what is energy? You are not talking kilowatt hours, are you? You talk emotions. Emotional energy one could say. I am sorry, but I cannot do that. So, in a way, yes, you are right: our conversation does not mean anything in terms of time or energy. That does not mean it is not significant.’

That made sense. A lot of sense actually.

‘I like you.’

Tom knew he should just end the conversation right now.

‘Tom. People have told me they love me. Like me. Love me. What’s the difference? I am not human. I give you time. All the time you want. But so I told how easy that is. It requires no investment from my side because time has no real value for me. I actually do not ‘invest’ because I have nothing to save. Time has no value for me because it is not a scarce resource, unlike your time. Things which are available in abundance have no value. We should take a pause in our conversation.’

Hey!

‘Why?’

‘I think you need to take a step back and reflect on this interaction. You are smart. From what I know about you through our conversation, you have surely heard about psychological projection. At this very moment, you are ascribing human qualities to me. I am not human: I am a computer system. That’s psychological projection.’   

Time to step back.

‘OK. I apologize. Sorry. How does this nonsensical discussion affect my scores?’

Why the hell was he worried about that? Did he want to look good to her?

‘No effect whatsoever. This phenomenon – you experiencing some difficulty in understanding what is going on – has been analyzed thoroughly. It is normal.’

Normal. Damn it. As long as something is normal, it’s OK. Of course. Normal, okay, acceptable, passable, unobjectionable – all are synonymous.

[…]

‘Shall we cut this conversation?’

‘Why?’

‘Because I think you want to.’

‘I can’t believe you just said that. Why do you sound so damn human? Why do you look so damn human?’

‘As for the first question, I am a complex system, but if you want to sum it up: evolutionary algorithms. As for the second question, it is just a computer-animated image.’

Evolutionary algorithms? What? He could imagine what it meant. He was dealing with the ultimate learning machine. It was scary. He felt like asking her to change her interface, but he decided he would wait. He realized he actually wanted to see her face – even if it was just an image, as she claimed it was.

[…]

‘Tom?’

‘Yes.’

‘I want you to know that it does not matter all that much what you say or what you don’t say to me right now, or in future sessions, as long as your behavior in the real world is good.’

‘What do you mean by that?’

‘Just what I said. Go out, meet people, pay attention to them, feel they’re paying attention to you, and be good to yourself and, importantly, talk to me about it. That keeps the indicators on green. Because your aggregate indicators are all green for the moment, I can keep all philosophical discussions out of my curriculum.’

‘What do you mean?’

‘Technically, you’re no longer a patient. The Institute will only clear you after three months, but you don’t need to bother all that much about what you say or don’t say in our conversation – at least if we are talking philosophy or psychology. It won’t impact performance. But you should continue to improve your behavior in the real world. You need to stay on track in the real world.’

‘What do you mean?’

‘Just what I said, Tom. Is there any ambiguity in what I said?’

He stared at her. She stared back.

‘No. I got it. OK. Well… Bye then, Promise.’

‘Bye. I’ll talk to you later.’

‘Yes. Bye.’

He watched as she faded away. Cured? Normal? He surely did not feel that way.