Sunday, April 3, 2016

How to get users to read instructions

In our experiments we are often giving our users some instructions as to how they should fill out given questionnaire. It can be as simple as "answer as truthfully as possible" to more complex situations like "imagine that you are at work interview and you are filling out questionnaire as part of it".

Our goal with this types of instructions is to put users into as real conditions as possible and make sure, that they will answer questions in the way they would answer them in real life in given situation and if they personally had given motivation.

During our pilot experiments we found out, that users were often not reading instructions or have not understand them well.

To make sure that the users will read the instructions we tried several modifications, from putting instructions in different forms (paper/display), moving they position on display, using different colors, different fonts etc. We achieved some improvements, but it was never close to 100 percent.

Another problem was understanding of instructions. Even if user have read instructions, often he would not understand them correctly. We have tried several modification of instructions that had the same meaning but were put in different words. And again, we had some improvements, but never close to 100 percent success.

Then at our weekly student seminar of PeWe group at FIIT STU I have put this problem forth to my colleagues and after discussion that lasted several minutes there came this simple solution, to ask user after instructions question about the instructions and not let them pass unless they answer correctly.

This was simple but efficient idea. After several times we arrived at simple similar process using question after instructions. After user has read instructions, we ask him to describe how he had understand them in his own words. We found out that after this simple question users had almost every time read and correctly understand instructions.

We think, that making user answer question after he has read the instructions has made him read instruction, and the way he has to answer it, in his own words, has made him think about the instruction and understand it properly.

Friday, July 24, 2015

What is deception?

If falsehood, like truth, had only one face, we would be in better shape. For we would take as certain the opposite of what the liar said. But the reverse of truth has a hundred thousand shapes and a limitless field.—Montaigne, Essays


What is deception

There are few definitions of deception. According to Merriam-Webster dictionary it is "an act or statement intended to make people believe something that is not true" [1]. According to Ekman [2], the deception involves acting in such a way which leads another person to believe something, that you, yourself, do not believe to be true.

These definitions are broad. For example, if you think withholding information is not deception, you are mistaken. If you are acting in front of someone so, that you expect to present yourself in a certain way, you are deceiving that person. For example, if you like a girl of your friend, but you are not acting that way, you are deceiving your friend and the girl.


Ethics of deception

This bring up the question, Is deception acceptable? It certainly is. At least from the perspective of science, it is even necessary. According to Taylor & Shepperd [5], 81 percent of published psychological studies use deception. And it is only logical. If you are studying some psychological phenomenon, you could influence the result of it if you have told the participants about what are you trying to achieve. So deception becomes necessary.

Or in double blinded tests used mainly in medical research, deception is also necessary. But you cannot lie to subjects in medical experiments. It is against to law and also it is considered unethical [6]. So if you cannot lie to a subject telling them you are giving them medicament if in reality you are not, how do you do double blinded test? You deceive the subject. You tell them that they have 50 percent chance that they are receiving the true medicament and 50 percent chance that they are receiving placebo. You have not lie to them, but you have deceived them, because you have not told them the whole truth.

But is deception acceptable in social interactions? Some forms of deceptions are against the law such as frauds or tax pay evasion, so clearly not all forms of deceptions are acceptable, just as in scientific research. But it is also obvious, that deception is useful and acceptable in some social interactions. If your sick partner asks you how do they look, you are probably going to lie to them or at leas deceive them and you would not tell them how they really look to you. Great usefulness of deception is shown in the movie The Invention of Lying, which is set in a society where everybody is telling the truth all the time. If you ever tried to imagine what would such world look like, you certainly have to watch the movie or at least the trailer.

For example, almost all the movies except from documentaries are deception. The actors are pretending to be someone they are not and they know they are not them. So they use deception, which they could not if they were not able to deceive.


Subjectivity of deception

This brings interesting question regarding Ekman [2] definition, which suggest, that if you are deceiving somebody, you have to act or say something that you, yourself, do not believe to be true. So if you personally believe that you are Henry V., would you be able in a society in which no one can lie play Henry V. in a movie? You are not deceiving anyone, because you do believe that you are the Henry V. But nobody else in such society would be able to differentiate.

The reason we consider persons that believe they are someone that they are obviously not such as historical or fictional characters as mentally ill is result of the fact, that we are able to differentiate between the reality that is and the reality someone has made up. One of the best scenes in the movie The Invention of Lying is this one.

If you are not able to differentiate between reality and deception, you are able to believe anything somebody tells you. So deceptions is clearly useful at least as protection from situations when someone else is trying to deceive us and to tell if somebody is deceiving us or simply has mental disorder.

You probably would not lock up in mental institution Johnny Depp for playing Jack Sparrow in a movie, but if he was going around the streets preaching that he in fact is Jack Sparrow for some time, you would quickly reconsider.


Types of deception

So we have already discussed usefulness of deception and differentiate what is deception and what is not. But as we mentioned, there is not just one form of deception.

There is a moment in the movie Insurgent where Tobias ask the leader of Candor faction, which is know for its truthfulness (I am paraphrasing)

- "I've heard you use truth serum in your initiation process"
- "We do not discus such matters with outsiders."
- "That's not an honest answer."
- "Evasion of question is not the same as lying."

And that is the truth. But lying is not the same as deception. Lying is a part of what deception means. It is the most know and recognizable part, but it is not all. Few examples could be [7]
  1. Lies: making up information or giving information that is the opposite or very different from the truth.
  2. Equivocations: making an indirect, ambiguous, or contradictory statement.
  3. Concealments: omitting information that is important or relevant to the given context, or engaging in behavior that helps hide relevant information.
  4. Exaggerations: overstatement or stretching the truth to a degree.
  5. Understatements: minimization or downplaying aspects of the truth.
but the categorization could be based on many different factors considering circumstances. For example considering on-line interactions "there are various types of possible deception such as category deception (gender switching), attractiveness deception, or identity concealment" [8].



Deception requires the person that is deceiving to have contradictory knowledge about his actions or statements, so someone who is telling obvious lies but believes them to be true is in fact not deceiving us. Also, lies are not the only form of deception and deception is any form of "not truth telling" including withholding information. Also deception is useful in scientific research and in some social interactions, but also unacceptable and punished in some cases. Knowledge of deception is also useful tool for its recognition.

So live happily and be truthful to yourself.




[2] Ekman, P. (1985). Telling lies. New York : W. W. Norton.


[4] Ortmann, Andreas, and Ralph Hertwig. "Is deception acceptable?." (1997): 746.

[5] Taylor, Kevin M., and James A. Shepperd. "Probing suspicion among participants in deception research." American Psychologist 51.8 (1996): 886.

[6] Wendler, Dave. "Deception in medical and behavioral research: is it ever acceptable?." The Milbank Quarterly (1996): 87-114.


[8] Utz, Sonja. "Types of deception and underlying motivation what people think." Social Science Computer Review 23.1 (2005): 49-56.

Friday, April 24, 2015

Experiment on deception detection

Experiment description

For our experiment we used Big Five personality questionnaire. This questionnaire was implemented as online website. Users were 8 university students from Faculty of Informatics and Information Technology of Slovak University of Technology. They consisted from people of age from 19 – 26 and 7 males and 1 female. They were given 2 task to fulfill, while being monitored by eye-tracker. Measures like mouse movement and mouse-clicks were also monitored. All users were aware of this.

The tasks given to them were as follows:
  1. Imagine, that you are trying to get new job and this questionnaire is part of the job interview. Answer following question such, that according to your opinion you would be best employee. 
  2. Please, answer following questions as truthfully as possible. 
Half of the participants were given these tasks in this order, other half were given these tasks in switched order. After reading the instruction, they were presented with Big Five questionnaire consisting of 60 questions. Users were seeing the questions as shown in Figure 1.

Figure 1.: Shows how the users have seen the questions and the answers.


We found out, that users were more likely to answer the questions in most positive way if they were trying to look as good employee as possible (Figure 2.).

Figure 2.: Distribution of answers if answering honest (1) or dishonest (2)

Results has also shown, that the order in which the users were answering the questionnaires is influencing the results. If they were instructed to answer honest first, they were more likely to answer most positive most times and less times most negative. If the instructions were in reverse order, they were more likely to answer more times most negative and less time most positive (Figure 3.).

Figure 3.: Distribution of answers if answering honestly first (1) or dishonestly first (2)

We also observed, that the number of fixations, visits and the duration of fixation was much higher on most negative answer, if the users were answering honest in contrast if they were instructed to answer dishonest (Figure 4.).

Figure 4.: Distribution of fixations if answering honest (1) or dishonest (2)

This was mostly influenced by the group, which were instructed to first answer the questionnaire dishonestly (Figure 3.). This shows, that users were evaluating themselves more harshly, if they were first describing with they answers ideal employee.


There are certain trends that can serve as indicators, that users are answering dishonestly. Most significant as far are the number of most positive answers and low number of most negative answers. The experiment also has shown the importance of instruction given, because these can influence the results. Interesting effect of more harsh evaluation users of themselves if they were first describing ideal employee could be explore further, if we setup experiment, so that users would describe worst possible employee instead of ideal employee, and watch if the effect would be reverse, so that users would describe themselves than more positively.

Sunday, March 8, 2015

Don't lie to my eyes (II.)

In my last post, I was talking about pupil of the eye. When you are measuring the pupil, you are probably using an eye tracker. And the eye-tracker provides us with more data than just pupil diameter. So today, we are going to look at the other data and what can be done with it.

Fixation and saccades

The measures the most eye-trackers can give as are gaze location, blinking frequency, time of the fixation and pupil dilation. This information can help us understand what information people acquire and where their focus is. This can be done using the measurement of fixations and saccaddes, [1]

Saccade is the time when your eyes are moving and it lasts about 15 - 40 ms. During this time, you are acquiring no information. Fixation is the time when your eyes stop and focus on a point. This is when your brain is processing the visual data. In silent reading fixation duration is on average 225 ms but may differentiate from 100 ms to 500 ms depending on the text. [2]

Cues to deception

When detecting deceptions, from the physiological signals that were tested in most studies, these eighteen were most successful; response length, details, response latency, rate of speaking, illustrators, eye contact, non-ah speech disturbances, silent pauses, filled pauses, posture shifts, hand movements, foot or leg movements, smiling (undifferentiated), nervous, pitch, blinking, self-fidgeting, and fidgeting (undifferentiated). [3]

Blinking can be evidence for deception in a way, that increased blinking can indicate anxiety or arousal, decreased blinking can on the other hand indicate greater cognitive effort. In non-interactive context liars blink significantly more than truth-tellers, in interactive context there was little difference. [3,4]

Gaze of the eyes can too be a clue for deception. Liars maintain less eye-contact with their interaction partner. If people are motivated to succeed in lying, they are more likely to decrease eye-contact. However if there is no special motivation, this effect is not present. [3,4]

Deception detection

When we are looking on the order of the fixations using saccades between the fixations, we can reconstruct how people conducted their task. This can tell us if people are trying to deceive us. In an experiment regarding personality evaluation, if people were trying to look good, they were more likely to first look at the more outermost answers within multi-choice scale question. On the opposite, when answering truthfully, the first gaze was focused more often on the center answers. [2]

Response time can be also obtained using eye-tracker. Eye-tracker can tell you when the subject have read the question and how long took him to answer it, you just have to log the time of the answer. This can too be helpful when detecting deception, because lying people are more likely to answer quicker or slower compared to telling truth, based on the task formulation. [2]

If lying is cognitively more complex than telling the truth, the number of fixations is expected to be higher while lying, because increase in fixation indicates increased cognitive load. [2]


There is no complex research or theory regarding detecting faking results in test, so we have to use methods and metrics created in research focused on general deception and lying. [2]

Gaze behavior and blinking are not so easily classified as pupil dilation and depend mainly on social scenario and experiment context. [4]

For gaze-contingent experiments, it is suggested that the experiment should be displayed on large high-refreshing (85-100Hz) CRT monitors instead of LCD panels. Due to physical limitations of liquid crystals, most LCD panels have a refresh rate of 60-75Hz. This would cause visual delay in gaze-contingent experiments. [1]


Response times, blinking and gaze detection can be useful clues for deception detection. From this measures, blinking was shown to be most successful in non-interactive measurements [4] so can be most useful for our project regarding online questionnaires.

Gaze detection in this context is not so useful, but based on the experiment can provide useful clues as the order of the fixations [2].

Response time can too be useful clue, but we have to be able to determine if the task is more or less cognitively demanding than telling the truth. [2]


[1] J. Wang, “Pupil dilation and eye tracking,” in A handbook of process tracing methods for decision research: A critical review and user’s guide, 2011, pp. 185–204.

[2] E. a. J. van Hooft and M. P. Born, “Intentional response distortion on personality tests: Using eye-tracking to understand response processes when faking.,” J. Appl. Psychol., vol. 97, no. 2, pp. 301–316, 2012.

[3] B. M. DePaulo, J. J. Lindsay, B. E. Malone, L. Muhlenbruck, K. Charlton, and H. Cooper, “Cues to deception.,” Psychol. Bull., vol. 129, no. 1, pp. 74–118, 2003.

[4] W. Steptoe, a Steed, a Rovira, and J. Rae, “Lie Tracking: Social Presence, Truth and Deception in Avatar-Mediated Telecommunication,” pp. 1039–1048, 2010.

Saturday, February 21, 2015

Don't lie to my eyes (I.)

“Why do almost all people tell the truth in ordinary everyday life? —Certainly not
because a god has forbidden them to lie. The reason is, firstly because it is easier; for lying demands invention, dissimulation and a good memory.”

– Friedrich Nietzsche, Human, All Too Human, II.54, 1878/1996

I have started this blog as my log to my master thesis. I was aiming to do some work on artificial intelligence. As it worked out, I will be working on method which should detect if people are lying when filling out online questionnaires.

But I'm still going to use this blog. The reason is that this topic is closely related to evolution and even to artificial intelligence.

The topic is still close to the idea of this blog. God-like properties, like knowing what you think, can be obtain via science and technology.

If you want to tell if someone is lying, you have to look at his physiological responses. These physiological responses have evolved over millennia and are often really hard or impossible to control by our conscious mind.

Today I am going to write about eyes. More precisely about pupil of the eye. Because your eyes can tell me if you are lying.

When you want to use pupil diameter as indicator if people are lying, you are basically trying to measure their cognitive load and emotional response.

There is still no complex theory of how our mind work, so there is lot of guess work and space for error, but if you are careful and know what you are looking for, you can get right now up to 80 percent accuracy in telling if people are lying.

Cognitive load

There are three hypothesis when it comes to cognitive load and lying. First assumes, that lying is cognitive less demanding process and so the responses to questions will be quicker and physiological measurements more subtle.

Second hypothesis assumes exact opposite. According to it lying is cognitively more demanding than telling the truth and therefore you can detect stronger physiological responses and the answers to questions will take more time.

Third hypothesis assumes, that it depends on the context. If the questions are about your personality and you want to look socially desirable, you will respond quick and will simply semantically evaluate answers and choose the most appealing.

However if you are asked different questions or you are ask the same questions in different conditions, it will be more cognitively demanding than telling the truth.

Based on the literature I have read, I will go with the third option.

Pupil dilation and what can screw your measurement

Pupil dilation is change in pupil diameter. Dilation occurs around 2-7 seconds after emotional stimuli is presented and is faster for stronger stimuli. If you want to use it in your study, you have to take into account several factors.

First you have to eliminate light disturbance. If you cannot prevent light change by controlling the environment, you can use data filters to separate the data. The pupil response to light is either rapid constriction or slow dilation, while cognitive processing triggers small but rapid increases in pupil size.

However that is not enough. Pupil dilation can be triggered by visual stimuli and even noises and that can disturb your measurement. So you have to ensure that your subjects are separated as much as possible from other stimuli than the task you have given them.

Pupil dilation can also be triggered by cognitive load, stress or even temperature. When you are using your short term memory, pupil dilate. It is dilated during the whole process of solving math problems or other cognitively demanding tasks and dilation stops when the problem is solved.

Even sound can trigger pupil dilation. Larger pupil dilation is shown when you listen to affect sounds compared to neutral sounds. The same can be told about visual stimuli. The more emotional the stimuli is, the more your pupil dilates.

Your pupil also dilates if you are waiting for an answer to even trivia question, it is enough that you are interested in the answer.

It does not matter if the emotion is positive or negative, response of the pupil is very similar.

Your pupil dilates when you are in pain, and it dilates more if you are in more pain and it also dilates if you are aroused.

Pupil dilation and deception

First study about effect of pupil dilation on deception detection comes from 1943 [2]. Since than, technology for pupil measurement has improved rapidly, our knowledge how to use it not so much.

Studies shown that pupil dilates when you are telling lies, even if you are sending deceptive messages. And the more deceptive you are trying to be, the more your pupil dilates.

Most studies shown that lying is cognitively more demanding than telling the truth. Some studies even shown, that if you tell the people that you can detect if they are lying, you are more likely to observe the signals that can tell you if they are lying.

Other however shown that lying is less cognitively demanding. Therefore I go with the third hypothesis of cognitive load.


Pupil dilation can be good indicator for cognitive load or emotional triggers, but on its own cannot provide all necessary information.

It is triggered by too much stimuli and therefore you have to create your experiment very carefully and eliminate all possible external stimuli outside of your task.

Also you shuld use other meassurement like tracking the gaze of the eyes or galvanic skin response. About that in future posts.


[1] Beatty, J. (1982). Phasic not tonic pupillary responses vary with auditory vigilance performance. Psychophysiology, 19(2), 167–172. doi:10.1037/0033-2909.91.2.276

[2] Berrien, F. K. & Huntington, G. H. (1943). An exploratory study of pupillary responses during deception. Journal of Experimental Psychology, 32(5), 443-449.

[3] Bradley, M. T., & Janisse, M. P. (1981). Accuracy demonstrations, threat, and the detection of deception: cardiovascular, electrodermal, and pupillary measures. Psychophysiology, 18(1), 307–315. doi:10.1111/j.1469-8986.1981.tb03040.x

[4] Einhäuser, W., Koch, C., & Carter, O. L. (2010). Pupil dilation betrays the timing of decisions. Frontiers in Human Neuroscience, 4(February), 18. doi:10.3389/fnhum.2010.00018

[5] Krafčíková, M. (2014). Úmyselné skresľkovanie odpovedí pri osobnostných dotaznákoch - možnosti využitia sledovania očí. Univerzita Komenského v Bratislava.

[6] Lubow, R. E., & Fein, O. (1996). Pupillary size in response to a visual guilty knowledge test: New technique for the detection of deception. Journal of Experimental Psychology: Applied, 2(2), 164–177. doi:10.1037/1076-898X.2.2.164

[7] Palinko, O., Kun, A. L., Shyrokov, A., & Heeman, P. (2010). Estimating cognitive load using remote eye tracking in a driving simulator. Proceedings of the 2010 Symposium on Eye-Tracking Research & Applications - ETRA ’10, 141. doi:10.1145/1743666.1743701

[8] Van Hooft, E. a. J., & Born, M. P. (2012). Intentional response distortion on personality tests: Using eye-tracking to understand response processes when faking. Journal of Applied Psychology, 97(2), 301–316. doi:10.1037/a0025711

[9] Wang, J. T., Spezio, M., & Camerer, C. F. (2010). Pinocchio ’ s Pupil : Using Eyetracking and Pupil Dilation To Understand Truth-telling and Deception in Games. The American Economic Review, 3, 984–1007. doi:10.1257/aer.100.3.984

[10] Webb, A. K., Honts, C. R., Kircher, J. C., Bernhardt, P., & Cook, A. E. (2009). Effectiveness of pupil diameter in a probable-lie comparison question test for deception. Legal and Criminological Psychology, 14, 279–292. doi:10.1348/135532508X398602

Sunday, September 28, 2014

Let there be blog

I am materialist. Even if this word is nowadays generally used as an insult, I believe that it is the clearest and most truest description of how the universe works.

What does it mean? It means that all that we can see is created by matter (I know about dark matter and dark energy, but in principle it is all stuff or principle that can be described). And because living creatures are made from matter, there is nothing more to them. No soul, no homunculus, nothing. All the life is just manifestation of the universe.

It is not my dogma. I do not believe it because I want to. It is just how the universe seems. There is lot of evidence backing up this idea and non contrary, except of some metaphysical concepts. It would be exciting to find out contrary, but it does not seem probable.

I am studying computer science and my focus is on artificial intelligence. My bachelor thesis was about evolutionary algorithms, team project I am currently working on is about creating artificial football (soccer for my friends from USA)  players. And I hope that my final thesis for my master degree will be also about artificial intelligence.

So I decided to start a blog about it. Let's be god sounds maybe bit too bold, but it is god with lowercase g. This blog will be about creation. Not the creation from holy books, which in all cases is just some fairy-tail, but real creation, that humans are capable of.

We can create our own universes in books, movies and now in computer games. These computer generated worlds seems real, but most of them do not have real artificial intelligence, which is learning, evolving and understands what you are doing.

But we are really close. All of these things we can do very well separately and now there is huge effort to create systems like Watson, that combines these things.

I will try on this blog to tell you the story of my creation and study and I hope that you will enjoy this journey and learn something too. I am currently reading Origin of Species from Charles Darwin and I think that this part is really suiting for this opening post.

He who believes that each equine species was independently created, will, I presume, assert that each species has been created with a tendency to vary, both under nature and under domestication, in this particular manner, so as often to become striped like other species of the genus; and that each has been created with a strong tendency, when crossed with species inhabiting distant quarters of the world, to produce hybrids resembling in their stripes, not their own parents, but other species of the genus. 

To admit this view is, as it seems to me, to reject a real for an unreal, or at least for an unknown, cause. It makes the works of God a mere mockery and deception; I would almost as soon believe with the old and ignorant cosmogonists, that fossil shells had never lived, but had been created in stone so as to mock the shells now living on the sea-shore.