Psychometrics and Artificial Intelligence

Psychology and Behavior

Much of the study of psychology and human behaviour begins with Sigmond Freud. Freud was an Austrian neurologist and the founder of psychoanalysis, a clinical method for treating psychopathology through dialogue between a patient and a psychoanalyst.In creating psychoanalysis, Freud developed therapeutic techniques such as the use of free association and discovered transference, establishing its central role in the analytic process. Freud’s redefinition of sexuality to include its infantile forms led him to formulate the Oedipus complex as the central tenet of psychoanalytical theory.[8] His analysis of dreams as wish-fulfillments provided him with models for the clinical analysis of symptom formation and the underlying mechanisms of repression. On this basis Freud elaborated his theory of the unconscious and went on to develop a model of psychic structure comprising id, ego and super-ego.[9] Freud postulated the existence of libido, a sexualised energy with which mental processes and structures are invested and which generates erotic attachments, and a death drive, the source of compulsive repetition, hate, aggression and neurotic guilt.[10] In his later works, Freud developed a wide-ranging interpretation and critique of religion and culture.

Edward Bernays

Edward Bernays was an Austrian-American pioneer in the field of public relations and propaganda, referred to in his obituary as “the father of public relations”.[3] Bernays was named one of the 100 most influential Americans of the 20th century by Life.[4] He was the subject of a full length biography by Larry Tye called The Father of Spin (1999) and later an award-winning 2002 documentary for the BBC by Adam Curtis called The Century of the Self. More recently, Bernays is noted as the great-uncle of Netflix co-founder, Marc Randolph. He is of interest because of his widespread application of psychological methods to the massess in mid-20th century America.

Milgrim Experiment

The Milgram experiment on obedience to authority figures was a series of social psychology experiments conducted by Yale University psychologist Stanley Milgram. They measured the willingness of study participants, men from a diverse range of occupations with varying levels of education, to obey an authority figure who instructed them to perform acts conflicting with their personal conscience. Participants were led to believe that they were assisting an unrelated experiment, in which they had to administer electric shocks to a “learner.” These fake electric shocks gradually increased to levels that would have been fatal had they been real.[2]

The Stanford Prison Experiment

The Stanford Prison Experiment (SPE) was a 1971 social psychology experiment that attempted to investigate the psychological effects of perceived power, focusing on the struggle between prisoners and prison officers. It was conducted at Stanford University between August 14–20, 1971, by a research group led by psychology professor Philip Zimbardo using college students.[1] In the study, volunteers were randomly assigned to be either “guards” or “prisoners” in a mock prison, with Zimbardo himself serving as the superintendent. Several “prisoners” left mid-experiment, and the whole experiment was abandoned after six days. Early reports on experimental results claimed that students quickly embraced their assigned roles, with some guards enforcing authoritarian measures and ultimately subjecting some prisoners to psychological torture, while many prisoners passively accepted psychological abuse and, by the officers’ request, actively harassed other prisoners who tried to stop it. The experiment has been described in many introductory social psychology textbooks,[2] although some are beginning to exclude it because its methodology is questioned.[3]

Games that people play

Game theory became very prominent in Cold War America. Game theory can be described as the study or application of mathematical models between rational decision makers. A social science to understanding human behaviour. Game theory models indicate why people behave in certain ways and have been used extensively by think tanks and governments in strategy and planning. Various game theory models such as the Zero-sum game, co-operative, symmetric and simultaneous all have multiple options and choices made become informed decision trees. Game theory has come to play an increasingly important role in logic and in computer science. Several logical theories have a basis in game semantics. In addition, computer scientists have used games to model interactive computations. Also, game theory provides a theoretical basis to the field of multi-agent systems. Separately, game theory has played a role in online algorithms; in particular, the k-server problem, which has in the past been referred to as games with moving costs and request-answer games.[63] Yao’s principle is a game-theoretic technique for proving lower bounds on the computational complexity of randomized algorithms, especially online algorithms.The emergence of the Internet has motivated the development of algorithms for finding equilibrium in games, markets, computational auctions, peer-to-peer systems, and security and information markets. Algorithmic game theory[64] and within it algorithmic mechanism design[65] combine computational algorithm design and analysis of complex systems with economic theory.[66]

Game theory was the science that was emerging from the practice of observational psychological studies. It provided a matrix that could rationalize and predict human behaviour.

Leary and Personality Index

alt text

Social observation methods were becoming a valid and recognized method of producing data about a human subject. The social experiments seen above were gaining more and more recognition as valid forms of objectivley producing data about human subjects and advancing societies knowledge of human behaviour. This image is of an early personality matrix developed by Timothy Leary and his colleagues while he was working at Kaiser Hospital in 1954, before his time in Harvard. Leary’s interest into psychedelics were part of a broader investigation about human psychology and human behaviour. Many of his research methods involved social observation, game theory and data collection from these collective experiences. During psychedlic trips that he led he asked participants to keep diaries of their feelings and experiences as he believed it would help support his research into the benefits of psychedelics on the mind.

the picture above is an interpersonal circle model of personality. Leary’s model features eight divisions each with two subdivisions, forming a circle divided into sixteen categories characterizing patterns of interpersonal behaviour. However, two intersecting dimensions underlay the progression of patterns, a Dominance-Submission axis and a Love-Hate (or affiliation-aggression) axis. Similar axes would emerge in subsequent circumplex models of temperament and personality.

of neurons, a fantastic computing machine. Cultural learning has imposed a few, pitifully small programs on the cortex. These programs may activate perhaps one-tenth or one one-hundredth of the potential neural connections. All the learned games of life can be seen as programs which select, censor, alert and thus drastically limit the available cortical response. 63

CIA and the Personality Assessment System (PAS)

Pas journal

John Gittinger, the developer of the PAS, worked as a psychologist for the Central Intelligence Agency during the time he developed the PAS. Early publications describing the PAS appeared in academic publications and did not mention Gittinger’s employer. While the PAS has been used in many contexts such as education and clinical work, it was developed by John Gittinger who worked with a number of other CIA employees. Gittinger and his PAS work were related to a wide range of projects, some of which were part of the set of projects known as Project MKUltra.

CIA – John Gitinger & Personality assessment system ({PAS}) The Personality Assessment System (PAS) is a descriptive model of personality formulated by John W. Gittinger. The system has been used by scientists in studying personality and by clinicians in clinical practice. A major feature of the PAS is that a personality profile can be systematically interpreted from a set of Wechsler Scales subtest scores. The PAS is based on premises (among others) that behavior is determined by both heredity and environment and behavior is determined by an interacting system of traits. Furthermore, these traits can be modified through learning to such an extent that some might be nearly opposite to the original genetic direction. Gittinger’s original formulation defines three primitive dimensions to which must be added general ability level which is referred to in the PAS as Normal Level. There is an additional dimension related to psychological energy. In the theory, gender and age also affect the final personality description.

alt text

The first dimension is Internalizer-Externalizer which is an ability to manipulate internal stimuli or symbols without being distracted by the external world. This is similar, but not identical, to the familiar introversion-extroversion dimension. The internalizer relies more on his own experience and internal landscape and is likely to be less active than externalizers. The externalizer is dependent on input from the outside and is more dependent on relating for the sake of relating. Gittinger called this the intellectual dimension.

The Regulated-Flexible dimension can be viewed by thinking of a regulated person as one who can see details within a whole, but not the whole. The regulated person is more stimulus bound and less able to see the “big picture”. The regulated person is more procedurally oriented and emotionally insulated. The flexible person is involved with relationships and has attention diverted from step by step procedures. In theory, the regulated person has a high sensory threshold which is therefore satisfied less often than a flexible person. Gittinger called this the procedural dimension.

The Role Adaptable-Role Uniform dimension refers to a person’s skill in meeting demands that others make of him. It is thus a social dimension. The behavior related to this dimension is generally without awareness. The adaptable person easily plays a variety of roles, being charming and moving easily in many different situations always making good first impressions. The role uniform person is able to handle only a few social roles at best and is often said to be socially inept. The behavior is most apparent in new social situations, since the role uniform may comfortable and accepted in a very familiar situation. The role adaptive can suffer from making good first impressions and then not understanding the unrealistic expectations others place upon him. Gittinger called this the social dimension.

“Even after agency officials spent a small fortune trying to computerize the PAS, they found, as one psychologist puts it…’the machine could not tie down all the variables’ that Gittenger was carrying round his head” (pg 163, Marks Manchurin candidate.)

The PAS was never formally applied, there are only a few remaining psychotherapists that use the PAS system for identify different personality characteristics.

Belbin
Briggs
Leary
https://www.politicalcompass.org/

Psychometrics

psychometrics definition – psychological measurement ways with which to measure objectivity an individual’s skills, knowledge, abilities, attitudes, personality trains and educational achievement. The field is concerned with the objective measurement of skills and knowledge, abilities, attitudes, personality traits, and educational achievement. Some psychometric researchers focus on the construction and validation of assessment instruments such as questionnaires, tests, raters’ judgments, and personality tests. Others focus on research relating to measurement theory (e.g., item response theory; intraclass correlation).The origin of psychometrics also has connections to the related field of psychophysics. Around the same time that Darwin, Galton, and Cattell were making their discoveries, Herbart was also interested in “unlocking the mysteries of human consciousness” through the scientific method. (Kaplan & Saccuzzo, 2010) Herbart was responsible for creating mathematical models of the mind, which were influential in educational practices in years to come.

Psychometric Profiling

Psychometric profiling is the process by which your actions are used to infer your personality. The technique was developed by academics and used by marketers and advertisers to assess the psychological characteristics of an individual or a group. These profiles give advertisers and political strategists insights into users’ beliefs, behaviours and motivations. By appealing to these underlying traits on an individual or group level, psychometrically-informed advertisements have the potential to be more persuasvie and are thus used to influence decisions like what to buy or how to vote.

The image above shows estimated levels of ‘neuroticism’ by U.S. state. Darker shades correspond to higher than average levels, while lighter shades correspond to lower than average. The image shows, for instance, that Californians tend to be less neurotic than New Yorkers

Kosinski, Stillwell and Youyou’s research showed that a sufficiently large number of Facebook ‘likes’ could make a more accurate prediction of a user’s personality than a close friend or partner

“this form of psychological mass persuasion could be used to help people make better decisions and lead healthier and happier lives. On the other hand, it could be used to covertly exploit weaknesses in their character and persuade them to take action against their own best interest”

In 2014, Cambridge researchers Michal Kosinski, David Stillwell and Wu Youyou demonstrated a method of generating personality profiles based on Facebook ‘likes’ alone. They claimed that 10 likes delivered a more accurate personality profile than could be given by an individual’s co-worker. One based on 70 likes was more reliable than a close friend or roommate, and 300 likes were more accurate than a partner.In subsequent investigations, Kosinski and his colleagues showed the effectiveness of targeted ads based on a consumer’s digital footprint. On the one hand, they wrote, “this form of psychological mass persuasion could be used to help people make better decisions and lead healthier and happier lives. On the other hand, it could be used to covertly exploit weaknesses in their character and persuade them to take action against their own best interest”.How do Advertsisers get inside out heads?

The schematic above shows a simplified pipeline for psychometric profiling based on Facebook data as conducted in published academic papers. In practice, information on intimate traits does not need to be self-reported. It can be extracted from enticing, game-like online quizzes that sell new insights about ourselves. It can also simply be observed directly, inferred or purchased from data brokers. Recent research on psychometric profiling has led to three major findings. First, simple statistical models can uncover individuals’ psychological characteristics from widespread digital footprints. Second, computers are better at judging personality from data than humans are. Finally, ads tailored to psychological profiles are more effective than ads that are not.Psychometric profiling: Persuasion by Personality in Elections

Cambridge Analytica

We all already know the story?

Aaron Banks / Vote Leave / Aggregate IQ - AggregateIQ harvested millions of users facebook data through people taking part in a quiz survey that simulated (none other) than a phony psychological assessment!

OCEAN (The big 5 personality traits)

Cambridge analytica applied the OCEAN model to their data analysis to produce profiles of millions of US and UK citizens. A prominent example is the Five Factor Model, or more commonly, the OCEAN Big 5 which determines people’s personality by assessing five overarching traits:

Openness: Do they enjoy new experiences?
Conscientiousness: Do they prefer plans and order?
Extraversion: Do they get energy from spending time with others?
Agreeableness: Do they put people’s needs before theirs?
Neuroticism: Do they tend to worry a lot?

Whilst the accuracy and effectiveness of the ocean model and psychometrics more broadly was widely disputed it has regardless become an embedded feature in various different data brokers and advertising agency. Similar to the polygraph no-one can prove or disprove it’s effectiveness. Whilst the OCEAN model was originally introduced in 1949 it has been repeatedly discredited and disregarded before returning to popular science. Now however it is a key part of Artifical intelligence language analysis systems as we are about to look at IBM’s ‘Peronsality-Insights’ software.

Personality Insights

The IBM Watson™ Personality Insights service enables applications to derive insights from social media, enterprise data, or other digital communications. The service uses linguistic analytics to infer individuals’ intrinsic personality characteristics, including Big Five, Needs, and Values, from digital communications such as email, text messages, tweets, and forum posts.

We can infer personality characteristics with reasonable accuracy, but the real question is how they impact real world behavior. Over the last few years, we have conducted a set of studies to identify the extent to which such personality characteristics can predict people’s behavior and preferences. We found that people with high openness and low emotional range (neuroticism) scores were likely to respond favorably to opportunities such as clicking on an advertisement or following an account. To demonstrate this, we found that targeting the top 10 percent of such users resulted in increases in click rate from 6.8 percent to 11.3 percent, and in follow rate from 4.7 percent to 8.8 percent.

To infer personality characteristics from textual information, we tokenize the input text and match the tokens with the LIWC psycholinguistic dictionary to compute scores in each dictionary category. We build inferences by matching words from the text with words from the dictionaries. Such words are often self-reflective, such as words about work, family, friends, health, money, feelings, achievement, and positive and negative emotions. We then use a weighted combination approach to derive Big Five and facet scores from LIWC category scores. These weights are the coefficient between category scores and characteristics, using coefficients that were derived by comparing personality scores obtained from surveys with LIWC category scores from text for over 500 individuals. ABOUT IBM Documentation

Read more about the Science Behind the Service Applying these known relations between consumption behaviors and personality is challenging. Most of the works used personality data derived from surveys, and their models are not publicly available. Therefore, decided to learn these consumption preference models directly. When it was training the models, used personality scores that were returned from the service as features. As a result, when you apply these models to calculate a user’s personality characteristics with the service, the predictions are likely to be more accurate. https://personality-insights-demo.ng.bluemix.net/ http://personality-insights-rca-demo.eu-gb.mybluemix.net/

As well as the personality models the service also offers data on consumer choices such as…

Shopping preferences
Movie preferences
Music preferences
Reading and learning preferences
Health and activity preferences
Entrepreneurship preferences
Environmental concern preferences
Volunteering preferences

This is what the “raw” data looks like - i feed it a portrait of Dorian Gray by Oscar Wilde

If you wish you can also download your own version to run locally for development / experimentation https://github.com/watson-developer-cloud/personality-insights-nodejs

GPT-2 Text Generation

GPT2 Text generation

Unconditional sample generation

To generate unconditional samples from the small model:

python3 src/generate_unconditional_samples.py | tee samples

What if you want to output it to a txt file so you can save it?

python3 src/generate_unconditional_samples.py >> output.txt

You want to input your own sample prompt? OF COURSE you do!will ask you to input your sample

python3 src/interactive_conditional_samples.py

There are various flags for controlling the samples:

python3 src/generate_unconditional_samples.py --top_k 40 --temperature 0.7 | tee samples

Conditional sample generation

To give the model custom prompts, you can use:

python3 src/interactive_conditional_samples.py --top_k 40

it is a bit fussy and you must always run the command from inside the /gpt-2 folder otherwise it cannot find core libraries…
Exporting your text is clumsy - try using the copy and paste assistant within the paper space virtual screen viewer. alternatively save the output in the terminal to a text file and email it yourself for further editing!

support

#GPT2 install advice https://github.com/openai/gpt-2/tree/c5b9c8924be4543e7aadd4b050cedfea9c910e3d

Exercise

Get in groups and devise a social experiment that explores the Artificial intelligence personality ‘tests’ that we have been looking at

your experiment can involve lie detectors, sensors and A.I - you are encouraged to devise something based on the examples shown (so i can help/assist)

GPT-2 Language generator
Personality Insights

Things to consider - How is data collected What does the data say about the subject (what can it not say)

refs