Read on Twitter
This was micro-popular enough that it made its way into my life - I wanted to explain how it works because I think this will become more common as a method https://twitter.com/margrev/status/1372978689328091137
The tool used is called the “Evaluative Lexicon” - this comes in a checklist form of 42 words, and a software form of around 1500 words.
Premise 1. Words have particular levels of emotional expression inherent to them, and words contribute to the overall emotionality of a text in a meaningful way independent from usage and context.
Premise 2. This “emotionality” level of words can be determined by asking 132 mechanical turkers how emotional each of 94 particular words is - not in any context or contexts, just for each word in general.
Example: “nice” is a much more emotional word than “dangerous,” based on this method, again regardless of usage or context.
Premise 3. Even though the responses usually have standard deviations of at least two points on a 0-9 point scale, these averages are somehow the “true” value for each words’ emotionality.
Premise 4. Once you have values for 94 words from 132 mechanical turkers, you can give hundreds more mechanical turkers sets of a few hundred words to rate on emotionality.
Premise 5. You will know they are good subjects if they correlate strongly with the ratings of the 132 original turkers on the original 94 words. You can simply exclude ratings for subjects that don’t correlate with these.
Premise 6. People have a particular level of a real property “emotionality” at any given point.
Premise 7. Use of words in texts such as reviews conveys the actual emotionality of the writer, rather than being a stylistic choice or influenced by a large number of factors.
Premise 7. Use of words in texts such as reviews conveys the actual emotionality of the writer, rather than being a stylistic choice or influenced by a large number of factors.
Premise 8. Therefore, texts can be analyzed for “emotionality” by comparing the context-free ratings of mechanical turkers on the “emotionality” of around 1500 words, and the average value of these words gives the true emotionality of the writer of the text.
Premise 9. Differences in “emotionality” as small as one tenth or one twentieth of a point on a 0-9 point scale (in which the individual items have standard deviations up to almost three points) are real and valid on large sample sizes of word observations.
Premise 10. Differences as small as one tenth or one twentieth of a point on a 0-9 point noisy scale are sufficient to make the generalization that “expertise causes consumers to become emotionally numb.”
Bonus: When turkers are asked to describe their reactions to photos by choosing four words from a list of 42 emotion words “validated’ by 132 other mechanical turkers, the average of the words chosen reveals their actual emotionality.
one last thing - if you thought preregistration and replication would save us, this one describes its own experiments as replications of each other ten times and describes itself as “pre-registered” five times
