Philip E. Tetlock
   HOME

TheInfoList



OR:

Philip E. Tetlock (born 1954) is a Canadian-American political science writer, and is currently the Annenberg University Professor at the
University of Pennsylvania The University of Pennsylvania (also known as Penn or UPenn) is a private research university in Philadelphia. It is the fourth-oldest institution of higher education in the United States and is ranked among the highest-regarded universitie ...
, where he is cross-appointed at the
Wharton School The Wharton School of the University of Pennsylvania ( ; also known as Wharton Business School, the Wharton School, Penn Wharton, and Wharton) is the business school of the University of Pennsylvania, a private Ivy League research university in P ...
and the School of Arts and Sciences. He was elected a Member of the
American Philosophical Society The American Philosophical Society (APS), founded in 1743 in Philadelphia, is a scholarly organization that promotes knowledge in the sciences and humanities through research, professional meetings, publications, library resources, and communit ...
in 2019. He has written several non-fiction books at the intersection of psychology, political science and organizational behavior, including '' Superforecasting: The Art and Science of Prediction''; '' Expert Political Judgment: How Good Is It? How Can We Know?''; ''Unmaking the West: What-if Scenarios that Rewrite World History''; and ''Counterfactual Thought Experiments in World Politics.'' Tetlock is also co-principal investigator of The Good Judgment Project, a multi-year study of the feasibility of improving the accuracy of probability judgments of high-stakes, real-world events.


Biography

Tetlock was born in 1954 in Toronto, Canada and completed his undergraduate work at the
University of British Columbia The University of British Columbia (UBC) is a public research university with campuses near Vancouver and in Kelowna, British Columbia. Established in 1908, it is British Columbia's oldest university. The university ranks among the top thre ...
and doctoral work at
Yale University Yale University is a Private university, private research university in New Haven, Connecticut. Established in 1701 as the Collegiate School, it is the List of Colonial Colleges, third-oldest institution of higher education in the United Sta ...
, obtaining his PhD in 1979. He has served on the faculty of the
University of California, Berkeley The University of California, Berkeley (UC Berkeley, Berkeley, Cal, or California) is a public land-grant research university in Berkeley, California. Established in 1868 as the University of California, it is the state's first land-grant u ...
(1979–1995, assistant professor), the
Ohio State University The Ohio State University, commonly called Ohio State or OSU, is a public land-grant research university in Columbus, Ohio. A member of the University System of Ohio, it has been ranked by major institutional rankings among the best publ ...
(the Burtt Endowed Chair in Psychology and Political Science, 1996–2001) and again at the University of California Berkeley (the Mitchell Endowed Chair at the
Haas School of Business The Walter A. Haas School of Business, also known as Berkeley Haas, is the business school of the University of California, Berkeley, a public research university in Berkeley, California. It was the first business school at a public university i ...
, 2002–2010). Since 2011, he has been the Annenberg University Professor at the University of Pennsylvania. Tetlock has received awards from scientific societies and foundations, including the
American Psychological Association The American Psychological Association (APA) is the largest scientific and professional organization of psychologists in the United States, with over 133,000 members, including scientists, educators, clinicians, consultants, and students. It ha ...
, American Political Science Association, American Association for the Advancement of Science,
International Society of Political Psychology The International Society of Political Psychology (ISPP) is an interdisciplinary not-for-profit organization, representing all fields of enquiry involved with the exploration of relationships between both psychological and political processes and ...
,
American Academy of Arts and Sciences The American Academy of Arts and Sciences (abbreviation: AAA&S) is one of the oldest learned societies in the United States. It was founded in 1780 during the American Revolution by John Adams, John Hancock, James Bowdoin, Andrew Oliver, a ...
, the National Academy of Sciences and the MacArthur,
Sage Sage or SAGE may refer to: Plants * ''Salvia officinalis'', common sage, a small evergreen subshrub used as a culinary herb ** Lamiaceae, a family of flowering plants commonly known as the mint or deadnettle or sage family ** ''Salvia'', a large ...
, Grawemeyer, and Carnegie Foundations.


Work

He has published over 200 articles in peer-reviewed journals and has edited or written ten books. Tetlock's research program over the last four decades has explored five themes: # the concept of good judgment (with special emphasis on the usefulness of forecasting tournaments in assessing one key component of good judgment: accuracy); # the impact of accountability on judgment and choice; # the constraints that sacred values place on the boundaries of the thinkable; # the difficult-to-define distinction between political versus politicized psychology; and # the usefulness of hypothetical-society experiments in disentangling fact and value judgments of the impact of competing policy proposals.


The Good Judgment Project

In his early work on good judgment, summarized in '' Expert Political Judgment: How Good Is It? How Can We Know?'', Tetlock conducted a set of small scale forecasting tournaments between 1984 and 2003. The forecasters were 284 experts from a variety of fields, including government officials, professors, journalists, and others, with many opinions, from Marxists to free-marketeers. The tournaments solicited roughly 28,000 predictions about the future and found the forecasters were often only slightly more accurate than chance, and usually worse than basic extrapolation algorithms, especially on longer–range forecasts three to five years out. Forecasters with the biggest news media profiles were also especially bad. This work suggests that there is an inverse relationship between fame and accuracy. As a result of this work, he received the 2008 University of Louisville Grawemeyer Award for Ideas Improving World Order, as well as the 2006 Woodrow Wilson Award for best book published on government, politics, or international affairs and the Robert E. Lane Award for best book in political psychology, both from the American Political Science Association in 2005. The expert political judgment project also compared the accuracy track records of "foxes" and "hedgehogs" (two personality types identified in Isaiah Berlin's 1950 essay " The Hedgehog and the Fox"). "Hedgehogs" performed less well, especially on long-term forecasts within the domain of their expertise. These findings were reported widely in the media and came to the attention of
Intelligence Advanced Research Projects Activity The Intelligence Advanced Research Projects Activity (IARPA) is an organization within the Office of the Director of National Intelligence responsible for leading research to overcome difficult challenges relevant to the United States Intellige ...
(IARPA) inside the United States intelligence community—a fact that was partly responsible for the 2011 launch of a four-year geopolitical forecasting tournament that engaged tens of thousands of forecasters and drew over one million forecasts across roughly 500 questions of relevance to U.S. national security, broadly defined. Since 2011, Tetlock and his wife/research partner
Barbara Mellers Barbara Ann Mellers is I. George Heyman University Professor in the psychology department at the University of Pennsylvania. Her research focuses on decision processes. Education Mellers earned her undergraduate degree in psychology from the Un ...
have been co-leaders of the
Good Judgment Project The Good Judgment Project (GJP) is an organization dedicated to "harnessing the wisdom of the crowd to forecast world events". It was co-created by Philip E. Tetlock (author of ''Superforecasting'' and ''Expert Political Judgment''), decision sci ...
(GJP), a research collaborative that emerged as the winner of the IARPA tournament. The original aim of the tournament was to improve geo-political and geo-economic forecasting. Illustrative questions include "What is the chance that a member will withdraw from the European Union by a target date?" or "What is the likelihood of naval clashes claiming over 10 lives in the East China Sea?" or "How likely is the head of state of Venezuela to resign by a target date?" The tournament challenged GJP and its competitors at other academic institutions to come up with innovative methods of recruiting gifted forecasters, methods of training forecasters in basic principles of probabilistic reasoning, methods of forming teams that are more than the sum of their individual parts and methods of developing aggregation algorithms that most effectively distill the wisdom of the crowd. Among the more surprising findings from the tournament were: # the degree to which simple training exercises improved the accuracy of probabilistic judgments as measured by Brier scores; # the degree to which the best forecasters could learn to distinguish many degrees of uncertainty along the zero to 1.0 probability scale (many more distinctions than the traditional 7-point verbal scale used by the National Intelligence Council); # the consistency of the performance of the elite forecasters (superforecasters) across time and categories of questions; # the power of a log-odds extremizing aggregation algorithm to out-perform competitors; and # the apparent ability of GJP to generate probability estimates that were "reportedly 30% better than intelligence officers with access to actual classified information." These and other findings are laid out in particularly accessible form in the Tetlock and Gardner (2015) book on "
Superforecasting ''Superforecasting: The Art and Science of Prediction'' is a book by Philip E. Tetlock and Dan Gardner released in 2015. It details findings from The Good Judgment Project. Reviews ''The Economist'' reports that superforecasters are clever (w ...
." The book also profiles several "superforecasters." The authors stress that good forecasting does not require powerful computers or arcane methods. It involves gathering evidence from a variety of sources, thinking probabilistically, working in teams, keeping score, and being willing to admit error and change course. There is a tension, if not contradiction, between the positions taken in the Good Judgment Project and those that Tetlock took in his earlier book ''Expert Political Judgment: How Good Is It? How Can We Know?'' (2005). The more pessimistic tone of ''Expert Political Judgment'' (2005) and optimistic tone of ''Superforecasting'' (2015) reflects less a shift in Tetlock’s views on the feasibility of forecasting than it does the different sources of data in the two projects. The ''Superforecasting'' book focused on shorter-range forecasts, the longest of which, about 12 months, being only as long as the shortest forecasts in the ''Expert Political Judgment'' project. Tetlock and Gardner (2015) also suggest that the public accountability of participants in the later IARPA tournament boosted performance. Apparently, "even the most opinionated hedgehogs become more circumspect" when they feel their accuracy will soon be compared to that of ideological rivals. Tetlock and Mellers see forecasting tournaments as a possible mechanism for helping intelligence agencies escape from blame-game (or accountability) ping-pong in which agencies find themselves whipsawed between clashing critiques that they were either too slow to issue warnings (false negatives such as 9/11) and too fast to issue warnings (false positives). They argue that tournaments are ways of signaling that an organization is committed to playing a pure accuracy game – and generating probability estimates that are as accurate as possible (and not tilting estimates to avoid the most recent "mistake").


Accountability and judgment processes

In a 1985 essay, Tetlock proposed that accountability is a key concept for linking the individual levels of analysis to the social-system levels of analysis. Accountability binds people to collectivities by specifying who must answer to whom, for what, and under what ground rules. In his earlier work in this area, he showed that some forms of accountability can make humans more thoughtful and constructively self-critical (reducing the likelihood of biases or errors), whereas other forms of accountability can make us more rigid and defensive (mobilizing mental effort to defend previous positions and to criticize critics). In a 2009 essay, Tetlock argues that much is still unknown about how psychologically deep the effects of accountability run—for instance, whether it is or is not possible to check automatic or implicit association-based biases, a topic with legal implications for companies in employment discrimination class actions. In addition to his work on the bias-attenuating versus bias-amplifying effects of accountability, Tetlock has explored the political dimensions of accountability. When, for instance, do liberals and conservatives diverge in the preferences for "process accountability" that holds people responsible for respecting rules versus "outcome accountability" that holds people accountable for bottom-line results? Tetlock uses the phrase "intuitive politician research program" to describe this line of work.


Taboo cognition and sacred values

Tetlock uses a different "functionalist metaphor" to describe his work on how people react to threats to sacred values—and how they take pains to structure situations so as to avoid open or transparent trade-offs involving sacred values. Real-world implications of this claim are explored largely in business-school journals such as the ''
Journal of Consumer Research The ''Journal of Consumer Research'' is a bimonthly peer-reviewed academic journal covering research on the psychological aspects of consumer behavior. It was established in 1974 and originally published by University of Chicago Press. Since 2015 ...
'', ''
California Management Review ''California Management Review'' is a quarterly peer-reviewed academic journal on management that is affiliated with the Walter A. Haas School of Business at the University of California, Berkeley. It was established in 1958 and covers the field o ...
'', and ''
Journal of Consumer Psychology The ''Journal of Consumer Psychology'' is a quarterly peer-reviewed academic journal covering psychology as it relates to consumer behavior. It was established in 1992 and is published by John Wiley & Sons on behalf of the Society for Consumer Psy ...
''. This research argues that most people recoil from the specter of
relativism Relativism is a family of philosophical views which deny claims to objectivity within a particular domain and assert that valuations in that domain are relative to the perspective of an observer or the context in which they are assessed. Ther ...
: the notion that the deepest moral-political values are arbitrary inventions of mere mortals desperately trying to infuse moral meaning into an otherwise meaningless universe. Rather, humans prefer to believe that they have sacred values that provide firm foundations for their moral-political opinions. People can become very punitive "intuitive prosecutors" when they feel sacred values have been seriously violated, going well beyond the range of socially acceptable forms of punishment when given chances to do so covertly.


Political versus politicized psychology

Tetlock has a long-standing interest in the tensions between political and politicized psychology. He argues that most political psychologists tacitly assume that, relative to political science, psychology is the more basic discipline in their hybrid field. In this view, political actors—be they voters or national leaders—are human beings whose behavior should be subject to fundamental psychological laws that cut across cultures and historical periods. Although he too occasionally adopts this reductionist view of political psychology in his work, he has also raised the contrarian possibility in numerous articles and chapters that reductionism sometimes runs in reverse—and that psychological research is often driven by ideological agenda (of which the psychologists often seem to be only partly conscious). Tetlock has advanced variants of this argument in articles on the links between cognitive styles and ideology (the fine line between rigid and principled) as well as on the challenges of assessing value-charged concepts like symbolic racism and unconscious bias (is it possible to be a "Bayesian bigot"?). Tetlock has also co-authored papers on the value of ideological diversity in psychological and social science research. One consequence of the lack of ideological diversity in high-stakes, soft-science fields is frequent failures of what Tetlock calls turnabout tests.


Experimental political philosophy

In collaboration with Greg Mitchell and Linda Skitka, Tetlock has conducted research on hypothetical societies and intuitions about justice ("experimental political philosophy"). The spotlight here is on a fundamental question in political theory: who should get what from whom, when, how, and why? In real-world debates over distributive justice, however, Tetlock argues it is virtually impossible to disentangle the factual assumptions that people are making about human beings from the value judgments people are making about end-state goals, such as equality and efficiency. Hypothetical society studies make it possible for social scientists to disentangle these otherwise hopelessly confounded influences on public policy preferences.


References


External links

*
Phil Tetlock
at
Social Psychology Network The Social Psychology Network (SPN) is an educational organization with more than 1,500 members worldwide. SPN was founded by psychology professor Scott Plous as a website in 1996. Development of SPN was supported by several grants from the Nation ...
maintained by
Scott Plous Scott Plous is an American academic social psychologist, currently working as a Professor of Psychology at the Department of Psychology, Wesleyan University. Biography His areas of research include the psychology of prejudice and discrimination, ...
{{DEFAULTSORT:Tetlock, Philip E. 1954 births Living people American political scientists Canadian political scientists Canadian economists Political science writers Psychology writers University of British Columbia alumni Yale University alumni University of Pennsylvania faculty Walter H. Annenberg Professor Futurologists Moral psychologists Members of the American Philosophical Society Political psychologists Haas School of Business faculty