Syllabus for Solon Barocas @ Cornell | INFO 4270: Ethics and Policy in Data Science

INFO 4270 – Ethics and Policy in Data Science
Instructor: Solon Barocas
Venue: Cornell University


Solon Barocas


A Canon, The Canon

In order of appearance in the syllabus, without the course cadence markers…

  • Danah Boyd and Kate Crawford, Critical Questions for Big Data; In <paywalled>Information, Communication & Society,Volume 15, Issue 5 (A decade in Internet time: the dynamics of the Internet and society); 2012; DOI:10.1080/1369118X.2012.678878</paywalled>
    Subtitle: Provocations for a cultural, technological, and scholarly phenomenon
  • Tal Zarsky, The Trouble with Algorithmic Decisions; In Science, Technology & Human Values, Vol 41, Issue 1, 2016 (2015-10-14); ResearchGate.
    Subtitle: An Analytic Road Map to Examine Efficiency and Fairness in Automated and Opaque Decision Making
  • Cathy O’Neil, Weapons of Math Destruction; Broadway Books; 2016-09-06; 290 pages, ASIN:B019B6VCLO: Kindle: $12, paper: 10+SHT.
  • Frank Pasquale, The Black Box Society: The Secret Algorithms That Control Money and Information; Harvard University Press; 2016-08-29; 320 pages; ASIN:0674970845: Kindle: $10, paper: $13+SHT.
  • Executive Office of the President, President Barack Obama, Big Data: A Report on Algorithmic Systems, Opportunity, and Civil Rights; The White House Office of Science and Technology Policy (OSTP); 2016-05; 29 pages; archives.
  • Lisa Gitelman (editor), “Raw Data” is an Oxymoron; Series: Infrastructures; The MIT Press; 2013-01-25; 192 pages; ASIN:B00HCW7H0A: Kindle: $20, paper: $18+SHT.
    Lisa Gitelman, Virginia Jackson; Introduction (6 pages)
  • Agre, “Surveillance and Capture: Two Models of Privacy”
  • Bowker and Star, Sorting Things Out
  • Auerbach, “The Stupidity of Computers”
  • Moor, “What is Computer Ethics?”
  • Hand, “Deconstructing Statistical Questions”
  • O’Neil, On Being a Data Skeptic
  • Domingos, “A Few Useful Things to Know About Machine Learning”
  • Luca, Kleinberg, and Mullainathan, “Algorithms Need Managers, Too”
  • Friedman and Nissenbaum, “Bias in Computer Systems”
  • Lerman, “Big Data and Its Exclusions”
  • Hand, “Classifier Technology and the Illusion of Progress” [Sections 3 and 4]
  • Pager and Shepherd, “The Sociology of Discrimination: Racial Discrimination in Employment, Housing, Credit, and Consumer Markets”
  • Goodman, “Economic Models of (Algorithmic) Discrimination”
  • Hardt, “How Big Data Is Unfair”
  • Barocas and Selbst, “Big Data’s Disparate Impact” [Parts I and II]
  • Gandy, “It’s Discrimination, Stupid”
  • Dwork and Mulligan, “It’s Not Privacy, and It’s Not Fair”
  • Sandvig, Hamilton, Karahalios, and Langbort, “Auditing Algorithms: Research Methods for Detecting Discrimination on Internet Platforms”
  • Diakopoulos, “Algorithmic Accountability: Journalistic Investigation of Computational Power Structures”
  • Lavergne and Mullainathan, “Are Emily and Greg more Employable than Lakisha and Jamal?”
  • Sweeney, “Discrimination in Online Ad Delivery”
  • Datta, Tschantz, and Datta, “Automated Experiments on Ad Privacy Settings”
  • Dwork, Hardt, Pitassi, Reingold, and Zemel, “Fairness Through Awareness”
  • Feldman, Friedler, Moeller, Scheidegger, and Venkatasubramanian, “Certifying and Removing Disparate Impact”
  • Žliobaitė and Custers, “Using Sensitive Personal Data May Be Necessary for Avoiding Discrimination in Data-Driven Decision Models”
  • Angwin, Larson, Mattu, and Kirchner, “Machine Bias”
  • Kleinberg, Mullainathan, and Raghavan, “Inherent Trade-Offs in the Fair Determination of Risk Scores”
  • Northpointe, COMPAS Risk Scales: Demonstrating Accuracy Equity and Predictive Parity
  • Chouldechova, “Fair Prediction with Disparate Impact”
  • Berk, Heidari, Jabbari, Kearns, and Roth, “Fairness in Criminal Justice Risk Assessments: The State of the Art”
  • Hardt, Price, and Srebro, “Equality of Opportunity in Supervised Learning”
  • Wattenberg, Viégas, and Hardt, “Attacking Discrimination with Smarter Machine Learning”
  • Friedler, Scheidegger, and Venkatasubramanian, “On the (Im)possibility of Fairness”
  • Tene and Polonetsky, “Taming the Golem: Challenges of Ethical Algorithmic Decision Making”
  • Lum and Isaac, “To Predict and Serve?”
  • Joseph, Kearns, Morgenstern, and Roth, “Fairness in Learning: Classic and Contextual Bandits”
  • Barocas, “Data Mining and the Discourse on Discrimination”
  • Grgić-Hlača, Zafar, Gummadi, and Weller, “The Case for Process Fairness in Learning: Feature Selection for Fair Decision Making”
  • Vedder, “KDD: The Challenge to Individualism”
  • Lippert-Rasmussen, “‘We Are All Different’: Statistical Discrimination and the Right to Be Treated as an Individual”
  • Schauer, Profiles, Probabilities, And Stereotypes
  • Caliskan, Bryson, and Narayanan, “Semantics Derived Automatically from Language Corpora Contain Human-like Biases”
  • Zhao, Wang, Yatskar, Ordonez, and Chang, “Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints”
  • Bolukbasi, Chang, Zou, Saligrama, and Kalai, “Man Is to Computer Programmer as Woman Is to Homemaker?”
  • Citron and Pasquale, “The Scored Society: Due Process for Automated Predictions”
  • Ananny and Crawford, “Seeing without Knowing”
  • de Vries, “Privacy, Due Process and the Computational Turn”
  • Zarsky, “Transparent Predictions”
  • Crawford and Schultz, “Big Data and Due Process”
  • Kroll, Huey, Barocas, Felten, Reidenberg, Robinson, and Yu, “Accountable Algorithms”
  • Bornstein, “Is Artificial Intelligence Permanently Inscrutable?”
  • Burrell, “How the Machine ‘Thinks’”
  • Lipton, “The Mythos of Model Interpretability”
  • Doshi-Velez and Kim, “Towards a Rigorous Science of Interpretable Machine Learning”
  • Hall, Phan, and Ambati, “Ideas on Interpreting Machine Learning”
  • Grimmelmann and Westreich, “Incomprehensible Discrimination”
  • Selbst and Barocas, “Regulating Inscrutable Systems”
  • Jones, “The Right to a Human in the Loop”
  • Edwards and Veale, “Slave to the Algorithm? Why a ‘Right to Explanation’ is Probably Not the Remedy You are Looking for”
  • Duhigg, “How Companies Learn Your Secrets”
  • Kosinski, Stillwell, and Graepel, “Private Traits and Attributes Are Predictable from Digital Records of Human Behavior”
  • Barocas and Nissenbaum, “Big Data’s End Run around Procedural Privacy Protections”
  • Chen, Fraiberger, Moakler, and Provost, “Enhancing Transparency and Control when Drawing Data-Driven Inferences about Individuals”
  • Robinson and Yu, Knowing the Score
  • Hurley and Adebayo, “Credit Scoring in the Era of Big Data”
  • Valentino-Devries, Singer-Vine, and Soltani, “Websites Vary Prices, Deals Based on Users’ Information”
  • The Council of Economic Advisers, Big Data and Differential Pricing
  • Hannak, Soeller, Lazer, Mislove, and Wilson, “Measuring Price Discrimination and Steering on E-commerce Web Sites”
  • Kochelek, “Data Mining and Antitrust”
  • Helveston, “Consumer Protection in the Age of Big Data”
  • Kolata, “New Gene Tests Pose a Threat to Insurers”
  • Swedloff, “Risk Classification’s Big Data (R)evolution”
  • Cooper, “Separation, Pooling, and Big Data”
  • Simon, “The Ideological Effects of Actuarial Practices”
  • Tufekci, “Engineering the Public”
  • Calo, “Digital Market Manipulation”
  • Kaptein and Eckles, “Selecting Effective Means to Any End”
  • Pariser, “Beware Online ‘Filter Bubbles’”
  • Gillespie, “The Relevance of Algorithms”
  • Buolamwini, “Algorithms Aren’t Racist. Your Skin Is just too Dark”
  • Hassein, “Against Black Inclusion in Facial Recognition”
  • Agüera y Arcas, Mitchell, and Todorov, “Physiognomy’s New Clothes”
  • Garvie, Bedoya, and Frankle, The Perpetual Line-Up
  • Wu and Zhang, “Automated Inference on Criminality using Face Images”
  • Haggerty, “Methodology as a Knife Fight”
    <snide>A metaphorical usage. Let hyperbole be your guide</snide>

Previously filled.

When Are You Really An Adult? | The Atlantic

When Are You Really An Adult?; Julie Beck; In The Atlantic; 2016-01-05.
Teaser: In an age when the line between childhood and adulthood is blurrier than ever, what is it that makes people grown up?

tl;dr → 7000 words; it depends; ultimately <concept>when one is secure with ones self</concept>


Recent book releases


  • What is it about 20-Somethings; Robin Marantz Henig; In The York Times (NYT), Magazine, 2010-08-18.
    Teaser: Why are so many people in their 20s taking so long to grow up?
    tl;dr → 8000 words, basically the same as this article, except done by someone else, and appearing in the NYT and executed five years ago.


  • Failure to Launch
  • Steven Mintz
  • Kelly Williams Brown
    • age 31
    • bloggist
  • Generational Model
    • Millennial
    • Generation X
    • Baby Boomer
  • Social Constructions
    • Chlidhood
    • Adulthood
  • Noel Cameron
    • professor, human biology, Loughborough University, U.K.
    • quoted
  • Laurence Steinberg
  • James Griffin
    • deputy chief, Child Development and Behavior Branch, National Institute of Child Health and Human Development (NICHD)
    • is quoted on emotions
      <quote>the four Fs—fight, flight, feeding, and fuckfooling around.</quote>
  • Jeffrey Jensen Arnett
    • research professor, psychology, Clark University
    • Emerging Adulthood
      a new category, proposed & defended by him (see the book)
    • The Big Three, a framework
      1. taking responsibility for yourself
      2. making independent decisions
      3. becoming financially independent
  • James Côté
    • sociology
    • “The Dangerous Myth of Emerging Adulthood: An Evidence-Based Critique of a Flawed Developmental Theory”; In Applied Developmental Science; Volume 18, Issue 4; 2015; paywalled.
  • <quote>Of the Big Three, two are internal, subjective markers. You can measure financial independence, but are you otherwise independent and responsible? That’s something you have to decide for yourself. </quote>
  • Erik Erikson
    • psychologist, development
  • Anthony Burrow
    • assistant professor, human development, Cornell University
    • Rachel Sumner, Anthony L. Burrow, Patrick L. Hill; “Identity and Purpose as Predictors of Subjective Well-Being in Emerging Adulthood; In Emerging Adulthood; 2014-04-30, updated 2015-01-08; paywall.
  • <quote>In other words, the flailing isn’t fun, but it matters.</quote>
    • Four-box model (not shown)
    • Something about Taylor Swift, lyrics from “22.”
      <quote>We’re happy, free, confused, and lonely at the same time.</quote>
  • Robert Havighurst
    • education researcher
    • era “the 20th-century”
    • A Life Stage model, with tasks
      • Finding a mate
      • Learning to live with a partner
      • Starting a family
      • Raising children
      • Beginning an occupation
      • Running a home.
  • The “Leave it to Beaveradulthood”, branding due to the reporter, Julia Beck.
    • <quote?These are the things Millennials are all-too-often criticized for not doing and not valuing.</quote>
    • Something about how this was a brief golden age that came and went.
      • Wasn’t thus before.
      • Isn’t thus now.
      • It’s a fiction of the Baby Boomers.
  • <quote>When people who are in their 50s, 60s, 70s now look at today’s emerging adults, they compare them to the yardstick that applied when they were in their 20s, and find them wanting. But to me that’s, ironically, kind of narcissistic, frankly, because that’s one of the criticisms that’s been made of emerging adults, that they’re narcissistic, but to me it’s just the egocentricity of their elders.</quote>, attributed to Jeffrey Jensen Arnett.
  • Rachel Sumner
    • graduate student, Anthony Burrow
    • Rachel Sumner, Anthony L. Burrow, Patrick L. Hill; “Identity and Purpose as Predictors of Subjective Well-Being in Emerging Adulthood; In Emerging Adulthood; 2014-04-30, updated 2015-01-08; paywall.
  • Denoument, Counterpoint & Onward
    • Many ways to become an adult
      but then the category means nothing; this rebuttal is rebutted.
    • Adulthood is
      • independence, but loneliness,
      • Responsibility causes stress.
    • Chroniclers & fictionalists
      • Saul Bellow
      • Mary McCarthy
      • Philip Roth
      • John Updike
    • Avatars & Actrons
      • old Hollywood visions of adulthood
      • Cary Grant
      • Katherine Hepburn
    • <quote>We live in a youth culture that believes life goes downhill after 26 or so. When I argue that we need to reclaim adulthood, I don’t mean a 1950s version of early marriage and early entry into a career, What I do mean is it’s better to be knowing than unknowing. It’s better to be experienced than inexperienced. It’s better to be sophisticated than callow.</quote> attributed to Steven Mintz,
    • <quote>[Adulthood is] taking care of people, taking care of things, and taking care of yourself.</quote>, attributed to Kelly Williams Brown.


largely by discursion & negation, the “post-modern” explanation.

  • not physical maturation, that varies by age
  • not by education, which is demarked by age anyway.
  • not by cultural (religious) rites, in theory only.
  • many paths
  • Milestones & Experiences


  • Henry David Thoreau
    • Harvard (undergrad)
    • odd jobs
    • A Week on the Concord and Merrimack Rivers
      • age 31
  • Maria Eleusiniotis
    • testifies
  • Stephen Grapes
    • testifies
  • Anonymous
    • testifies
    • roles
      • OB/GYN
      • mom
  • Anonymous
    • testifies
    • role
      • then-intern
      • (now?) doctor
    • <concept>You become an adult when you are in charge, responsible, accountable.</quote>
    • <quote>The question of when a tree becomes a tree and no longer a sapling is obviously impossible to determine. Same with any slow and gradual process. All I can say is that the adult potential was there, ready to grow up and be responsible and accountable. I think personal industry, devotion to something bigger than oneself, part of a historical process, and peers who grow with you all play roles.Without focus, work, hardship, or a pathway with other humans, I can imagine someone still believing they are a child at 35-45: I meet them sometimes! And it is horrific.</quote>
  • Deb Bissen
    • testifies
    • a new mom
  • Anonymous
    • age 53
    • testifies
    • manages
      • her parent’s transition ot managed care via “micro betrayals” (white lies)
      • the parent’s subsequent death, 2013.
  • Anonymous
    • testifies
    • 1st-generation immigrant
    • milestones
      • age 27
      • married
      • living alone (with spouse?)
      • employed, as a manager, stable.
    • adulthood came too quickly
  • Anonymous
    • testifies
    • quibbles with the term ‘adult’ as being synonymous with “reserved” or “passionless.”
  • Anonymous
    • testifies
    • milestones
      • age 56
      • married
      • masters degree
      • stable job, apparently a teacher (has students).
      • has traveled
      • no children
    • charged with “You never really grew up, did you?”
    • rebuts
      • have experienced death
      • have made end-of-life decisions (of a pet)
      • takes care of elderly parents
      • care about retirement
      • grey hair
      • knees hurt



  • Some Statistic, Bureau of the Census, United States
    evidence towards marriage age
  • Some Statistic, Bureau of the Census, United States.
    evidence towrds marriage age occurring later in life
  • The Case for Delayed Adulthood; Laurence Steinberg; In The New York Times (NYT); 2014-09-21.
    tl;dr → a book promotion
    Laurence Steinberg

  • Some Statistic, Department of Labor, United States.
    evidence for the statement: <quote>kids can hold a job as young as 14, depending on state restrictions</quote>
  • Some Statistic, Department of Labor, United States.
    evidence for the statement: <quote>[children can] deliver newspapers, babysit, or work for their parents even younger than that</quote>.
  • Some Statistic, National Institute of Child Health and Human Development (NICHD), United States.
    evidence for the statement: <quote>9 and 14 for boys, and still be considered “normal.”</quote>
  • Some Statistic, Department of Education?, United States; WHEN?
    evidence for the statement: <quote>by 1918, every state had compulsory [school] attendance laws.</quote>
  • Leo B. Hendry, Marion Kloep; “How universal is emerging adulthood? An empirical example”; In Journal of Youth Studies, Volume 13, Issue 2, 2010; paywalled.
  • James Côté; “The Dangerous Myth of Emerging Adulthood: An Evidence-Based Critique of a Flawed Developmental Theory”; In Applied Developmental Science; Volume 18, Issue 4; 2015; paywalled.
  • Rachel Sumner, Anthony L. Burrow, Patrick L. Hill; “Identity and Purpose as Predictors of Subjective Well-Being in Emerging Adulthood; In Emerging Adulthood; 2014-04-30, updated 2015-01-08; paywall.
  • Koen Luyckx, Luc Goossens, Bart Soenens, Wim Beyers; “Unpacking commitment and exploration: Preliminary validation of an integrative model of late adolescent identity formation”; In Journal of Adolescence; Volume 29, Issue 3; 2006-06; pages 361–378; paywall.
    tl;dr → something about forming an identity
  • Koen Luycks, Seth J. Schwartz, Luc Goossens, Sophie Pollock; “Employment, Sense of Coherence, and Identity Formation: Contextual and Psychological Processes on the Pathway to Sense of Adulthood”; In Journal of Adolescent Research; Vol. 23, No. 5; 2008-09; pages 566-591; paywall.
    tl;dr → something about how people who’ve committed to an identity are more likely to see themselves as adults.

Via: backfill.