CS 5/692, Winter 2020

February 24 / Week 8

Fairness, Accountability, and Transparency

Readings

Start with this piece, from the popular media:

• Merchant, B. (2018). Predictim Claims its AI Can Flag “Risky” Babysitters. So I Tried it on the People Who Watch My Kids. Gizmodo, 12/6/18

Next, read the following articles, discussing ways in which we can analyze and communicate about the behavior of machine learning models. Keep Predictim’s product in mind as you reflect on the various proposals and ideas. What would fairness, accountability, and transparency look like for a commercial product like Predictim?

• Diakopoulos, N. (2016). Accountability in Algorithmic Decision Making. Commun. ACM, 59(2), 56–62.
• One of:
  • Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H. M., Daumé, H., III, & Crawford, K. (2018). Datasheets for Datasets. arXiv preprint 1803.09010. (also presented at FAT-ML ‘18)
  • Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Ben Hutchinson, et al. (2019). Model Cards for Model Reporting (pp. 220–229). Presented at the Proceedings of the Conference on Fairness, Accountability, and Transparency, FAT* 2019, Atlanta, GA, USA, January 29-31, 2019.

Finally, consider the following piece. Make sure to read all the way to the end- there is additional content after the bibliography.

• Keyes, O., Hutson, J., & Durbin, M. (2019). A Mulching Proposal: Analysing and Improving an Algorithmic System for Turning the Elderly into High-Nutrient Slurry (pp. alt06:1–alt06:11). Presented at the Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, New York, NY, USA: ACM.
  • Note that the ACM Digital Library entry for this paper includes a short video of it being presented at last year’s CHI; look under “Supplemental Material”. You may find the presentation and ensuing Q&A interesting.

What do

Optional (but recommended)

• Gajane, P., & Pechenizkiy, M. (2017). On Formalizing Fairness in Prediction with Machine Learning. arXiv preprint 1710.03184
• For a deeper dive, consider Barocas, Hardt, & Narayanan’s “Fairness and Machine Learning: Limitations and Opportunities”. Chapters 1, 2, and 5 are particularly useful to this week’s content.
• Daume, H. (2017) “Chapter 8: Bias and Fairness”, in A Course in Macine Learning