CS 5/579, Winter 2020

February 3/5 / Week 5

Parallelism, Distributed Math & ML

Monday: MPI and Dask

Readings

• LLNL MPI Tutorial
• Jonathan Durst, “HPC is dying, and MPI is killing it” and “Objections, Continued”
• Dask: 
  • Matthew Rocklin, “Dask: Parallel Computation with Blocked algorithms and Task Scheduling”
  • Useful videos: 
    • 2015 Project Overview: https://www.youtube.com/watch?v=1kkFZ4P-XHg
    • 2016: “Outgrowing strict data structures, and Architecture”: https://www.youtube.com/watch?v=PAGjm4BMKlk
    • 2017: “Dask advanced scheduling architectures”:https://www.youtube.com/watch?v=ZxNPVTHQwGo
  • If you’re really interested in Dask, and want more, here’s a 3-hour interactive tutorial 

Resources

• MPI4Py Documentation
• MPI with Python Tutorial
• jbornschein/mpi4py-examples

Wednesday: Distributed Math & ML (Part 1)

• “Distributed Training Strategies for the Structured Perceptron”, Ryan McDonald et. al., Proc NAACL, 2010.
• “Scalable k-means++.” Bahmani, Bahman, Benjamin Moseley, Andrea Vattani, Ravi Kumar, and Sergei Vassilvitskii. Proceedings of the VLDB Endowment 5, no. 7 (2012): 622-633.