creative-ml-fall-2024

Stars
2
View Code on GitHub View on X
Ecosystems: JavaScript

Creative Applications of Machine Learning

Instructor: Sam Lavigne Time: Thursdays 7pm to 9:40pm Office Hours: By appointment

Description

In this class students will experiment with emerging techniques in machine learning, developing a series of creative works that address topics like classification, fakery, automating language, surveillance, and embodiment. The approach will be both practical and critical. How does machine learning produce and reproduce repressive status quo political, economic, ecological, and social conditions, and how might these technologies be used and mis-used to further more just political and social outcomes? How does the project of “AI” continue a long and bleak history of imperialist attempts to shape the world with data? How does creative experimentation reveal the qualities of machine learning as a medium and approach? Is it even possible to make an interesting project using stable diffusion? We will primarily use open source models and tools, but will also explore commercial products to probe their limitations, biases, politics, and business models.

Readings

Readings are listed on the day they are assigned, not due. All readings that aren't already linked will be shared over Canvas. We'll devote a portion of each class to discussing the readings, with students taking turns leading the conversations.

Schedule

Week 1: Thursday, August 29, 2024 (SKIPPED!)

Week 2: Thursday, September 5, 2024

  • Introductions
  • Introduction to machine learning
  • Review of p5js
  • Introduction to ml5 (time permitting)

Readings

  • Race After Technology (just the introduction), by Ruha Benjamin
  • Why A.I. Isn’t Going to Make Art, by Ted Chiang
  • IF NEEDED: review p5.js with Xin Xin's videos or Dan's videos.

Assignment: Embodiment (due Sepetember 19th)

  • Option 1: Use a machine learning model that tracks the body to create a project that monitors, surveils, or enforces a behavior
  • Option 2: Create an interface that allows a user to interact with a computational system using their body.
  • In either option, you'll be making a system that responds to the human body in some way.

Week 3: Thursday, September 12, 2024

  • Body tracking
  • Face tracking
  • Hand tracking
  • Intro to the command line
  • Using ml5 outside the browser

Readings

Week 4: Thursday, September 19, 2024

  • Embodiment project presentations

Week 5: Thursday, September 26, 2024

  • Introduction to large language models
  • llamafile
  • Creating prompts

Assignment: Language (due October 17th)

  • Using a llamafile, create a virtual friend, assistant, therapist, romantic partner, politician, or some other type of algorithmic agent. Consider who will interact with this "person" and how. ALSO: create an appropriate interface for interacting with your virtual agent.

Readings

Week 6: Thursday, October 3, 2024

  • Connecting llamafile to other applications
  • Making your own interface for chat

Week 7: Thursday, October 10, 2024

  • Combining speech recognition, speech synthesis, and language models

Week 8: Thursday, October 17, 2024

  • Language project presentations

Week 9: Thursday, October 24, 2024

  • Introduction to classification
  • Collecting datasets

Assignment: Create a classifier (due November 7th)

  • Option 1: Train your own image or text classifier using data that you collect yourself. Consider the politics of classification. What are you trying to classify and why? Who would benefit? Who would be harmed?
  • Option 2: Use an existing classifier to build a project of your choosing.

Readings

  • Seeing Like a State (chapter 2) by James C. Scott

Week 10: Thursday, October 31, 2024

  • Training a model
  • Deploying a model
  • Facial recognition

Week 11: Thursday, November 7, 2024

  • Classification project presentations

Week 12: Thursday, November 14, 2024

  • Introduction to fakes
  • Intro to ComfyUI
  • Creating fake images

Assignment: Create a Fake (due December 12th)

  • Option 1: Look at historical examples of fakery in art and politics. Then, use machine learning to create a series of fakes or forgeries. These could be fake people, fake events, alternate realities, and so on.
  • Option 2: Elaborate or polish a previous project you made in the class.

Readings:

  • Watch "The Yes Men Fix the World"

Week 13: Thursday, November 21, 2024

  • Face cloning
  • Voice cloning

Readings:

Thursday, November 28, 2024 - No class!

Week 14: Thursday, December 5, 2024

  • More ComfyUI workflows
  • Individual meetings

Week 15: Thursday, December 12, 2024

  • Final presentations

Optional Readings