CS 6106 Autumn 2026 · Dept. of Computer Science & Engineering
Efficient AI
Algorithmic and hardware-aware approaches to making modern AI cheaper to train and faster to serve — pruning, quantization, parameter-efficient fine-tuning, speculative decoding, KV-cache compression, and the systems that carry them.
Slot TBA · Venue TBA · 6 credits · jump to schedule
Updates
-
Jul 05
The course website is live. The lecture slot and venue will be announced here before the semester begins.
-
Jul 05
Registration opens with the institute pre-registration window. Seats for non-CSE students are by consent of the instructor.
Course staff
Prof. Aditya Desai
Instructor
instructor@cse.iitb.ac.in
KReSIT Building, Room KR 208 · office hours by appointment
Logistics
- Lectures
- Slot TBA — two sessions a week. Recordings, if any, will be linked from the schedule.
- Venue
- TBA
- Credits
- 6
- Prerequisites
- A first course in machine learning. Familiarity with computer architecture helps but is not required. Others by consent of the instructor.
- Grading
-
- Course project40%
- Paper presentations25%
- Assignments20%
- Reviews & participation15%
- Communication
- Announcements on this page and Moodle. The discussion-forum link will be shared in the first lecture.
Contents
Five arcs, ordered roughly as the semester runs.
-
I. Compression & quantization
Model compression · knowledge distillation · quantization · hardware support for quantization
-
II. Sparsity & pruning
Pruning · the lottery ticket hypothesis · hardware support for sparsity · sparse training
-
III. Architectures & adaptation
Neural architecture search · low-rank structure (butterfly & monarch matrices) · PEFT — LoRA and variants, SketchTune · elastic parameter memory · hardware-aware hashing
-
IV. Efficient attention & inference
Sparse attention (prefill & decode, training-time) · speculative decoding · KV-cache compression · semantic prompt caching · attention alternatives — SSMs & linear attention
-
V. Systems & hardware
IR techniques in LLM inference (SLIDE and successors) · upcoming GPU IR languages · context extension & compaction
Schedule
Dates are tentative until the academic calendar is finalised. Slides and readings land in the resources column after each week.
| No. | Date | Description | Resources |
|---|---|---|---|
| 1 | Jul 28 | Why efficient AI — the cost of training and serving; roofline model, memory hierarchy; course logistics. | MIT 6.5940 — the closest course elsewhere; good introductory slides. |
| 2 | Aug 04 | Model compression & knowledge distillation — the compression landscape; teacher–student training. | — |
| 3 | Aug 11 | Pruning & sparsity — magnitude and structured pruning; the lottery ticket hypothesis. | Frankle ’23 — the thesis behind the lottery ticket hypothesis. |
| 4 | Aug 18 | Hardware support for sparsity — sparse formats, structured sparsity on accelerators. | Sky-Light — Berkeley project; see its reading list. |
| 5 | Aug 25 | Quantization I — post-training quantization and quantization-aware training. | — |
| 6 | Sep 01 | Quantization II — hardware support; low-precision formats (INT4, FP8) and kernels. | — |
| 7 | Sep 08 | Elastic parameter memory & hardware-aware hashing — parameter sharing at extreme scale. | Desai ’24, Chen ’20 — the two theses this lecture draws on. |
| Sep 14 – 20 · mid-semester examinations, no lecture | |||
| 8 | Sep 22 | Neural architecture search — search spaces, hardware-aware objectives, once-for-all networks. | — |
| 9 | Sep 29 | Low-rank structure — butterfly and monarch matrices; structured matrices as compute savers. | Quang ’23 — hardware-aware algorithms, including monarch matrices. |
| 10 | Oct 06 | Parameter-efficient fine-tuning — LoRA and its variants, SketchTune. | — |
| 11 | Oct 13 | Sparse attention — inference-time and training-time sparsity; prefill versus decode. | — |
| 12 | Oct 20 | Speculative decoding — draft models, verification, acceptance schemes. | SpecDec papers — living list of speculative-decoding papers. |
| 13 | Oct 27 | KV-cache compression & semantic prompt caching — eviction, quantized caches, cache reuse across requests. | kvpress — NVIDIA’s library and paper list for KV-cache compression. |
| 14 | Nov 03 | Attention alternatives & systems — SSMs and linear attention; IR techniques (SLIDE), GPU IR languages; context extension & compaction. Wrap-up. | — |
References
Foundational theses and living paper lists. Weekly readings are linked from the schedule.
- Desai, A. (2024). Elastic Parameter Memory for Efficient Machine Learning. Doctoral dissertation, Rice University.
- Frankle, J. (2023). The Lottery Ticket Hypothesis: On Sparse, Trainable Neural Networks. Doctoral dissertation, Massachusetts Institute of Technology.
- Quang, T. D. P. (2023). Hardware-Aware Algorithms for Efficient Machine Learning. Doctoral dissertation, Stanford University.
- Chen, B. (2020). Locality Sensitive Sampling for Extreme-Scale Optimization and Deep Learning. Doctoral dissertation, Rice University.
- TinyML and Efficient Deep Learning Computing — MIT 6.5940, a close cousin of this course.
- Sky-Light, UC Berkeley — see the list of papers.
- Speculative decoding paper list — curated on GitHub.
- NVIDIA kvpress — KV-cache compression papers and code.