ECE405: Introduction to Large-Scale Artificial Intelligence Systems

Meeting Days/Time: Monday, Wednesday & Friday, 10:30 – 11:20 AM (CRN 57997, 3 credits)
Location: Holmes Hall 211
Semester Dates: January 11, 2027 – May 5, 2027
Instructor: Haoyue Tang — Department of Electrical and Computer Engineering, University of Hawaiʻi at Mānoa
Email: haoyue@hawaii.edu
Office Hours: TBA, POST 205E (and by appointment)

[Course Syllabus]

Outline

Language models are the core of modern AI systems. In this course, students build one from scratch. Following Stanford CS336: Language Modeling from Scratch and the previous offering of ECE405 by Prof. Igor Molybog, we go through the full pipeline of a language model: tokenization, Transformer architecture and training, GPU systems and kernels, distributed training, scaling laws, data collection and filtering, inference, evaluation, and post-training (supervised fine-tuning, preference alignment, and reinforcement learning for reasoning). Aim: for students to gain the engineering ability to reproduce results from AI papers. The course is implementation-heavy: four large coding assignments run on GPU clusters.

Prerequisites

Linear algebra, calculus, probability & statistics, and basic machine learning. A solid coding background is required: Python, PyTorch, bash, and git.

Grading

  • Assignment 1 (Basics): 25%

  • Assignment 2 (Systems & Scaling): 25%

  • Assignment 3 (Data): 25%

  • Assignment 4 (Alignment & Reasoning RL): 25% No exams. Letter grades follow the UH Mānoa plus/minus scale (see the syllabus).

Course Policies

  • Late Days: 5 late days for the semester, at most 3 per assignment. Each late day extends a deadline by 24 hours.

  • Regrades: Within 3 days after grades are released, for objective grading errors only.

  • Collaboration: Study groups are allowed, but each student writes and submits their own code and write-up. List study-group members on your submission.

  • AI Tools: LLMs may be used for low-level programming questions and high-level conceptual questions, but not to solve assignment problems directly. Please turn off AI autocomplete in your IDE. Do not consult third-party implementations unless a handout says so.

  • Academic Integrity: Students are expected to adhere to the UH Mānoa Student Conduct Code.

  • Accessibility (KOKUA): Students needing accommodations should contact the KOKUA Program.

Assignments

Assignments are adapted from the Stanford CS336 handouts and due Fridays at 11:59 PM HST unless noted. Submit on Lamaku.

Assignment Topic Released Due
A1: Basics BPE tokenizer, Transformer LM, AdamW; train on TinyStories / OpenWebText Jan 11 Feb 12
A2: Systems & Scaling Profiling, FlashAttention-2 in Triton, distributed data-parallel training, scaling laws Feb 12 Mar 12
A3: Data Common Crawl to pretraining data: extraction, filtering, deduplication Mar 12 Apr 9
A4: Alignment & Reasoning RL SFT, expert iteration, GRPO for math reasoning; optional DPO Apr 9 May 5 (Wed)

Compute

Assignments need GPUs. Onboarding is in Weeks 1–2. Debug on CPU first and start early, because queues are long right before deadlines.

General References

No required textbook. Lectures draw on publicly available resources:

Schedule

University holidays are marked below. This schedule is a draft and may be adjusted as the semester progresses.

Week Dates Topics Coursework
Week 1 Jan 11 – 15 Course overview; language modeling; tokenization (byte-pair encoding) A1 (Basics) released; Koa onboarding
Week 2 Jan 18 – 22 PyTorch tensors, einops; resource accounting (FLOPs, memory). Jan 18: MLK Day (No Class)
Week 3 Jan 25 – 29 Transformer architecture & hyperparameters
Week 4 Feb 1 – 5 Attention alternatives; mixture of experts
Week 5 Feb 8 – 12 GPUs & TPUs; memory hierarchy; benchmarking & profiling A1 due Fri Feb 12; A2 (Systems & Scaling) released
Week 6 Feb 15 – 19 Kernels & Triton; FlashAttention. Feb 15: Presidents’ Day (No Class)
Week 7 Feb 22 – 26 Parallelism I: data parallelism, ZeRO / FSDP
Week 8 Mar 1 – 5 Parallelism II: tensor, pipeline, sequence / context parallelism
Week 9 Mar 8 – 12 Scaling laws (Chinchilla, compute-optimal training, muP) A2 due Fri Mar 12; A3 (Data) released
— Mar 15 – 19 Spring Recess (No Class)
Week 10 Mar 22 – 26 Data I: sources, Common Crawl, text extraction. Mar 26: Kūhiō Day / Good Friday (No Class)
Week 11 Mar 29 – Apr 2 Data II: quality filtering, deduplication, data mixing, synthetic data
Week 12 Apr 5 – 9 Post-training I: supervised fine-tuning; RLHF; preference optimization (PPO, DPO) A3 due Fri Apr 9; A4 (Alignment & Reasoning RL) released
Week 13 Apr 12 – 16 Post-training II: RL with verifiable rewards (GRPO); reasoning models
Week 14 Apr 19 – 23 Inference: KV caching, quantization, speculative decoding, serving (vLLM)
Week 15 Apr 26 – 30 Evaluation: benchmarks & metrics; agents and tool use
Week 16 May 3 – 5 Alignment & multimodality; course wrap-up A4 due Wed May 5