ENGG 5501: Foundations of Optimization

2023-24 First Term

Announcements

NEW: Here is the solution to Homework 5.
Here are some practice problems for the final examination.
Homework 5 is now posted. It is due on December 13, 2023. No late homework will be accepted, as the solution will be posted soon after the due date to facilitate students' review for the final examination.
Handout 7 is posted.
Here is the midterm solution.
Welcome to ENGG 5501! Students who are interested in taking the course but have not yet registered (or are not able to register) should contact the course instructor.
To better facilitate discussions and Q&As, we have set up a forum on Piazza. Please follow this link to sign up.
The lecture videos can be accessed via the link provided on Piazza.

General Information

Instructor: Anthony Man-Cho So (manchoso at se.cuhk.edu.hk)
Office Hours: By appointment, in ERB 604 or online
Lecture Time/Location:
- Mondays 12:30pm - 2:15pm, in UCC C1
- Thursdays 2:30pm - 4:15pm, in MMW LT1

Teaching Assistants:
- He Chen (hchen at se.cuhk.edu.hk)
- Lemin Kong (lkong at se.cuhk.edu.hk)
- Lai Tian (tianlai at se.cuhk.edu.hk)
- Linglingzhi Zhu (llzzhu at se.cuhk.edu.hk)
Office Hours: Thursdays 4:30pm - 6:00pm, in ERB 905 or online

Online Q&A Forum: Follow this link.

Course Description

In this course we will develop the basic machinery for formulating and analyzing various optimization problems. Topics include convex analysis, linear and conic linear programming, nonlinear programming, optimality conditions, Lagrangian duality theory, and basics of optimization algorithms. Applications from different fields, such as combinatorial optimization, communications, computational economics and finance, machine learning, and signal and image processing, will be used to complement the theoretical developments. No prior optimization background is required for this class. However, students should have workable knowledge in multivariable calculus, real analysis, linear algebra and matrix theory.

Course Requirements

Homework sets (35%), midterm examination (30%), and final examination (35%).

General References

Ben-Tal, Nemirovski: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications, SIAM, 2001.
Boyd, Vandenberghe: Convex Optimization, Cambridge University Press, 2004.
Güler: Foundations of Optimization, Springer New York, 2010.
Luenberger, Ye: Linear and Nonlinear Programming (5th Edition), Springer Cham, 2021.
Nesterov: Lectures on Convex Optimization (2nd Edition), Springer Cham, 2018.

Handouts

Handout A: Information Sheet
Handout B: Linear Algebra Cheat Sheet
Handout C: Real Analysis Cheat Sheet

The mathematical prerequisites for this course are summarized in Handouts B and C. Students are strongly advised to go through them carefully.
Handout 1: Introduction
Handout 2: Elements of Convex Analysis
Handout 3: Elements of Linear Programming
Handout 4: Some Applications of Linear Programming
Handout 5: Elements of Conic Linear Programming
Handout 6: Some Applications of Conic Linear Programming
Handout 7: Optimality Conditions and Lagrangian Duality

Lecture Notes

Homework Sets (Assignment Box: B21, ERB 5/F)

Homework 1 (due September 28, 2023) Solution
Homework 2 (due October 19, 2023) Solution
Homework 3 (due November 7, 2023) Solution
Homework 4 (due November 28, 2023) Solution
Homework 5 (due December 13, 2023) Solution

Last Updated: December 15, 2023