Course ID ENS Lyon: Inverse problems and parcimony - MATH5232

Course ID ECL: Parcimonie et grande dimension - S22_ING_S09_MIR3_1

Course Description

Sparsity and convexity are ubiquitous notions in Machine Learning and Statistics. In this course, we study the mathematical foundations of some powerful methods based on convex relaxation: L1-regularisation techniques in Statistics and Signal Processing. The theoretical part of the course will focus on the guarantees of these algorithms under the sparsity assumption. The practical part of this course will present the standard solvers of these learning problems.

Bibliography

  • [Foucart] A Mathematical Introduction to Compressive Sensing, Simon Foucart and Holger Rauhut, Applied and Numerical Harmonic Analysis (ANHA), Springer
  • [Bubeck] Convex Optimization: Algorithms and Complexity, Sébastien Bubeck, Foundations and Trends in Machine Learning.
  • [blondel] The Elements of Differentiable Learning, Mathieu Blondel and Vincent Roulet.
  • [Wainwright] High-Dimensional Statistics: A Non-Asymptotic Viewpoint, Martin J. Wainwright, Cambridge University Press
  • [Boyd] Convex Optimization, Stephen Boyd and Lieven Vandenberghe, Cambridge University Press
  • [d'Aspremont] Acceleration methods, Alexandre d’Aspremont, Damien Scieur and Adrien Taylor.

Course Organizers

Yohann
De Castro
Rémi
Gribonval

Projects

The guidelines can be found in this document. The couples ([ref paper], {names}) are to be announced.
Based on the references:
  • [SR-Lasso] C. Poon and G. Peyré, "Super-Resolved Lasso" (https://arxiv.org/abs/2311.09928)
  • [Prox-P&P]S. Hurault, A. Leclaire, and N. Papadakis, "Proximal Denoiser for Convergent Plug-and-Play Optimization with Nonconvex Regularization" (https://arxiv.org/abs/2201.13256)
  • [Gap Safe] E. Ndiaye, O. Fercoq, A. Gramfort, and J. Salmon, “Gap Safe Screening Rules for Sparsity Enforcing Penalties”, J. Mach. Learn. Res., 2017.
  • [Low-Rank NSP] M. Kabanava, R. Kueng, and H. Rauhut, “Stable low-rank matrix recovery via null space properties,” Information and Inference, vol. 5, no. 4, pp. 405–441, 2016.
  • [FW] Q. Denoyelle, V. Duval, G. Peyré, and E. Soubies, “The sliding Frank–Wolfe algorithm and its application to super-resolution microscopy,” Inverse problems, vol. 36, no. 1, p. 014001, Jan. 2020.
  • [Rep.IR] Mairal, J., Elad, M., & Sapiro, G. (2008). Sparse Representation for Color Image Restoration. IEEE Transactions on Image Processing, 17(1), 53–69. http://doi.org/10.1109/tip.2007.911828
  • [Group] Jacob, L., Obozinski, G., & Vert, J. P. (2009, June). Group lasso with overlap and graph lasso. In Proceedings of the 26th annual international conference on machine learning (pp. 433-440). http://members.cbio.mines-paristech.fr/~ljacob/documents/overlasso.pdf
  • [Sketch] Two papers on Sketching:
    Antoine Chatalic, Rémi Gribonval, Nicolas Keriven. Large-Scale High-Dimensional Clustering with Fast Sketching . In ICASSP, 2018.
    Rémi Gribonval, Antoine Chatalic, Nicolas Keriven, Vincent Schellekens, Laurent Jacques, Philip Schniter. Sketching Datasets for Large-Scale Learning (long version). arXiv preprint arXiv:2008.01839.

Past exams

2023_exam

Clarifications

Nous avons deux cours de Master « Optimisation et Représentation Parcimonieuse » (ORP/Master) et « Parcimonie et Grande Dimension » (PGD/Master), et un cours 3A Centrale « Parcimonie et Grande Dimension » (PGD/Option)

Il y a deux examens :
Exam1 -- Lundi 25 mars 2024 de 14h à 16h pour tout le monde (centraliens, normaliens, Lyon 1, master ou non, option ou non)
Exam2 -- Jeudi 28 mars 2024 de 8h à 10h pour les centraliens de l'option (en master ou non)

Dans PGD/Option nous avons donc deux examens pour la note finale = (Exam1+Exam2)/2
Dans ORP/Master, la note de master MeA sera basée sur l’examen Exam2 et un examen/BE à venir (3 élèves à ma connaissance)
Dans PGD/Master, la note du premier examen Exam1 et une note de projets (qui a eu lieu hier) servent pour la note finale = (Exam1+Projets)/2

Les centraliens en Master GRAF auront une note GRAF = (Exam1+Projets)/2

On a donc un seul cours, le cours PGD, aux publics et notes différentiés.

Pour le lundi 25 mars 2024, tout le monde est convoqué, les centraliens et les masters MeA, MA, et GRAF.
Pour le jeudi 28 mars 2024, seuls les centraliens sont convoqués.

Class Time and Location

Winter quarter (January - March, 2024).
Lecture: Monday afternoon 2:00-5:15
Ecole Centrale Lyon

Contact Info

Yohann: yohann.de-castro@ec-lyon.fr
Rémi: remi.gribonval@inria.fr
Students involvedDate and timeLecturerDescriptionMaterials and references
S22_ING_S09_MIR3_1 and MATH5212
-> Full group		 Jan., 29
(2-5pm)		 Rémi Lecture 1 [Foucart] Chapter 2
S22_ING_S09_MIR3_1 and MATH5212
-> Full group		 Feb., 5
(2-5pm)		 Yohann Lecture 2 [Boyd] Part I;
[Foucart] Chapter 3;
S22_ING_S09_MIR3_1 and MATH5212
-> Full group
 Feb., 12
(2-5pm)		 Rémi Lecture 3
S22_ING_S09_MIR3_1 and MATH5212
-> Full group
 Feb., 19
(2-5pm)		 Yohann Lecture 4 [Foucart] Chapters 4 and 6;
None Feb., 26 None BREAK
S22_ING_S09_MIR3_1 and MATH5212
-> Full group
 Mar., 4
(2-5pm)		 Rémi Lecture 5
S22_ING_S09_MIR3_1 and MATH5212
-> Full group		 March, 18
(2-5pm)		 Examinators: Remi and Yohann Projects Guidelines
S22_ING_S09_MIR3_1 and MATH5212
-> Full group		 March, 25
(2-5pm)		 Examinator: ECL's agent Exam