Speakers and Workshop Details

Zhenyu LIAO
Huazhong University of Science and Technology (HUST)

Title: Random Matrix Methods for Machine Learning: An Application to “Lossless” Compression of Large and Deep Neural Networks

Abstract:

The advent of the Big Data era has triggered a renewed interest in large-dimensional machine learning (ML) and deep neural networks (DNNs). These methods, being developed from small-dimensional intuitions, often behave dramatically different from their original designs and tend to be inefficient on large-dimensional datasets. By assuming both dimension and size of the datasets to be large, recent advances in random matrix theory (RMT) provide novel insights, allowing for a renewed understanding and the possibility to design more efficient machine learning approaches, thereby opening the door to completely new paradigms.

In this talk, we will start with the “curse of dimensionality” phenomenon in high dimensions and highlight how these counterintuitive phenomena arise in ML practice when large-dimensional data are considered. By focusing on the concrete problem of compressing both shallow and deep neural networks, we show how the proposed theory can be applied to design efficient DNN compression schemes with strong performance guarantee.

Biography:

Zhenyu Liao received his M.Sc. in Signal and Image Processing in 2016, and his Ph.D. in Computer Science in 2019, both from University of Paris-Saclay, France. In 2020 he was a postdoctoral researcher with the Department of Statistics, University of California, Berkeley. He is currently an associated professor with Huazhong University of Science and Technology (HUST), China. His research interests are broadly in machine learning, signal processing, random matrix theory, and high-dimensional statistics. He published more than 20 papers on top-tier machine learning conferences such as ICML, NeurIPS, ICLR, COLT, AISTATS, etc., and he co-authored the book “Random Matrix Methods for Machine Learning.” He is the recipient of the 2021 Wuhan Youth Talent Fellowship, and the 2019 ED STIC Ph.D. Student Award of University of Paris-saclay, France.

Tiefeng JIANG
University of Minnesota

Title: Distances between Random Orthogonal Matrices and Independent Normals

Abstract:

We study the distance between Haar-orthogonal matrices and independent normal random variables. The distance is measured by the total variation distance, the Kullback-leibler distance, the Hellinger distance and the Euclidean distance. They appear different features. Optimal rates are obtained. This is a joint work with Yutao Ma.

Biography:

Prof. Jiang obtained his Ph.D. degree from Stanford University in 1999. His research covers Random matrices, Random graphs, Probability, Symmetric polynomials, High-dimensional statistics, Connections to physics and computer science.

Cosme LOUART
GIPSA Lab, Université Joseph Fourier

Title: Rigorous non-Gaussian Approach of the Statistics of Convex Problems in Machine Learning

Abstract:

Convex problem formulation is a very convenient setting in machine learning algorithm because the uniqueness of the solution ensure the reproducibility and the efficiency of the computation. However, the implicit definition of the solution and the non-linearity makes the study of the statistics and the performances quite difficult.

This talk takes the M-estimators as a toy model for the study of convex problems. Some heuristics already provided close form formulas for the statistics, and we will further provide in the presentation mathematical tools to prove rigorously those heuristics. The study goes beyond the traditional Gaussian assumption thanks to a concentration of the measure hypothesis on the data that has the two advantages of (i) representing a very wide range of realistic applications (ii) sharing with the Gaussian distribution the concentration properties necessary for the statistical inferences. Concentration of the measure perfectly fits the random matrix theory requirements which allows to tackle smoothly and rigorously the high dimension of the problem.

Biography:

After a diploma from ENS Paris, Cosme Louart received a M.Sc. on Machine learning (MVA) at ENS Paris Saclay. Then he started a Ph.D. at the GIPSA lab in Grenoble that will be defended on the beginning of next year. He published several articles in mathematics and machine learning journals (ICML, AISTATS, IEEE, Annals of applied probability).

Zeng LI
Southern University of Science and Technology

Title: Asymptotic Normality for Eigenvalue Statistics of a General Sample covariance Matrix When p/n →∞ and Applications

Abstract:

The asymptotic normality for a large family of eigenvalue statistics of a general sample covariance matrix is derived under the ultra-high dimensional
setting, that is, when the dimension to sample size ratio p/n→∞. Based on this CLT result, we extend the covariance matrix test problem to the new
ultrahigh dimensional context and apply it to test a matrix-valued white noise. Simulation experiments are conducted for the investigation of finite-sample
properties of the general asymptotic normality of eigenvalue statistics, as well as the two developed tests.

Biography:

Dr. Li is currently an associate professor in the Department of Statistics and Data Science, Southern University of Science and Technology. Previously she was a postdoctoral fellow in the Department of Statistics at the Pennsylvania State University. Dr. Li obtained her Ph.D. degree from the Department of Statistics and Actuarial Science at the University of Hong Kong. Dr. Li’s research covers random matrix theory and high dimensional statistics.​