Online learning and applied data analysis

Many large datasets are best modelled as a sum of nonnegative parts: a face is a sum of feature patches, a temperature record a sum of seasonal patterns, an epidemic curve a sum of waves. Nonnegative matrix factorization makes this precise — it represents a data matrix \(X \approx W H\) with both factors entrywise nonnegative — but classical NMF needs the entire dataset in memory at once.

Online NMF processes data as it arrives, updating the dictionary \(W\) and the activations \(H\) one mini-batch at a time. With Hanbaek Lyu, Deanna Needell, and Christopher Strohmeier we worked out applications of online NMF to streaming image and time-series data, including a temporal dictionary-learning scheme that captures correlations across multiple correlated series simultaneously. We applied the same machinery to early COVID-19 case counts: jointly learning short-evolution dictionaries across countries lets one extrapolate latent correlation structure into a short-term forecast.

I am interested in convergence guarantees for online matrix factorization under realistic dependence — Markovian and weakly mixing data streams — and in the use of these methods as inexpensive baselines and feature extractors alongside heavier deep-learning pipelines.

Further reading.

• Applications of Online Nonnegative Matrix Factorization to Image and Time-Series Data, with H. Lyu, D. Needell, C. Strohmeier. arXiv:2011.05384.
• COVID-19 Time-series Prediction by Joint Dictionary Learning and Online NMF, with H. Lyu, C. Strohmeier, D. Needell. arXiv:2004.09112.