New preprint out! Neural collapse is a phenomenon in deep learning where features of a classifier converge to the vertices of a simplex as training progresses. We studied this phenomenon where the number of classes exceeds the dimension–so this emergent structure is no longer a simplex, but a spherical code.

Neural collapse is a phenomenon in deep learning where features of a classifier converge to the vertices of a simplex as training progresses. We studied this phenomenon where the number of classes exceeds the dimension.

They say journal acceptances are like buses¹–for a long while you don’t see any, and then two come along at once. To be more specific, this week two of my papers were accepted to the journals NMR in Biomedicine (NMRB) and SIAM Multiscale Modeling and Simulation (SIAM MMS) respectively.

We study how much information classical statistical estimators hold about underlying random variables in myelin water imaging, and show that input layer regularization with automated hyperparameter tuning can outperform copious deep learning alone.

We propose algorithms for learning collective variables that respect permutational symmetry, with case studies on Lennard-Jones clusters using quantitative coarse-graining theory and tools from geometric data science.

Last month marked the culmination of a two major projects regarding collective variable discovery, a fundamental interdisciplinary problem in drug discovery, computational statistical physics, and stochastic processes. From a probabilistic perpsective, this problem asks: how can we map a stochastic process to low dimensions and still preserve its statistics? In two case study-style papers on the butane molecule and Lennard-Jones clusters we provide some answers by resorting to quantitive coarse graining theory and proposing algorithms that use some of my favourite tools from geometric data science.

We turn Legoll and Lelievre’s quantitative coarse-graining theory into an algorithm for learning collective variables that preserve transition rates in molecular systems, with a case study on butane.

Two papers I’ve been working on are out. One tells a story about how you can combine classical and deep learning methods for magnetic resonance imaging in the brain. The other one shows how to use the Neumann eigenvectors of subgraphs for dimension reduction–with nearly isometric embeddings!

We propose a new dimensionality reduction technique termed Neumann eigenmaps which uses landmarks to enhance standard spectral embedddings, while preserving an isometry property of diffusion maps.

We leverage the approximation theory of target measure diffusion maps to obtain sharp error estimates with explicit prefactors, yielding concrete accuracy gains for rare-event quantification via the committor problem in molecular dynamics.

We study fundamental limits on explaining neural network decisions through descrambling methods, identifying when such approaches can and cannot recover meaningful structure from learned representations.

Our paper on Short Time Fourier Transform (STFT) phase retrieval was recently published at the 34th European Signal Processing Conference! We showed that neural networks can perform speech phase retreival given far fewer samples than what is mathematically required. Read it here.

We show that neural networks can reconstruct speech from undersampled Short Time Fourier Transform magnitude measurements, recovering phase information from far fewer samples than classical methods require.

I participated in the super exciting Mathematical Research Community (MRC) on Explainable, Interpretable, and Adversarial AI at Beaver Hollow, NY! A lot of things were discussed–including the capabilities of transformers, latent geometries of language models, and neural collapse. You can hear more on these matters at the upcoming Joint Math Meetings in Seattle at the special session for this MRC.

‘Tis the season to drop preprints! My work titled Sharp error estimates for target measure diffusion maps with applications to the committor problem with Dr. Maria Cameron and Luke Evans is submitted and up on arXiv. This paper is about leveraging the approximation theory of diffusion maps to gain concrete speedups in accuracy for applications in molecular dynamics.

I attended the Measure Transport, Diffusion Processes and Sampling Workshop at the Flatiron Institute! My poster featured new results on Target Measure Diffusion Maps (TMDmap), including a stability estimate for Dirichlet BVPs using TMDmaps and an application to machine learning-based rare event quantification in the butane molecule.

I recently presented a poster titled Error analysis of Target Measure Diffusion Maps and applications to transition path theory at the Optimal Transport for Data Science workshop at ICERM, Brown University!

Next meeting: December 4th, 3:00 pm, Kirwan Hall 1310. Speaker: Meenakshi Krishnan

I spent last week attending the Applied Harmonic Analysis and Machine Learning Summer School at the Machine Learning Genoa Center in Genova, Italy!

I worked on creating and implementing a new algorithm for using information theory to make phylogenetic trees from aligned DNA sequences at the Cummings Lab in UMD.

This project motivated many interesting biological questions, but here is a mathematical one that caught my fancy: Given a function \(f: \{0,1\}^n \to \mathbb{R}\), can one find a polynomial time algorithm to compute the global maximum of \(f\)? One might be tempted to say “ah this is just integer programming” but what happens when there is no meaningful extension of \(f\) to \([0,1]^n\)? This problem arises in computing the optimal split in a tree, where we must maximize the function \(\vert A \vert H(A) + \vert A^c \vert H(A^c)\) over all possible partitions \(A \sqcup A^c = X\) of a finite set \(X\). Here \(H(A)\) is the (empirical) entropy of \(A\). Email me if you have suggestions and find out more about this project here.

The initial commit of projects is in (hopefully with many more to come). Particular highlights include my undergraduate thesis, code for a couple of projects, and a stash of notes meant for the cramming undergraduate (aka me 2 years ago). Check them out here.

Finally the website is up! It looks a bit minimal, but that’s kind of the style I meant to go for. I used this template designed by Marc Weitz to start with. The wonders of open source never cease to amaze.

We construct and analyze Sobolev orthogonal polynomials on the Sierpinski gasket, extending classical orthogonal polynomial theory to fractal domains.