Posts by Collection

Illustration of the inverse and forward rendering between 2D visual features (produced by DINOv2) and a 3D Gaussian Splatting scene. In the inverse rendering (or uplifting) phase, features are created for each 3D Gaussian by aggregating coarse 2D features over all viewing directions. For forward rendering, the 3D features are projected on any given viewing direction as in regular Gaussian Splatting.

PCA of image features for 30 classes of the CUB Bird dataset. Distilling a large pretrained teacher (top, left) to train a small task-specific student model (top, right) results in a better clustering of the representations compared to simply finetuning the student on the task (bottom, right). Distillation can be improved by using a Mixup-inspired class-agnostic data augmentation based on Stable Diffusion (grey features in teacher plot).

The three most likely and least likely augmentations for different domains, as estimated by SLACK - a method for automatically learning optimal data augmentation policies.

Heart segmentation in a 3D ultrasound image using a pretrained model

Phylogenetic tree reconstruction, colored by the three main cell types.

Mesh representation of a drone optimized using Geometric Deep Learning.

SLACK: Stable Learning of Augmentations with Cold-start and KL regularization

Published in IEEE / CVF Computer Vision and Pattern Recognition Conference (CVPR), 2023

Juliette Marrie, Michael Arbel, Diane Larlus, Julien Mairal

This paper proposes a method for automatically learning optimal data augmentation policies.

Download Paper

On Good Practices for Task-Specific Distillation of Large Pretrained Visual Models

Published in Transactions on Machine Learning Research (TMLR), 2024

Juliette Marrie, Michael Arbel, Julien Mairal, Diane Larlus

This paper delineates good practices for leveraging current large pretrained visual models to train a small model on a specific task.

Download Paper

LUDVIG: Learning-free Uplifting of 2D Visual features to Gaussian Splatting scenes

Published in arXiv preprint, 2024

Juliette Marrie, Romain Ménégaux, Michael Arbel, Diane Larlus, Julien Mairal

This paper proposes an efficient, learning-free approach for transferring 2D visual representations into 3D Gaussian Splatting scenes.

Download Paper