PhD Student in Deep Learning at Cnam Paris - AI/ML Consultant
Office 37.0E.36
2 rue Conté
Paris, France
I am a PhD student in Deep Learning at Le Cnam in the Machine Learning team advised by Prof. Nicolas Thome and Clément Rambour. The focus of my PhD is on the efficient adaptation of deep learning foundation models with a particular interest in improving their robustness. Lately, I have focused on improving the trade-off between accuracy and robustness for few-shot learning methods, e.g. prompt learning, of vision-language foundation models like CLIP.
Prompt learning has been widely adopted to efficiently adapt vision-language models (VLMs), e.g. CLIP, for few-shot image classification. Despite their success, most prompt learning methods trade-off between classification accuracy and robustness, e.g. in domain generalization or outof-distribution (OOD) detection. In this work, we introduce Global-Local Prompts (GalLoP), a new prompt learning method that learns multiple diverse prompts leveraging both global and local visual features. The training of the local prompts relies on local features with an enhanced vision-text alignment. To focus only on pertinent features, this local alignment is coupled with a sparsity strategy in the selection of the local features. We enforce diversity on the set of prompts using a new “prompt dropout” technique and a multiscale strategy on the local prompts. GalLoP outperforms previous prompt learning methods on accuracy on eleven datasets in different few shots settings and with various backbones. Furthermore, GalLoP shows strong robustness performances in both domain generalization and OOD detection, even outperforming dedicated OOD detection methods. Code and instructions to reproduce our results will be open-sourced.
@article{LafonRamzi2024,author={Lafon, Marc and Ramzi, Elias and Rambour, Clément and Audebert, Nicolas and Thome, Nicolas},journal={Proceedings of the 18th European Conference on Computer Vision, Milan, Italy, 2024},title={{GalLoP: Learning Global and Local Prompts for Vision-Language Models}},year={2024},}
ICML
Hybrid Energy Based Model in the Feature Space for Out-of-Distribution Detection
Marc Lafon, Elias Ramzi, Clément Rambour, and
1 more author
Proceedings of the 40 th International Conference on Machine
Learning, Honolulu, Hawaii, USA. PMLR 202, 2023, 2023
Out-of-distribution (OOD) detection is a critical requirement for the deployment of deep neural networks. This paper introduces the HEAT model, a new post-hoc OOD detection method estimating the density of in-distribution (ID) samples using hybrid energy-based models (EBM) in the feature space of a pre-trained backbone. HEAT complements prior density estimators of the ID density, e.g. parametric models like the Gaussian Mixture Model (GMM), to provide an accurate yet robust density estimation. A second contribution is to leverage the EBM framework to provide a unified density estimation and to compose several energy terms. Extensive experiments demonstrate the significance of the two contributions. HEAT sets new state-of-the-art OOD detection results on the CIFAR-10 / CIFAR100 benchmark as well as on the large-scale Imagenet benchmark. The code is available at: github.com/MarcLafon/heatood.
@article{Lafon2023,author={Lafon, Marc and Ramzi, Elias and Rambour, Clément and Thome, Nicolas},journal={Proceedings of the 40 th International Conference on Machine
Learning, Honolulu, Hawaii, USA. PMLR 202, 2023},title={{Hybrid Energy Based Model in the Feature Space for Out-of-Distribution Detection}},year={2023},}
ICML
Beyond First-Order Uncertainty Estimation with Evidential Models for Open-World Recognition
Charles Corbière, Marc Lafon, Nicolas Thome, and
2 more authors
ICML Workshop on Uncertainty and Robustness in Deep Learning, 2021
In this paper, we tackle the challenge of jointly quantifying in-distribution and out-of-distribution (OOD) uncertainties. We introduce KLoS, a KL-divergence measure defined on the class-probability simplex. By leveraging the second-order uncertainty representation provided by evi-dential models, KLoS captures more than existing first-order uncertainty measures such as predic-tive entropy. We design an auxiliary neural network , KLoSNet, to learn a refined measure directly aligned with the evidential training objective. Experiments show that KLoSNet acts as a class-wise density estimator and outperforms current uncertainty measures in the realistic context where no OOD data is available during training. We also report comparisons in the presence of OOD training samples, which shed a new light on the impact of the vicinity of this data with OOD test data.
@article{Corbiere2021,author={Corbière, Charles and Lafon, Marc and Thome, Nicolas and Cord, Matthieu and Pérez, Patrick},journal={ICML Workshop on Uncertainty and Robustness in Deep Learning},title={{Beyond First-Order Uncertainty Estimation with Evidential Models for Open-World Recognition}},year={2021},}
