CERN Machine Learning
A quick look into my personal involvement with bringing state-of-the-art Machine Learning techniques to High Energy Physics
My Projects
Generative Adversarial Networks for Physics Simulation
My main research interest during the final years of my PhD revolved around training Generative Adversarial Networks (GANs) to speed up event simulation in Particle Physics.
Large simulated datasets are crucial in Particle Physics, just like in Nuclear Physics, Cosmology and other Physical Sciences, for interpreting results of ongoing experiments and for estimating yields of different proposed experimental setups. However, the complex simulation packages are very computationally expensive and time consuming. GANs are a deep learning solution that circumvents these problems by learning the data distribution and generating new samples from it.
In order to take advantage of the pixel-level symmetries of Physics events in our detectors while explicitly inducing location-based feature detection, we introduced an architecture modification to the ACGAN formulation which we call LAGAN (Location-Aware Generative Adversarial Network). Details of this study can be found in this paper, which I presented at the 2017 CERN Interexperimental Machine Learning (IML) Workshop:
The LAGAN unit can be use to construct more complex GAN systems for realistic simulations of subatomic particles interacting with multi-layer, heterogeneous, 3D calorimeter detectors. You can find more details in our CaloGAN paper (click).
The summary of my work carried out thanks to the generosity of the HEP Center for Computational Excellence can be found at this link (click).
Parameter Tuning in Monte Carlo Simulators with Surrogate Models and Gaussian Processes
At the 2018 IML workshop at CERN on April 10, 2018, I presented an update on the last (unpublished) project I worked on while in graduate school, titled "Adversarial Tuning of Perturbative Parameters in Non-Differentiable Physics Simulators". It revolved around the use of deep learning for Monte Carlo generator tuning. This method for tuning perturbative parameters in Monte Carlo simulation used a classifier loss in high dimensions to separate datasets generated with different input parameteres by analyzing batches of output examples, until the generated dataset with tunable input parameters looked indistinguishable from the ground-truth dataset with unknown input parameters. We used an LSTM trained on the radiation pattern inside jets to learn the parameters of the final state shower in the Pythia Monte Carlo generator. This represented a step forward compared to unidimensional distributional template-matching methods.
Machine Learning for b-Tagging
An ongoing project of mine is to contribute to the improvement of the $b$-tagging pipeline by providing scalable Machine Learning solutions.
In collaboration with colleagues from UC Irvine, Stanford and UGeneva, I've developed recurrent and convolutional architectures to improve upon the current impact parameter based tagging techniques. Drawing inspiration from NLP, I introduced the use of bidirectional recurrent units and 1D convolutional layers for jet tagging using the characteristics of the tracks that compose them. The first public results on this project are available in the ATLAS PUB-NOTE-2017-003, which was presented at the 2017 Connecting the Dots Workshop.
Updated performance results are presented, in the larger context of a larger b-tagging algorithms performance report, in the ATLAS PUB-NOTE-2017-013.
My authorship qualification task for the ATLAS collaboration also included the optimization of a lower level tagger using Boosted Decision Trees.
Dark Knowledge for the Matrix Element Method (MEM)
The MEM is a Physics-driven, computationally expensive technique which is widely used in HEP due to the clear physical insights it provides. It directly connects theoretical and experimental Physics, by allowing us to estimate several parameters at once using integration over phase space and detector response. Its output is usally combined with high level features, such as the 4-vectors of the particles in the event and the missing transverse energy, as inputs to a NN in order to augment the raw features with a Physics-instilled engineered variable.
Our goal is to use Dark Knowledge to construct a net that would be able to mimic the performance of MEM, while reducing computational requirements. This net is expected to capture not only the performance but also the overall distribution shape of the output of MEM, in order to preserve its discriminating power as an input feature to the original NN. Our benchmark is the $ttH$, with $H \rightarrow bb$ ATLAS analysis.
Deep Learning for Top and Boson Tagging in Boosted Topologies
During my first year at CERN, I developed a boosted boson tagger using Deep Learning for binary and multiclass classification purposes for the $W’\rightarrow WZ$ analysis, as well as a boosted top tagger which outperformed ordinary substracture variable scans.Additionally, I investigated the model's ability to learn high order correlations between tracking and calorimeter variables, and I performed variable selection studies and hyperparameters grid searches.
At the time, I used a feed-forward neural network from the AGILEPack library, with stacked auto-encoders and unsupervised pre-training.
My initial work later resulted in the published results outlined in the ATLAS ATL-PHYS-PUB-2017-004.
RNN for Event Classification
As a member of the $hh\rightarrow \gamma \gamma bb$ analysis group, I'm looking for evidence of the production of Higgs pairs decaying to two photons and a pair of $b$ quark-antiquark. I use Machine Learning techniques inspired by Natural Language Processing to explore an event-level approach to Physics analysis. I design, train and test Recurrent Neural Networks using Keras, in order to classify events with variable numbers of particles.
This method allows for a holistic study of each event, by combining event-level variables with jet and photon specific features, without limiting the number of jets or particles under consideration. This represents a huge step forward from classic cut-based analyses.
Machine Learning and Artificial Intelligence at CERN and in HEP
If you want to learn more about the people involved in Machine Learning research at CERN and in High Energy Physics, consult this opt-in directory.
You may find more information about the community of scientists involved in bringing Machine Learning methods to the physical sciences by following the efforts of the Dark Machines collective, the HEP Software Foundation (HSF), the CERN Interexperimental Machine Learning (IML) group, and the Fermilab Machine Learning group, among others.
