SuperRAENN--a new SN classifier

After a year of work, I am so happy to announce my new paper titled, SuperRAENN: A Semi-supervised Supernova Photometric Classification Pipeline Trained on Pan-STARRS1 Medium Deep Survey Supernovae. This paper appears with a companion paper led by Griffin Hosseinzadeh titled, Photometric Classfication of 2315 Pan-STARRS1 Supernovae with Superphot. Both of these papers build off work discussed in a previous blog post.

This paper presents a large set of supernova-like light curves from the Pan-STARRS1 Medium Deep Survey (PS1 MDS). These were originally studied in D. Jones et al. (2018). PS1 MDS ran from 2010-2014 over 70-square degrees in griz filters. PS1 MDS is kind of like a “mili”-LSST, in that it has very similar survey properties but 1/1000th of the discovery power.

Of the approximately 5,000 SN-like transients we present, about 560 have spectroscopic classification. In other words, someone took a spectrum of the supernova and was able to match it to a known SN type. The Harvard team (before I joined!) led a large effort to target the host galaxies of these SN-like transients. After significant validation, we confidently present the redshift measurements of approximately 2,300 transients.

In addition to a data release, we present a new photometric classifier for SNe, which I have named SuperRAENN. “RAENN” stands for recurrent autoencoder neural network. SuperRAENN uses a semi-supervised approach to classify transients.

First, we train a novel neural network architure on unlabeled SNe. Very briefly, our neural network reads in light curves one data point at a time (in griz filters). We “encode” the light curve into just a few numbers – similar to fitting a physical model to a light curve! What is cool about our neural network determines the model from the data. After encoding the light curve, the neural network then makes this “model” into a callable function by appending time values to the encoded light curve. Through this method, we are actually able to interpolate/extrapolate light curves using our model (see below). We actually don’t really use this interpolation/extrapolation in our paper, but we plan to use it in future work using live streaming data. SuperRAENN is open source and available on GitHub! (It’s also my first pip installable package, which was a ton of fun. I greatly benefited from this tutorial which Griffin sent me!)

Next, we move on to our “supervised” step, in which we use the spectroscopically labeled SN data to train a random forest to predict the SN classes. We use hand-selected features (like peak luminosity and duration) in addition to our RAENN features in order to achieve the highest confidence classifications. We use this step to classify the remaining 1,800 SNe with redshift measurements.

We do quite a few tests to see how reliable our classifications are. I think a nice summary is simply comparing our sample to the ZTF Bright Transient Survey sample, shown below. In short, we clearly have some small biases in our classifier, but we actually seem to understand these biases. If we correct for them, we seem to be in good agreement with the ZTF results.

I think a really fun personal aspect of this paper is how many times I found myself writing “which will be explored in future work”. This paper presents many new ideas, but there are still so many questions left to be answered! I’m really excited to keep pushing on how we can use machine learning and data driven methods to push time domain astrophysics forward!

On another personal note, this is the last chapter of my thesis! I recently moved to NYC to start a postdoc at Columbia (primary) and the Flatiron Institute. So excited to spend this year meeting new collaborators!