Broadly applicable solutions to multimodal and multisensory fusion problems across domains remain a challenge because effective solutions often require substantive domain knowledge and engineering. The chief questions that arise for data fusion are in when to share information from different data sources, and how to accomplish the integration of information. The solutions explored in this work remain agnostic to input representation and terminal decision fusion approaches by sharing information through the learning objective as a compound objective function. The objective function this work uses assumes a one-to-one learning paradigm within a one-to-many domain which allows the assumption that consistency can be enforced across the one-to-many dimension. The domains and tasks we explore in this work include multi-sensor fusion for seismic event location and multimodal hyperspectral target discrimination. We find that our domain- informed consistency objectives are challenging to implement in stable and successful learning because of intersections between inherent data complexity and practical parameter optimization. While multimodal hyperspectral target discrimination was not enhanced across a range of different experiments by the fusion strategies put forward in this work, seismic event location benefited substantially, but only for label-limited scenarios.
The impressive performance that deep neural networks demonstrate on a range of seismic monitoring tasks depends largely on the availability of event catalogs that have been manually curated over many years or decades. However, the quality, duration, and availability of seismic event catalogs vary significantly across the range of monitoring operations, regions, and objectives. Semisupervised learning (SSL) enables learning from both labeled and unlabeled data and provides a framework to leverage the abundance of unreviewed seismic data for training deep neural networks on a variety of target tasks. We apply two SSL algorithms (mean-teacher and virtual adversarial training) as well as a novel hybrid technique (exponential average adversarial training) to seismic event classification to examine how unlabeled data with SSL can enhance model performance. In general, we find that SSL can perform as well as supervised learning with fewer labels. We also observe in some scenarios that almost half of the benefits of SSL are the result of the meaningful regularization enforced through SSL techniques and may not be attributable to unlabeled data directly. Lastly, the benefits from unlabeled data scale with the difficulty of the predictive task when we evaluate the use of unlabeled data to characterize sources in new geographic regions. In geographic areas where supervised model performance is low, SSL significantly increases the accuracy of source-type classification using unlabeled data.
The use of gradient-based data-driven models to solve a range of real-world remote sensing problems can in practice be limited by the uniformity of available data. Use of data from disparate sensor types, resolutions, and qualities typically requires compromises based on assumptions that are made prior to model training and may not necessarily be optimal given over-arching objectives. For example, while deep neural networks (NNs) are state-of-the-art in a variety of target detection problems, training them typically requires either limiting the training data to a subset over which uniformity can be enforced or training independent models which subsequently require additional score fusion. The method we introduce here seeks to leverage the benefits of both approaches by allowing correlated inputs from different data sources to co-influence preferred model solutions, while maintaining flexibility over missing and mismatching data. In this paper, we propose a new data fusion technique for gradient updated models based on entropy minimization and experimentally validate it on a hyperspectral target detection dataset. We demonstrate superior performance compared to currently available techniques and highlight the value of the proposed method for data regimes with missing data.
Low-frequency sound ≤20 Hz, known as infrasound, is generated by a variety of natural and anthropogenic sources. Following an event, infrasonic waves travel through a dynamic atmosphere that can change on the order of minutes. This makes infrasound event classification a difficult problem, as waveforms from the same source type can look drastically different. Event classification usually requires ground-truth information from seismic or other methods. This is time consuming, inefficient, and does not allow for classification if the event locates somewhere other than a known source, the location accuracy is poor, or ground truth from seismic data is lacking. Here,we compare the performance of the state of the art for infrasound event classification, support vector machine (SVM) to the performance of a convolutional neural network (CNN), a method that has been proven in tangential fields such as seismology. For a 2-class catalog of only volcanic activity and earthquake events, the fourfold average SVM classification accuracy is 75%, whereas it is 74% when using a CNN. Classification accuracies from the 4-class catalog consisting of the most common infrasound events detected at the global scale are 55% and 56% for the SVM and CNN architectures, respectively. These results demonstrate that using a CNN does not increase performance for infrasound event classification. This suggests that SVM should be the preferred classification method, as it is a simpler and more trustworthy architecture and can be tied to the physical properties of the waveforms. The SVM and CNN algorithms described in this article are not yet generalizable to other infrasound event catalogs. We anticipate this study to be a starting point for development of large and comprehensive, systematically labeled, infrasound event catalogs, as such catalogs will be necessary to provide an increase in the value of deep learning on event classification.
The use of gradient-based data-driven models to solve a range of real-world remote sensing problems can in practice be limited by the uniformity of available data. Use of data from disparate sensor types, resolutions, and qualities typically requires compromises based on assumptions that are made prior to model training and may not necessarily be optimal given over-arching objectives. For example, while deep neural networks (NNs) are state-of-the-art in a variety of target detection problems, training them typically requires either limiting the training data to a subset over which uniformity can be enforced or training independent models which subsequently require additional score fusion. The method we introduce here seeks to leverage the benefits of both approaches by allowing correlated inputs from different data sources to co-influence preferred model solutions, while maintaining flexibility over missing and mismatching data. In this work we propose a new data fusion technique for gradient updated models based on entropy minimization and experimentally validate it on a hyperspectral target detection dataset. We demonstrate superior performance compared to currently available techniques using a range of realistic data scenarios, where available data has limited spacial overlap and resolution.
The capability to discriminate low-magnitude earthquakes from low-yield anthropogenic sources, both detectable only at local distances, is of increasing interest to the event monitoring community. We used a dataset of seismic events in Utah recorded during a 14-day period (1–14 January 2011) by the University of Utah Seismic Stations network to perform a comparative study of event classification at local scale using amplitude ratio (AR) methods and a machine learning (ML) approach. The event catalog consists of 7377 events with magnitudes MC ranging from −2 and lower up to 5.8. Events were subdivided into six populations based on location and source type: tectonic earthquakes (TEs), mining-induced events (MIEs), and mining blasts from four known mines (WMB, SMB, LMB, and CQB). The AR approach jointly exploits Pg-to-Sg phase ARs and Rg-to-Sg spectral ARs in multivariate quadratic discriminant functions and was able to classify 370 events with high signal quality from the three groups with sufficient size (TE, MIE, and SMB). For that subset of the events, the method achieved success rates between about 80% and 90%. The ML approach used trained convolutional neural network (CNN) models to classify the populations. The CNN approach was able to classify the subset of events with accuracies between about 91% and 98%. Because the neural network approach does not have a minimum signal quality requirement, we applied it to the entire event catalog, including the abundant extremely low-magnitude events, and achieved accuracies of about 94%–100%. We compare the AR and ML methodologies using a broad set of criteria and conclude that a major advantage to ML methods is their robustness to low signal-to-noise ratio data, allowing them to classify significantly smaller events.
During the development of new seismic data processing methods, the verification of potential events and associated signals can present a nontrivial obstacle to the assessment of algorithm performance, especially as detection thresholds are lowered, resulting in the inclusion of significantly more anthropogenic signals. Here, we present two 14 day seismic event catalogs, a local-scale catalog developed using data from the University of Utah Seismograph Stations network, and a global-scale catalog developed using data from the International Monitoring System. Each catalog was built manually to comprehensively identify events from all sources that were locatable using phase arrival timing and directional information from seismic network stations, resulting in significant increases compared to existing catalogs. The new catalogs additionally contain challenging event sequences (prolific aftershocks and small events at the detection and location threshold) and novel event types and sources (e.g., infrasound only events and long-wall mining events) that make them useful for algorithm testing and development, as well as valuable for the unique tectonic and anthropogenic event sequences they contain.
Low frequency sound %3C zo Hz, known as infrasound, is generated by a variety of natural and anthropogenic sources. Following an event, infrasonic waves travel through a dynamic atmosphere that can change on the order of minutes. This makes infrasound event classification a difficult problem as waveforms from the same source type can look drastically different. Event classification usually requires ground truth information from seismic or other methods. This is time consuming, inefficient, and does not allow for a classification if the event locates somewhere other than a known source, the location accuracy is poor, or ground truth from seismic data is lacking. Here we compare the performance of the state of the art for infrasound event classification, support vector machine (SVM), to the performance of a convolutional neural network (CNN), a method that has been proven in tangential fields such as seismology. For a 1-class catalog consisting of only volcanic activity and earthquake events, the 4-fold average SVM classification accuracy is 75%, while it is 74% when using a CNN. Classification accuracies from the 4-class catalog consisting of the most common infrasound events detected at the global scale are 55% and 56% for the SVM and CNN architectures, respectively. These results demonstrate that using a CNN does not increase performance for infrasound event classification. This suggests that SVM should be the preferred classification method as it is a simpler and more trustworthy architecture and can be tied to the physical properties of the waveforms. The SVM and CNN algorithms described in this paper are not yet generalizable to other infrasound event catalogs. We anticipate this study to be a starting point for the development of large and comprehensive, systematically labeled, infrasound event catalogs as such catalogs will be necessary to provide an increase in the value of deep learning on event classification.
Long-term seismic monitoring networks are well positioned to leverage advances in machine learning because of the abundance of labeled training data that curated event catalogs provide. We explore the use of convolutional and recurrent neural networks to accomplish discrimination of explosive and tectonic sources for local distances. Using a 5-year event catalog generated by the University of Utah Seismograph Stations, we train models to produce automated event labels using 90-s event spectrograms from three-component and single-channel sensors. Both network architectures are able to replicate analyst labels above 98%. Most commonly, model error is the result of label error (70% of cases). Accounting for mislabeled events (~1% of the catalog) model accuracy for both models increases to above 99%. Classification accuracy remains above 98% for shallow tectonic events, indicating that spectral characteristics controlled by event depth do not play a dominant role in event discrimination.