One of the best-known secrets of machine learning (ML) is that the most reliable way to get more accurate models is by getting more training data and more accurate labels. Learn about the ways that our engineering team build systems to handle the volume of data required for our state-of-the-art security model.
Generating up to date, well labeled datasets for machine learning (ML) security models is a unique engineering challenge, as large data volumes, complexity of labeling, and constant concept drift makes it difficult to generate effective training datasets. Here we describe a simple, resilient cloud infrastructure for generating ML training and testing datasets, that has enhanced the speed at which our team is able to research and keep in production a multitude of security ML models.