The Unforgiving Asymptote of Chasing Machine Learning Gains

1. Executive Summary

2. Case Study 1: Synthesizing Robustness Introduction

2.1. Rare Aircraft Detection Performance

Figure 1. Scores over time for the RarePlanes project (click figure for interactive plot).
Figure 2. Scores over time for the RarePlanes project, with the “work stoppage” included (click figure for interactive plot).
Figure 3. Scores as a function of person-hours (click figure for interactive plot).

2.2 Optimizing Efforts

3. Case Study 2: Where in the World

Figure 4. Model performance (as measured by top-percentile score) versus time during the Where in the World project. The timespan shown is approximately one year. Each colored line denotes a consistent set of test conditions.

4. Conclusions

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store