PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.
Performance metrics that are used by the academia to evaluate the performance of video object detection algorithms are usually not informative for assessing whether or not the algorithms of interest are suitable (mature enough) for real-world deployment. We propose an approach to define the performance metrics that are suitable for various operational scenarios. In particular, we define four operational modes: Surveillance (alarm), situational awareness, detection and tracking. We then describe the performance metrics for the needs of each operational scenario, and explain the underlying reasoning. The metrics are compatible with the common practices for constructing in-house video datasets. We believe that these metrics provide useful insight for the usability of the algorithms. We also demonstrate that an algorithm which (at first glance) seem to have insufficient performance for deployment can be used in a real-world system (with simple post-processing) if its parameters are configured to provide high performance scores for scenario-specific metrics. We also show that the same underlying algorithm can be used for different operational scenarios if its parameters (and/or post-processing steps) are adjusted to meet the criteria based on relevant scenario-based performance metrics.
Yoldaş Ataseven
"Performance metrics with operational context for video object detection algorithms", Proc. SPIE 12276, Artificial Intelligence and Machine Learning in Defense Applications IV, 122760A (28 October 2022); https://doi.org/10.1117/12.2636106
ACCESS THE FULL ARTICLE
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.
The alert did not successfully save. Please try again later.
Yoldaş Ataseven, "Performance metrics with operational context for video object detection algorithms," Proc. SPIE 12276, Artificial Intelligence and Machine Learning in Defense Applications IV, 122760A (28 October 2022); https://doi.org/10.1117/12.2636106