Data dependency on measurement uncertainties in speaker recognition evaluation

Jin Chu Wu; Alvin F. Martin; Craig S. Greenberg; Raghu N. Kacker

doi:10.1117/12.918467

7 May 2012 Data dependency on measurement uncertainties in speaker recognition evaluation

Jin Chu Wu, Alvin F. Martin, Craig S. Greenberg, Raghu N. Kacker

Proceedings Volume 8382, Active and Passive Signatures III; 83820D (2012) https://doi.org/10.1117/12.918467
Event: SPIE Defense, Security, and Sensing, 2012, Baltimore, Maryland, United States

Abstract

The National Institute of Standards and Technology conducts an ongoing series of Speaker Recognition Evaluations (SRE). Speaker detection performance is measured using a detection cost function defined as a weighted sum of the probabilities of type I and type II errors. The sampling variability can result in measurement uncertainties. In our prior study, the data independency was assumed in using the nonparametric two-sample bootstrap method to compute the standard errors (SE) of the detection cost function based on our extensive bootstrap variability studies in ROC analysis on large datasets. In this article, the data dependency caused by multiple uses of the same subjects is taken into account. The data are grouped into target sets and non-target sets, and each set contains multiple scores. One-layer and two-layer bootstrap methods are proposed based on whether the two-sample bootstrap resampling takes place only on target sets and non-target sets, or subsequently on target scores and non-target scores within the sets, respectively. The SEs of the detection cost function using these two methods along with those with the assumption of data independency are compared. It is found that the data dependency increases both estimated SEs and the variations of SEs. Some suggestions regarding the test design are provided.

Citation Download Citation

Jin Chu Wu, Alvin F. Martin, Craig S. Greenberg, and Raghu N. Kacker "Data dependency on measurement uncertainties in speaker recognition evaluation", Proc. SPIE 8382, Active and Passive Signatures III, 83820D (7 May 2012); https://doi.org/10.1117/12.918467

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available