Failure-to-enroll/-to-extract

[FlatErik90]

Hi,

I have a question regarding the evaluation and reporting of failure-to-enroll and failure-to-extract. Your report on page 40 mentions that failure-to-extract (the inability to produce a template for search) is not reported. However, the table header in the results section reads "Failure to extract features." Could you clarify if this actually refers to failure-to-enroll, which is the inability to produce a template for enrollment? Additionally, are the same evaluation metrics used for both Face and Iris in IREX10, including the failures mentioned above?

[paddygr]

Page 40 in which report (?). In our evaluations a failure to convert an image to a template will elevate 1:1 FNMR or 1:N FNIR if the template is used in a mated comparison. It will not cause increase 1:1 FMR or 1:N FPIR. The effect is the same as if the template had been produced but it always produces a low similarity score. From: Erik ***@***.***> Date: Monday, May 13, 2024 at 11:34 AM To: usnistgov/frvt ***@***.***> Cc: Subscribed ***@***.***> Subject: [usnistgov/frvt] Failure-to-enroll/-to-extract (Issue #133) Hi, I have a question regarding the evaluation and reporting of failure-to-enroll and failure-to-extract. Your report on page 40 mentions that failure-to-extract (the inability to produce a template for search) is not reported. However, the table header in the results section reads "Failure to extract features." Could you clarify if this actually refers to failure-to-enroll, which is the inability to produce a template for enrollment? Additionally, are the same evaluation metrics used for both Face and Iris in IREX10, including the failures mentioned above? — Reply to this email directly, view it on GitHub<#133>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ADVZFNITMKXX2P4N3YTSIY3ZCDMRZAVCNFSM6AAAAABHUOQJ5GVHI2DSMVQWIX3LMV43ASLTON2WKOZSGI4TGMJVHE4DSMY>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[xsacha]

I came across this issue before. The wording is a bit misleading, and it suggests it isn't counted.
Think of it logically: Of course it is counted in the results. Otherwise, it is in your interest to fail to extract any marginally difficult images.

It is counted and will be considered the worst score, so it is in your interest to actually extract it usually. You get given enough time to try various methods of extraction.

[FlatErik90]

@paddygr : I'm referring to the latest FRTE 1:N report, specifically section 3.5, which states, "We do not report FTX...". Given this, I assume the numbers reported in Table 15 under the column "failure to extract features" are only FTE – the proportion of templates that could not be generated during enrollment. My follow-up question concerns the IREX10 submissions: Does the FTE Rate reported on the table on the website also refer to the proportion of template generation during enrollment?