Benchmark area:
Secure Template Fingerprint Verification
|
This benchmark area contains fingerprint verification benchmarks for algorithms relying on protected templates to enhance privacy. Algorithms submitted to these benchmarks are required to enroll a given fingerprint into a protected template (that is, a template from which the fingerprint features cannot be extracted) and to compare it against a given fingerprint image.
|
Within the context of this benchmark area, “protected template” is a fingerprint representation which cannot be reverse-engineered to extract the fingerprint features. Note that encrypting the template (with symmetric or asymmetric keys) is not sufficient to qualify an algorithm for this benchmark area: only non-invertible feature transformations are allowed. The participant is required to reference a technical paper proving the security of the technique. Note that the algorithm will be tested for accuracy and efficiency only; no security analysis will be performed, hence
FVC-onGoing organizers do not certify in any way the template protection claims.
Benchmarks
Currently, this benchmark area contains the following benchmarks:
- STFV-TEST: A simple dataset useful to test algorithm compliancy
with the testing protocol
(results obtained on this benchmark are only visible in the participant
private area and cannot be published).
- STFV-STD-1.0:
Contains fingerprint images acquired in operational conditions using high-quality optical scanners. Results should reflect the expected accuracy in large-scale fingerprint-based applications.
- STFV-HARD-1.0:
Contains a relevant number of difficult cases (noisy images, distorted impressions, etc.) that makes fingerprint verification more challenging. Results do not necessarily reflect the expected accuracy in real applications but allow to better discriminate the performance of various fingerprint recognition algorithms.
The table below reports the main characteristics of each benchmark:
Benchmark |
Scanner Type |
Resolution |
Minimum Image Size |
Maximum Image Size |
Genuine Attempts |
Impostor Attempts |
STFV-TEST |
Optical |
500 dpi |
440x500 |
440x500 |
280 |
45 |
STFV-STD-1.0 |
Optical |
500 dpi |
440x500 |
440x500 |
27720 |
87990 |
STFV-HARD-1.0 |
Optical |
500 dpi |
260x374 |
448x500 |
19320 |
20850 |
Some sample images with the same format used in the benchmarks of this area are
available in the download page.
The following sections report the testing protocol and the performance
indicators common to all benchmarks in this area.
Protocol
Each participant is required to submit, for each algorithm, two executables
in the form of Win32 console applications.
- Both executables will take the input from command-line arguments and will
append the output to a text file.
- The first executable (enroll.exe) enrolls a fingerprint image and produces a
template file; the command-line syntax is:
enroll.exe <imagefile> <templatefile>
<outputfile>
where:
imagefile |
the input image pathname |
templatefile |
the output template pathname |
outputfile |
the output text-file, where a log string (of the form
imagefile templatefile result) must be appended;
result is "OK" if the enrollment can be
performed or "FAIL" if the input image cannot be processed by the algorithm |
- The second executable (match.exe)
matches a fingerprint template against a fingerprint image and produces
a similarity score; the command-line syntax is:
match.exe <templatefile> <imagefile>
<outputfile>
where:
templatefile |
the input template pathname |
imagefile |
the input image pathname |
outputfile |
the output text-file, where a log string (of the form
templatefile1 templatefile2 result similarity)
must be appended; result
is "OK" if the matching can be performed or "FAIL" if the matching cannot be
executed by the algorithm; similarity is a
floating point value ranging from 0 to 1 which indicates the similarity between
the two templates: 0 means no similarity, 1 maximum similarity |
- Both executables have to
operate only on the explicitly-given inputs, without exploiting any
learning technique or template consolidation/update based on previous
enrolls/matches.
- C, C# and Matlab language skeletons for enroll.exe
and match.exe are available in the
download page to reduce the participants
implementation efforts.
Constraints
During test execution the following constraints will be enforced:
Benchmark |
Maximum time for each enroll |
Maximum time for each match |
Maximum template size |
Memory allocation limit for enroll and match processes |
STFV-TEST |
15 seconds |
5 seconds |
No limit |
No limit |
STFV-STD-1.0 |
15 seconds |
5 seconds |
No limit |
No limit |
STFV-HARD-1.0 |
15 seconds |
5 seconds |
No limit |
No limit |
Each enrollment or matching attempt that violates one of the above constraints
results in a failure to enroll or failure to match, respectively.
The following time breaks are enforced between two consecutive submissions to the
same benchmark by the same participant.
Benchmark |
Minimum break |
STFV-TEST |
12 hour(s)
|
STFV-STD-1.0 |
30 day(s)
|
STFV-HARD-1.0 |
30 day(s)
|
Performance Evaluation
For each algorithm, genuine (matching two fingerprints of
the same finger) and impostor (matching two fingerprints originating from different
fingers) attempts are performed to compute False Non
Match Rate FNMR (also referred as False Rejection Rate - FRR) and
False Match Rate FMR (also referred as False
Acceptance Rate - FAR).
Although it is possible to reject images in enrollment, this is
strongly discouraged. In fact, rejection in
enrollment is fused with other error rates for measuring the final accuracy; in particular,
each rejection in enrollment will produce a "ghost" template which will
not match (matching score 0) with all the
remaining templates.
For each algorithm the following performance indicators are reported:
- REJENROLL (Number of rejected
fingerprints during enrollment)
- REJNGRA (Number of rejected
fingerprints during genuine matches)
- REJNIRA (Number of rejected
fingerprints during impostor matches)
- EER (equal-error-rate)
- FMR100 (the lowest FNMR for FMR≤1%)
- FMR1000 (the lowest FNMR for FMR≤0.1%)
- FMR10000 (the lowest FNMR for FMR≤0.01%)
- ZeroFMR (the lowest FNMR for FMR=0%)
- ZeroFNMR (the lowest FMR for FNMR=0%)
- Average enrollment time
- Average matching time
- Average and maximum template size
- Maximum amount of memory allocated
- Impostor and
Genuine score distributions
- FMR(t)/FNMR(t) curves, where t is the
acceptance threshold
- DET(t) curve
|