Login

Benchmark area: Fingerprint Matching (ISO)

This benchmark area contains fingerprint matching benchmarks using a standard minutiae-based template format [ISO/IEC 19794-2 (2005)]. Algorithms submitted to these benchmarks are required to compare ISO fingerprint templates to determine whether they are impressions of the same finger or not (one-to-one comparisons). No fingerprint enrollment (feature extraction) is required, only the minutiae matching algorithms are evaluated by these benchmarks.

Benchmarks

Currently, this benchmark area contains the following benchmarks:

  • FMISO-TEST: A simple dataset useful to test algorithm compliancy with the testing protocol (results obtained on this benchmark are only visible in the participant private area and cannot be published).
  • FMISO-STD-1.0: Contains ISO templates created from fingerprint images acquired in operational conditions using high-quality optical scanners. Results should reflect the expected accuracy in large-scale fingerprint-based applications.
  • FMISO-HARD-1.0: Contains a relevant number of difficult cases (noisy images, distorted impressions, etc.) that makes fingerprint verification more challenging. Results do not necessarily reflect the expected accuracy in real applications but allow to better discriminate the performance of various fingerprint recognition algorithms.

The table below reports the main characteristics of each benchmark:

Benchmark Scanner Type Resolution Minimum Template Size Maximum Template Size Genuine Attempts Impostor Attempts
FMISO-TEST Optical 500 dpi 440x500 440x500 280 45
FMISO-STD-1.0 Optical 500 dpi 440x500 440x500 27720 87990
FMISO-HARD-1.0 Optical 500 dpi 260x374 448x500 19320 20850

The following sections report the testing protocol and the performance indicators common to all benchmarks in this area.

Protocol

Each participant is required to submit, for each algorithm, an executable in the form of Win32 console application.

  • The executable (match.exe) will take the input from command-line arguments and will append the output to a text file. It matches two ISO templates and produces a similarity score; the command-line syntax is:
    match.exe <ISOtemplatefile1> <ISOtemplatefile2> <outputfile>
    where:
    ISOtemplatefile1 the first input  ISO template pathname
    ISOtemplatefile2 the second input ISO template pathname
    outputfile the output text-file, where a log string (of the form ISOtemplatefile1 ISOtemplatefile2 result similarity) must be appended; result is "OK" if the matching can be performed or "FAIL" if the matching cannot be executed by the algorithm; similarity is a floating point value ranging from 0 to 1 which indicates the similarity between the two templates: 0 means no similarity, 1 maximum similarity

  • The executable has to operate only on the explicitly-given inputs, without exploiting any learning technique or template consolidation/update based on previous matches.
  • C, C# and Matlab language skeletons for match.exe are available in the download page to reduce the participants implementation efforts.

Constraints

During test execution the following constraints will be enforced:

Benchmark Maximum time for each match Memory allocation limit for match process
FMISO-TEST 3 seconds No limit
FMISO-STD-1.0 3 seconds No limit
FMISO-HARD-1.0 3 seconds No limit

Each matching attempt that violates one of the above constraints results in a failure to match.

The following time breaks are enforced between two consecutive submissions to the same benchmark by the same participant.

Benchmark Minimum break
FMISO-TEST 12 hour(s)
FMISO-STD-1.0 30 day(s)
FMISO-HARD-1.0 30 day(s)

Performance Evaluation

For each algorithm, genuine (matching two ISO templates of the same finger) and impostor (matching two ISO templates originating from different fingers) attempts are performed to compute False Non Match Rate FNMR (also referred as False Rejection Rate - FRR) and False Match Rate FMR (also referred as False Acceptance Rate - FAR).

For each algorithm the following performance indicators are reported:

  • REJNGRA (Number of rejected fingerprints during genuine matches)
  • REJNIRA (Number of rejected fingerprints during impostor matches)
  • EER (equal-error-rate)
  • FMR100 (the lowest FNMR for FMR1%)
  • FMR1000 (the lowest FNMR for FMR0.1%)
  • FMR10000 (the lowest FNMR for FMR0.01%)
  • ZeroFMR (the lowest FNMR for FMR=0%)
  • ZeroFNMR (the lowest FMR for FNMR=0%)
  • Average matching time
  • Maximum amount of memory allocated
  • Impostor and Genuine score distributions
  • FMR(t)/FNMR(t) curves, where t is the acceptance threshold
  • DET(t) curve
For information or suggestions: fvcongoing@csr.unibo.it Copyright © 2017 Biometric System Laboratory