Login

Benchmark area: Palmprint Verification

This benchmark area contains palmprint verification benchmarks. Palmprint verification consists in comparing two palmprints to determine whether they are impressions of the same palm or not (one-to-one comparisons). Algorithms submitted to these benchmarks are required to enroll palmprints into proprietary or standard templates and to compare such templates to produce a similarity score.

Benchmarks

Currently, this benchmark area contains the following benchmarks:

  • PV-TEST-FULL: A simple dataset useful to test algorithm compliancy with the testing protocol (results obtained on this benchmark are only visible in the participant private area and cannot be published). See more details here.
  • PV-TEST-PARTIAL: A simple dataset useful to test algorithm compliancy with the testing protocol (results obtained on this benchmark are only visible in the participant private area and cannot be published). See more details here.
  • PV-FULL-1.0: Contains full images of palms acquired in operational conditions using high-quality optical scanners. Test comparing full images to full images of palm.
  • PV-PARTIAL-1.0: Contains partial and full images of palms acquired in operational conditions using high-quality optical scanners. Test comparing partial impressions to full impressions of palm. This test is to simulate the scenario of matching latent palmprints to full palmprints in law enforcement applications. Note that partial palmprints have been collected using optical scanner, they are not real latent palmprints.

The table below reports the main characteristics of each benchmark:

Benchmark Scanner Type Resolution Minimum Image Size Maximum Image Size Genuine Attempts Impostor Attempts
PV-TEST-FULL Optical 500 dpi 2552x2500 2552x2500 280 45
PV-TEST-PARTIAL Optical 500 dpi 416x1632 1704x1744 40 360
PV-FULL-1.0 Optical 500 dpi 2552x2500 2552x2500 2800 4950
PV-PARTIAL-1.0 Optical 500 dpi 688x768 2552x2496 400 10000

The palmprints of the PV-TEST-FULL and PV-TEST-PARTIAL benchmarks are available as sample images in the download page.

The following sections report the testing protocol and the performance indicators common to all benchmarks in this area.

Protocol

Each participant is required to submit, for each algorithm, two executables in the form of Win32 console applications.

  • Both executables will take the input from command-line arguments and will append the output to a text file.
  1. The first executable (enroll.exe) enrolls a palmprint image and produces a template file; the command-line syntax is:
    enroll.exe <type> <imagefile> <templatefile> <outputfile>
    where:
    type palmprint type. "F" means full palmprint and "P" means partial palmprint
    imagefile the input image pathname
    templatefile the output template pathname
    outputfile the output text-file, where a log string (of the form imagefile templatefile result) must be appended; result is "OK" if the enrollment can be performed or "FAIL" if the input image cannot be processed by the algorithm

  2. The second executable (match.exe) matches two palmprint templates and produces a similarity score; the command-line syntax is:
    match.exe <templatefile1> <templatefile2> <outputfile>
    where:
    templatefile1 the first input template pathname. For PV-PARTIAL benchmark, templatefile1 is the partial palmprint
    templatefile2 the second input template pathname. For PV-PARTIAL benchmark, templatefile2 is the full palmprint
    outputfile the output text-file, where a log string (of the form templatefile1 templatefile2 result similarity) must be appended; result is "OK" if the matching can be performed or "FAIL" if the matching cannot be executed by the algorithm; similarity is a floating point value ranging from 0 to 1 which indicates the similarity between the two templates: 0 means no similarity, 1 maximum similarity
  • Both executables have to operate only on the explicitly-given inputs, without exploiting any learning technique or template consolidation/update based on previous enrolls/matches.
  • C, C# and Matlab language skeletons for enroll.exe and match.exe are available in the download page to reduce the participants implementation efforts.

Constraints

During test execution the following constraints will be enforced:

Benchmark Maximum time for each enroll Maximum time for each match Maximum template size Memory allocation limit for enroll and match processes
PV-TEST-FULL 90 seconds 10 seconds No limit No limit
PV-TEST-PARTIAL 90 seconds 5 seconds No limit No limit
PV-FULL-1.0 90 seconds 10 seconds No limit No limit
PV-PARTIAL-1.0 90 seconds 5 seconds No limit No limit

Each enrollment or matching attempt that violates one of the above constraints results in a failure to enroll or failure to match, respectively.

The following time breaks are enforced between two consecutive submissions to the same benchmark by the same participant.

Benchmark Minimum break
PV-TEST-FULL 12 hour(s)
PV-TEST-PARTIAL 12 hour(s)
PV-FULL-1.0 30 day(s)
PV-PARTIAL-1.0 30 day(s)

Performance Evaluation

For each algorithm, genuine (matching two palmprints of the same palm) and impostor (matching two palmprints originating from different palms) attempts are performed to compute False Non Match Rate FNMR (also referred as False Rejection Rate - FRR) and False Match Rate FMR (also referred as False Acceptance Rate - FAR).

Although it is possible to reject images in enrollment, this is strongly discouraged. In fact, rejection in enrollment is fused with other error rates for measuring the final accuracy; in particular, each rejection in enrollment will produce a "ghost" template which will not match (matching score 0) with all the remaining templates.

For each algorithm the following performance indicators are reported:

  • REJENROLL (Number of rejected palmprints during enrollment)
  • REJNGRA (Number of rejected palmprints during genuine matches)
  • REJNIRA (Number of rejected palmprints during impostor matches)
  • EER (equal-error-rate)
  • FMR100 (the lowest FNMR for FMR1%)
  • FMR1000 (the lowest FNMR for FMR0.1%)
  • FMR10000 (the lowest FNMR for FMR0.01%)
  • ZeroFMR (the lowest FNMR for FMR=0%)
  • ZeroFNMR (the lowest FMR for FNMR=0%)
  • Average enrollment time
  • Average matching time
  • Average and maximum template size
  • Maximum amount of memory allocated
  • Impostor and Genuine score distributions
  • FMR(t)/FNMR(t) curves, where t is the acceptance threshold
  • DET(t) curve
For information or suggestions: fvcongoing@csr.unibo.it Copyright © 2017 Biometric System Laboratory