Benchmark area: Fingerprint Orientation Extraction

The estimation of local fingerprint orientations is a fundamental step in fingerprint analysis and recognition (e.g., it is a prerequisite for image enhancement). This benchmark area contains benchmarks for local orientation extraction algorithms. Algorithms submitted to these benchmarks are required to extract local orientations from fingerprint images and to save them into a specific format. The extracted orientations are compared to the ground-truth in order to assess the algorithm accuracy.

Benchmarks

Each benchmark in this benchmark area consists of two datasets: a good quality dataset and a bad quality dataset. The challenge is to obtain a good orientation extraction accuracy on the bad quality dataset without losing too much accuracy on the good quality dataset. In fact, to reduce noise on low quality fingerprints, some approaches tend to oversmooth the orientation image and this could compromise accuracy on good quality fingerprints.
Currently, the following benchmarks are available:

• FOE-TEST: A small benchmark useful to test algorithm compliancy with the testing protocol (results obtained on this benchmark are only visible in the participant private area and cannot be published). Fingerprints have been acquired using optical scanners; the ground-truth has been manually marked using an ad-hoc software tool.
• FOE-STD-1.0: Fingerprint orientation extraction benchmark on fingerprints acquired using optical scanners; the ground-truth has been manually marked using an ad-hoc software tool.

The table below reports the main characteristics of each benchmark:

The datasets of the FOE-TEST benchmark are available, together with ground truth data and a viewer software, in the download page.

The following sections report the testing protocol and the performance indicators common to all benchmarks in this area.

Protocol

Algorithms submitted to these benchmarks must be able to extract the orientation image from a fingerprint. A fingerprint orientation image (or directional image), is a matrix whose elements encode the local orientation of the fingerprint ridges. Each element corresponds to a node of a squared-mesh grid with a given step. The value of each element denotes the average orientation of the fingerprint ridges at the corresponding pixel location.

Each participant is required to submit, for each algorithm, one executable named Extractor.exe in the form of Win32 console application.

The executable will take the input from command-line arguments and will save the output to file into a specific format.

• The command-line syntax is:

  Extractor.exe <indexfile> <outputfolder>

  where:
  

  indexfile

  is a test file containing the file path of each fingerprint to be processed. The first line of the file contains the number of fingerprints; each of the following lines has the format: <imagepath> <step> <border> where: <imagepath> is the file path of the fingerprint (bitmap file format); <step> is the step to be used to calculate the orientation image; <border> is the distance from the image border in pixels: this parameter determines the position of the top-left element. The parameter <step> and the number of sampling points are selected in a way that do not allow any sampling point to be closer to any of the borders than <border> pixels. This enables an algorithm to center, at each sampling point, a filter that extends up to <border> pixels in each direction. Note: you can assume that the border is not smaller than 14 pixels.

  outputfolder

  is the path of the folder where the output files have to be saved.

• For each input fingerprint, a further file with the same path and extension .fg (instead of .bmp) is provided. This file defines which elements of the orientation image belong to the fingerprint foreground. The algorithm is required to compute the orientation of those elements only: the remaining elements of the orientation image can be left to zero. The foreground file is a text file where the first line contains the number of rows and columns of the orientation image and the remaining lines denote foreground ('1') and background elements ('0'). Source code to read this file format is provided in the download page.
• The algorithm is required to extract the orientation image from each input fingerprint and save it in the output folder to a file with the same name of the fingerprint but extension .dirmap (instead of .bmp). The output must be saved as a binary file with a specific format: source code to save the file is provided in the download page. In the output file, each orientation element must be stored in one byte in units of 0.703125 (180/256) degrees; for example, a byte orientation value of 128 represents 90 degrees and a byte orientation value of 32 represents 22.5 degrees (note that a byte orientation value of 255 is 179,296875 degrees and not 180 degrees). Each orientation angle shall be measured increasing counter-clockwise starting from the horizontal axsis to the right.
• The algorithm is also required to write the file name (or the full path) of each processed fingerprint to a named pipe, in order to inform the caller about its progress status.
• C and C# language skeletons for Extractor.exe are available in the download page to reduce the participants implementation efforts. These source files perform all the necessary I/O (including loading image and foreground, saving the orientation image and writing to the named pipe).

Constraints

During test execution the following constraints will be enforced:

Each algorithm that violates one of the above constraints results in an execution failure and no performance indicators are provided.

The following time breaks are enforced between two consecutive submissions to the same benchmark by the same participant.

Performance Evaluation

On both datasets, the Root Mean Square Deviation (RMSD) between the ground-truth and the orientation extracted is calculated over all foreground elements. The Average Error over a dataset is calculated as the average RMSD over all the fingerprints in the dataset.

For each algorithm, the following performance indicators are reported:

• AvgErr_BQ (Average Error on the Bad Quality Dataset)
• AvgErr_GQ (Average Error on the Good Quality Dataset)
• Average orientation extraction time
• Maximum amount of memory allocated
• Orientation deviation distribution for both datasets (Histogram of the distribution of individual orientation extraction errors)
• Average error distribution for both datasets (Histogram of the distribution of average errors over the various fingerprints)