ISO/IEC 19794-5 Compliance

Traditional identity documents are now being replaced by electronic documents with biometric features onboard; this enable machine-assisted check of person identity, both when the document is issued and successively for identity verification. The International Civil Aviation Organization (ICAO) established a specific working group to determine the most suitable way of "uniquely encoding a particular physical characteristic of a person into a biometric-identifier that can be machine-verified to confirm the presenter’s identity". Face is chosen as the primary globally interoperable biometric characteristic for machine assisted identity confirmation in Machine Readable Travel Documents (MRTD). The use of face as a biometric identifier has some drawbacks: in particular the recognition accuracy is lower with respect to other characteristics mainly due to the large intra-class variability of face images. At today, a high accuracy level can only be achieved in supervised/cooperative application scenarios and in the presence of good quality images. For this reason machine-assisted face recognition from identity documents requires quite restrictive quality standards to be fulfilled. To this aim, the ISO/IEC 19794-5:2011 standard, starting from the guidelines initially proposed by ICAO, specifies rules for encoding, recording and transmitting the facial image information and defines scene constraints, photographic properties and digital image attributes of facial images. Overall the ISO/IEC 19794-5:2011 standard provides quite generic guidelines and several examples of acceptable/unacceptable face images; a clear and unambiguous description of all the requirements is still missing. Several vendors of biometric technologies started to produce and distribute SDKs able to automatically verify the compliance of face images to the ISO/ICAO standard, in the attempt of semi-automating the document issuing process. Unfortunately, due to the lack of clear specifications and of an appropriate benchmark, comparing the performance of such systems is not easy.

The BioLab-ICAO Benchmark

The BioLab-ICAO benchmark, developed in our laboratory and made available to the scientific community through the FVC-onGoing website, consists of:

• a set of requirements, directly derived from the ISO/ICAO standard;
• a large database of face images and related ground truth data;
• a testing protocol for objective performance evaluation and comparison.

Requirements

Starting from the indications provided into the ISO standard , a set of 30 tests has been defined to evaluate the accuracy of automatic systems for compliance verification:

• Feature extraction accuracy: these tests evaluate the accuracy of eyes and face detection;
• Geometric properties of the image: the tests of this category are aimed to verify if the image fulfills the ISO standard constraints related to the face size and its position within the image. The geometric requirements are specified as a function of some measures represented in the following figure;
• Photographic and pose-specific tests: the face must be clearly visible and recognizable; this requirement implies several constraints that generate most of the uncertainties in the interpretation of the ISO standard. A precise formalization is here proposed to limit as much as possible the ambiguity.

Database

An ad hoc database has been created using, where possible, public databases and acquiring additional images to cover the missing defects. The database contains 5,588 images, gathered from different sources:

• 1,741 images from the AR database, size 768×576;
• 1,935 images from the FRGC database, size 1,704×2,272 or 1,200×1,600;
• 291 images from the PUT database, size 2,048×1,536;
• 804 images artificially generated by applying ink-marked/creased, pixelation and washed-out effects to compliant images from the AR database;
• 817 newly acquired images of size 1,600×1,200.

• coordinates of the main facial features (i.e., eye center and corners, nose center, left and right nostrils, mouth corners and upper and lower lip center);
• compliance to the photographic and pose requirements expressed on the basis of a three-state logic: compliant, non-compliant and dummy. The dummy value is used for the cases of uncertainty (e.g., when the person wears dark tinted lenses it is not possible to understand if the eyes are open even for human experts).

Testing Protocol

The algorithms for compliance verification are usually characterized by a set of parameters that need to be tuned according to the kind of images the algorithm has to process. For this reason a training set of 720 images (of which 50 are fully compliant) has been extracted from the BioLab-ICAO database to be used for parameters setup and training. The rest of images are used for testing. Some information about the composition of both training and test set are given in the following table where for each characteristic the number of compliant (C), non-compliant (NC) and dummy (D) images are reported. In order to perform balanced tests, the testing set has been further divided into 24 subsets, each related to a specific characteristic.

An Example

This is only an example of how the images of the BioLab-ICAO benchmark look like. Images have been reduced in size and compressed.

Evaluation

Due to privacy restrictions the database cannot be downloaded. A new area (FICV) has been added to FVC-onGoing to evaluate SDKs for compliance verification on the benchmark described above.

Bibliography

	M. Ferrara, A. Franco, D. Maio and D. Maltoni, "FICV: a new FVC-onGoing benchmark area on Face Compliance Verification to ISO/IEC 19794-5:2011 standard", EAB Newsletter, no.1, pp.15-17, April 2014.
	M. Ferrara, A. Franco, D. Maio and D. Maltoni, "Face Image Conformance to ISO/ICAO standards in Machine Readable Travel Documents", IEEE Transactions on Information Forensics and Security, vol.7, no.4, pp.1204-1213, August 2012. Abstract
	M. Ferrara, A. Franco and D. Maio, "A multi-classifier approach to face image segmentation for travel documents", Expert Systems With Applications, vol.39, no.9, pp.8452-8466, July 2012. Abstract
	M. Ferrara, A. Franco, D. Maltoni and Y. Sun, "On the Impact of Alterations on Face Photo Recognition Accuracy", in proceedings International Conference on Image Analysis and Processing (ICIAP2013), Naples (Italy), vol.8156, pp.743-751, September 2013. Abstract
	D. Maltoni, A. Franco, M. Ferrara, D. Maio and A. Nardelli, "Biolab-ICAO: A New Benchmark to Evaluate Applications Assessing Face Image Compliance to ISO/IEC 19794-5 Standard", in proceedings 16th International Conference on Image Processing (2009), Cairo, pp.41-44, November 2009. Abstract
	M. Ferrara, A. Franco and D. Maltoni, "Evaluating systems assessing face-image compliance with ICAO/ISO standards", in proceedings European Workshop on Biometrics and Identity Management (BIOID 2008), Roskilde, Denmark, pp.191-199, May 2008. Abstract
	G. Borghi, A. Franco, N. Di Domenico and D. Maltoni, "TONO: a Synthetic Dataset for Face Image Compliance to ISO/ICAO Standard", to appear on proceedings Workshop on Synthetic Data for Computer Vision, September 2024. Abstract
	A. Alessandroni, M. Ferrara, A. Franco, D. Maio, D. Maltoni and L Palumbo, "Evaluating systems assessing face-image compliance with ICAO/ISO standards", Technical Report, DEIS - University of Bologna, March 2008. Abstract
	A. Alessandroni, M. Ferrara, A. Franco, D. Maio, D. Maltoni and L Palumbo, "Test compliance ICAO su foto del volto in formato Tokenizzabile", Technical Report, DEIS - University of Bologna, March 2008.