Face-off

Facial-recognition technology has improved significantly during the past few years, making it an effective tool for verifying access to buildings and computers. But it's less useful for identifying unknown individuals in a crowded stadium or airport.

Facial recognition is hard work. Different lighting conditions, varying expressions, aging, weight changes, facial hair, sunglasses and, of course, intentional disguises can present daunting challenges for any facial-recognition system.

In fact, studies show that alert humans do a better job of matching faces than computers do. The key word here, of course, is "alert." Facial-recognition technologies don't get tired and they don't come to work with hangovers. The drawback is that they make more mistakes than people do.

In practice, facial-recognition technologies are used for two fundamentally different chores: verification and identification. Typical verification tasks determine that people are who they claim to be before allowing entrance to a facility or access to data. In such cases, facial-recognition technology compares a current image to images in a database. Match rates are good with this method because people voluntarily allow officials to capture their facial images under controlled circumstances, yielding higher-quality images than pictures taken under more challenging circumstances.

Typical identification tasks attempt to match unknown individuals from sources such as surveillance videos with images from passports or criminal records, for example. Identification matches are more challenging because images obtained for this purpose are generally not created with the subjects' cooperation under controlled conditions. In practice, officials often use facial-recognition technologies to narrow the list of potential matches that a human analyst should examine.

Current facial-recognition technologies use one or more of four basic methods: appearance-based, rule-based, feature-based and texture-based.

Appearance-based methods measure the similarities of two or more images rather than attempting to extract facial features from the images. Rule-based methods analyze facial components, such as the eyes, nose and mouth, to measure their relationship between images. Feature-based methods analyze the characteristics of facial features, including edge qualities, shape and skin color. Texture-based analysis, a newer technique, examines the different texture patterns of faces.

For all of these methods, facial-recognition tools generate a template using algorithms to define and store data. When a product captures an image for verification or identification, it processes the data and compares it with the template's information.

Performance comparisons among facial-recognition solutions are difficult because they must account for several factors that differentiate the products, such as the methods used and the type and quality of the algorithm for analyzing data. One solution might be better for verification in certain conditions, but another solution might be superior for identification chores that require sifting through large numbers of low-quality images for matches.

Additionally, some facial-recognition solutions can be combined with other biometrics, such as fingerprint readers. Although biometric combinations are generally more appropriate for verification than for identification purposes, some vendors are working on solutions that combine facial-recognition with gait analysis. These studies provide a detailed look at how a person walks and could be used for surveillance.

We conducted our last major review of facial-recognition technology three years ago when we examined its underlying principles. For our most recent look at facial-recognition solutions, we planned to review technologies from Cognitec Systems, Eyematic Interfaces and Identix, the three top performers in the Defense Advanced Research Projects Agency's most recent facial-recognition shootout. Officials at Cognitec and Eyematic Interfaces, now known as Neven Vision, agreed to participate. After Identix officials declined to participate, we decided to include Acsys Biometrics' FRS Discovery, a solution that incorporates a neural network algorithm that claims to learn as it goes along.

For our tests, we enrolled images with all three systems using a live video feed from a standard USB camera. We also enrolled 16 static facial images.

We attempted to authenticate under standard ambient lighting conditions and relatively low light conditions. We turned subjects' heads up, down, left and right to test the maximum angles at which we could authenticate. We also varied the distance from the camera from about 1 foot away to about 5 feet away. We changed facial expressions and attempted to authenticate with a false mustache and a pair of clear, frameless glasses that were not worn during enrollment.

Federal officials who are considering implementing a facial-recognition solution should be aware that the performance of such products is situational. The solution that works best under one set of conditions might not work as well under other conditions. You should explore several solutions before settling on one.

For our review, we examined customized examples of the technologies, not out-of-the-box solutions. If you want to implement one of these technologies, you should contact the vendor or a systems integrator who can tailor an application to your needs. The expectation is that you will potentially achieve significantly better results than we saw in our tests.

Finally, we cannot provide meaningful estimates of pricing for the technologies we tested because pricing is set by systems integrators and varies greatly according to the type and magnitude of an organization's implementation.

Acsys FRS Discovery

In our limited set of tests, FRS Discovery fell a tad behind the others in recognition accuracy. Bear in mind, however, that the program uses a neural net that analyzes image data and promises to improve results over time.

FRS Discovery is a functional product, but it's designed primarily for potential customers to evaluate the technology and get a feel for facial recognition. Acsys offers a product called VeraShield for PC log-on control and another called VeraPort for physical access.

Acsys' Holographic/Quantum Neural Technology learns changes in facial features, which allows the software to accommodate the aging process and other cosmetic differences.

FRS Discovery uses a client/server architecture. Both the client and server applications perform facial tracking, enrollment, verification and identification. The tracking function lets the solution locate and follow a face as it moves within the frame.

In addition, FRS Discovery's client applications are fully functional when disconnected from the server components.

The system is geared primarily toward surveillance applications that use identification, but it can also be integrated with physical access-control systems for verification uses.

FRS Discovery can authenticate someone's identity and then perform an action, such as opening a door, after the user presents a proximity card. The solution can also send messages or alerts to a remote computer, such as a wireless handheld device.

FRS Discovery can store as many as 32,000 facial images. When Acsys technology is integrated with custom applications, administrators can store 45 million faces on 1 terabyte of space in Microsoft SQL Server databases.

Discovery's template size is 12K noncompressed and 5K compressed. Those sizes are large compared with Cognitec's 456-byte template and Neven Visions' range of 350 bytes to 1.6K.

Administrators can store more templates if they are smaller. However, smaller templates don't have as much detail so recognition performance might suffer. Neven Vision offers three template sizes so administrators can decide what works best for them.

Enrolling from a live video image was more challenging than we liked. During live enrollment, FRS Discovery takes 100 pictures of the face in about 30 seconds. In comparison, Cognitec's FaceVACS-Entry took eight pictures in about 1.5 seconds, and Neven Vision took 10 pictures in about 1.5 seconds.

When being photographed for enrollment, subjects should constantly move their heads left, right, up and down while also moving forward and backward within a 4- to 5-foot range. That activity ensures that the database will contain images of the face at many different angles and sizes, thereby aiding recognition.

The concept makes sense, but the process is cumbersome. Because the system will not enroll duplicate pictures, the process takes longer if the person does not move enough. The process essentially freezes if the subject stays still too long. Neither the manual nor the voice prompts warn users about this potential problem. And no prompt told us to continue moving during the procedure.

After enrollment, an administrator can review the database images and delete those of poor quality. Additional enrollment images can be added for any user at any time.

Despite the thoroughness of the enrollment process, we found FRS Discovery to be a bit finicky, even when we lowered the threshold required for matches.

The biggest impact on successful authentication was the angle at which people presented their faces. FRS Discovery is supposed to authenticate faces at up to a 90-degree horizontal angle from center, with the default value at 45 degrees. But we couldn't authenticate beyond about 60 degrees, despite lowering the threshold values for tracking and matching. And the lighting had to be optimal at the higher-degree angles.

Donning a pair of clear, frameless glasses did not affect authentication as long as there was no glare on either lens. Performance also was not significantly affected when we lowered the lighting level. Most facial expressions did not affect performance, but when we yawned, the system did not recognize us.

When we tried to authenticate with a false mustache, we noticed what was likely the neural technology's learning capability in action. There was a slight delay the first time, but subsequent authentication was instantaneous. In all cases, the system verified and identified us accurately.

We also enrolled using static JPEG images. The process was straightforward, and a batch enrollment feature allows you to enroll multiple images at once. The manual does not list an optimal number of images but suggested that subjects enroll multiple images.

We enrolled with 16 images per subject. The system failed to verify or identify us after multiple attempts, even after lowering thresholds. When we asked Acsys officials about this flaw, they told us that static image matching is not one of the product's strengths. The company sells a utility that specializes in this function for customers who need it.

When you submit an image for matching, the system displays a list of up to the five closest matches in the database. This is helpful because no facial-recognition product is perfect. Obviously, if more than one match is returned, it's up to human analysts to decide which is the closest match.

FRS Discovery can search and match as many as 5,000 images per second when operating from a live video feed, although Acsys' software development kit (SDK) technology can search as many as 80,000 templates per second. That's significantly slower than Cognitec's 2.7 million templates per second and Neven Vision's 300,000 templates per second.

All three systems in our review performed facial tracking and could locate and identify multiple faces simultaneously. This is the essence of surveillance: When an unknown person comes within the camera's range, the system searches a database of images to look for a match.

When we presented FRS Discovery with two faces, it accurately identified both simultaneously. It can only track and identify four faces at once, but Acsys' SDK can track and identify as many as 16. Cognitec's technology can track between five and 10 faces at once. Neven Vision's product does not have an official limit, but the system must see the entire face and the image must be large enough to fit at least 25 pixels between the eyes, so the number is finite.

Overall, FRS Discovery was not as solid a performer as the other two products. If conditions were less than optimal, the system either did not authenticate or took longer to do so. The somewhat cumbersome enrollment process did not pay off as it should have. The product did, however, authenticate at larger facial angles than the other two systems. But those angles were smaller in our tests than the system's advertised capabilities.

FaceVACS-Entry

Cognitec was one of the top performers in DARPA's last round of facial-recognition shootouts, and the technology is still at the head of the pack. In our tests, FaceVACS-Entry performed slightly better than FRS Discovery in recognition accuracy. Cognitec uses a 2-D method of facial recognition that combines feature recognition with texture and light intensity analysis.

The FaceVACS-Entry product is designed for access-control applications and primarily used for verification. But it also supports identification because some applications require it. FaceVACS-Alert, another Cognitec product, is designed more specifically for identification chores.

FaceVACS-Entry runs on a client/server architecture composed of one or more administration stations, one or more verification stations and an optional repository station.

Administration stations are used for enrollment and other administrative tasks. Users authenticate data at the product's verification stations. And repository stations synchronize data across the administration and verification stations in addition to moving data from local stations to the central repository.

Enrolling from a live video feed was easy. The system enrolls eight images in about 1 second. Subjects can enroll multiple times to add more images to the database.

Cognitec's template size is small at 456 bytes, and that size does not vary.

Verification was quick and accurate. In fact, Cognitec officials claim the solution can search 2.7 million records per second using a 2 GHz Intel Pentium 4 with 2G of system memory. And hardware is the only limitation on the number of records that can be managed.

The system performed well in low light conditions and when subjects wore glasses. The system also verified a yawning face, something Neven Vision could also do but FRS Discovery could not. We did not notice a change in speed or accuracy when subjects wore a false mustache.

As with FRS Discovery, the biggest authentication variable was the angle for presenting faces. Cognitec advertises authentication at no more than 15 degrees from center, both horizontally and vertically.

Our tests supported this estimate, although when we enrolled images with faces turned at larger angles, up to perhaps 30 degrees, the system verified users when they presented their faces at the same angles. The important factor is that both eyes must be clearly visible.

All three systems featured adjustable thresholds that could make matching requirements less stringent if necessary. But FaceVACS-Entry stood out with its personal threshold feature.

The other systems' thresholds can only be applied on the global level, but FaceVACS-Entry allows you to set personal, user- specific thresholds. That way if one person struggles with verification, administrators don't need to lower the thresholds for everyone. This feature is only available for verification and not for identification.

Enrolling static images was a simple process, but it took some time. Like FRS Discovery, FaceVACS-Entry has a batch enrollment feature.

The biggest static enrollment difference between the two systems was image-processing time. FRS Discovery enrolled the images almost instantly, while FaceVACS-Entry took several minutes to process and approve them. In contrast with FRS Discovery, the Cognitec system verified and identified us when searching only the static enrolled images, although we did have to lower the thresholds slightly.

But even when thresholds were set to default levels, the system produced a correct match — that is, it presented an image of the correct person even though the system's message stated the person could not be authenticated.

In a real-world situation, although the system couldn't match the images closely enough to authenticate the user, a person reviewing the results still could have identified him or her by looking at the two images.

FaceVACS-Entry's identification speed was faster than FRS Discovery's, although it's hard to say whether that had more to do with software performance or the number of images in our test databases — FaceVACS-Entry takes only eight pictures at enrollment while FRS Discovery takes 100.

These results extend to a larger scale because Cognitec's technology can search as many as 1.5 million images per second while FRS Discovery can only search 5,000 images per second. Acsys' SDK, however, includes the capability to search up to 50,000 images per second.

Like the other systems, FaceVACS-Entry can track and identify multiple faces at once. Company officials told us the maximum number of faces that the system can identify simultaneously is about five to 10, depending on the computer's power and the response-time requirement. That's more than FRS Discovery's limit of four faces but less than Neven Vision's claim of tracking an unlimited number. The result, however, is limited to the number of faces that can physically fit within the camera's field of vision while meeting minimum size requirements.

FaceVACS-Entry was a strong performer overall. It authenticated accurately when lighting and facial expressions changed. Standout features include the personal threshold setting and the extremely fast matching speed.

Neven Vision

Neven Vision's performance in our tests was on par with Cognitec's product. It recognized us quickly and accurately, and various challenges, such as glasses and facial hair, did not affect it.

The company did not have an off-the-shelf product to send us, so we tested a demonstration application. The demo did not have verification capability, so we only tested identification. Neven Vision stands out as the only facial-recognition engine on the market that can run directly on handheld devices such as personal digital assistants, wireless phones and mobile terminals.

The company will use this capability in a new product called Mobile Identifier due out in the coming months. Geared toward law enforcement, Mobile Identifier features an on-board database and image-recognition engine. Results are immediate because officers don't need to wait for data to be transmitted via wireless connections.

Neven Vision's engine is based on neural network mapping algorithms. A face-finder component determines the position and size of the face, and a second process determines the position of local features. The template is then computed from those local features.

The number of templates the system can store depends on the hardware and amount of memory available. As an example, a typical PC with 1G of memory can hold more than 1 million templates.

Neven Vision's storage capability resembles FaceVACS-Entry's because the companies' template sizes are similar. Neven Vision lets programmers choose one of three template sizes that vary between 350 bytes and 1.6K. Larger templates trade search speed for accuracy, so programmers can choose a template size that best meets their needs.

Neven Vision's demonstration application took 10 images in about 1.5 seconds during the live enrollment process.

Neven Vision officials advertise facial recognition at angles of up to 20 degrees from frontal and up to 15 degrees rotation from upright, and our tests upheld those claims.

Not surprisingly, recognition at an angle took a little longer if the only enrolled images were frontal. When we had subjects turn their heads slightly during enrollment, recognition speed and accuracy at angles improved.

Neven Vision performed as well as FaceVACS-Entry did when we tried to authenticate with a yawning face, glasses, dim lighting and a false mustache. None of those challenges had a noticeable effect on recognition speed or accuracy.

Like the other two products, Neven Vision's demo application allowed us to enroll with static images. At default threshold settings, it accurately matched a live image with one of the enrolled static images. However, confidence levels, which measure an image's match to an enrolled image, were 10 percent to 20 percent lower than when matching with live enrolled images.

Neven Vision, like the other products, can track and identify multiple faces at once. There is no official maximum, but to recognize a face, the system must see the entire face and the image must be large enough to fit at least 25 pixels between a person's eyes.

Therefore, the limit is the number of faces that meet those requirements that can fit within the camera's field of vision.

Facial-recognition technology's template search speed is hardware-dependent. For example, a typical PC with a 3 GHz Pentium 4 processor can search about 300,000 templates per second.

Neven Vision's off-the-shelf products feature global threshold settings, but they don't include personal threshold settings like FaceVACS-Entry. However, the company's application program interface allows programmers to define multiple thresholds so it would be possible to create thresholds for individuals.

In our tests, Neven Vision was a strong performer and comparable to FaceVACS-Entry.

The bottom line

Facial-recognition technology can't yet be relied on to pick a terrorist out of a crowd. For it to work effectively, subjects must cooperate and allow a photo to be taken under controlled conditions, as in the case of a criminal suspect who is photographed at booking.

If our tests are any indication, surveillance technology is a long way from identifying someone who walks across a room or is spotted in a crowd. That's because the systems need to obtain a full view of the face with both eyes visible and at an angle that doesn't stray far from center.

But for cooperative applications, such as computer access or physical access, facial recognition can be an effective tool. Although the technology is susceptible to changes in a person's appearance, the applications allow for re-enrollment and threshold adjustments if a user's appearance changes significantly.