Face attributes Detection

Last updated November 15th, 2017

Overview

The Faces Attributes Detection model is useful to determine if an image contains faces and get descriptive attributes on those faces (gender, sex, face landmarks, presence of sun glasses etc).

Face Detection

The values returned (x1,x2,y1,y2) can locate the faces present on the image.

Use-cases

  • Detect people or faces in images or videos
  • Force users to upload pictures of their face
  • Prevent users from uploading group pictures
  • Group or classify your images/videos

Limitations

  • Faces that are very small (such as in large crowds) or are very blurry may go undetected
  • Faces that are not visible to human eyes (mostly hidden for instance or too small) might not be detected

Face Features / Face Landmarks

For each detected face, the positions of the main face features are returned. Face features include:

  • Left and right eyes
  • Left and right mouth corners
  • Nose tip

Use-cases

  • Apply masks, transforms or other augmented-reality elements on faces
  • Adjust face crops based on the relative locations of key face landmarks

Gender Detection

For each detected face, the Face Attribute Model will return a "gender" field that will help you determine if a face is a male or female face, solely based on the characteristics of this face.

Gender properties are determined solely using the face. So other signs such as clothes or context will not influence the result. Males dressing up as females or females dressing up as males should therefore be correctly classified.

The API returns a "female" value and a "male" value. The sum of those are 1, each one being between 0 and 1. The largest of the two corresponds to the expected gender of the face. The closer to 1 it is, the higher the API's confidence.

Use-cases

  • Create profile data based on profile pictures
  • Check that users correctly entered their gender
  • Group or classify your images

Limitations

  • Gender detection is less accurate for babies and young infants
  • Make-up and fake mustaches or fake beards might trick the API, just like they might trick humans
  • Gender detection is less accurate if a very large proportion of the face is veiled

Example results

Female: (female value 0.99, male value 0.01)
Male: (female value 0.05, male value 0.95)

Age group Detection

For each detected face, the Face Attribute Model will return a "minor" field that will help you determine if a given face belongs to someone that is less than 18 years or more than 18 years old.

The 18-year threshold corresponds to the legal age of majority (adulthood) in many countries.

The Age Group information is determined solely using the face. Other signs such as clothes or context will not influence the result.

The returned value is between 0 and 1, face with a minor value closer to 1 indicate that the person is minor while faces with a minor value closer to 0 indicate that the person is major

Use-cases

  • Remove profile pictures belonging to minors
  • Prevent users from posting pictures /videos of their children
  • Blur child faces in images or videos
  • Group or classify your images based on content

15-20 year olds

Determining if someone is 17 or 19 based on a single face can be tricky. The API has therefore been developed to show that its confidence is low whenever it encounters pictures of users that are visually close to 18. The "minor" value would in this case be close to 0.5.

Minor: (minor value 0.93)
Major: (minor value 0.04)

Sunglasses

For each detected face, the Face Attribute Model will return a "sunglasses" field that will help you determine if a face is covered with sunglasses or not.

The returned value is between 0 and 1, face with a sunglasses value closer to 1 indicate that the person wear sunglasses while faces with a sunglasses value closer to 0 indicate that the person doesn't wear sunglasses.

Use-cases

  • Detect if a person is wearing sunglasses
  • Prevent users from hiding their faces with sunglasses
  • Group or classify your images

Limitations

  • The limitations applicable to face detection remain relevant
Sunglasses: (sunglasses value 0.92)
No Sunglasses: (sunglasses value 0.01)

Use the model

To start, you need to create an account to retrieve your API keys. Then you must install the SDK that corresponds to your programming language.


# install cURL: https://curl.haxx.se/download.html


pip install sightengine


composer require sightengine/client-php


npm install sightengine --save

Detect face attributes

Let say you want moderate the following image:

You can either upload a public URL to the image, or upload the raw binary image. Here's how to upload the


curl -X GET -G 'https://api.sightengine.com/1.0/check.json' \
    -d 'models=face-attributes' \
    -d 'api_user={api_user}&api_secret={api_secret}' \
    -d 'url=https://d3m9459r9kwism.cloudfront.net/img/examples/example7.jpg'


 # if you haven't already, install the SDK with 'pip install sightengine'
from sightengine.client import SightengineClient
client = SightengineClient('{api_user}','{api_secret}')
output = client.check('face-attributes').set_url('https://d3m9459r9kwism.cloudfront.net/img/examples/example7.jpg')


// if you haven't already, install the SDK with 'composer require sightengine/client-php'
use \Sightengine\SightengineClient;
$client = new SightengineClient('{api_user}','{api_secret}');
$client->check(['face-attributes'])->set_url('https://d3m9459r9kwism.cloudfront.net/img/examples/example7.jpg');


// if you haven't already, install the SDK with 'npm install sightengine --save'
var sightengine = require('sightengine')('{api_user}','{api_secret}');
sightengine.check(['face-attributes']).set_url('https://d3m9459r9kwism.cloudfront.net/img/examples/example7.jpg').then(function(result) {
    // The result of the API
}).catch(function(err) {
    // Error
});

Here is the result:

                    
                    
{
    "status": "success",
    "request": {
        "id": "req_0MsK5ptZx713xt5aRmckl",
        "timestamp": 1494406445.3718,
        "operations": 1
    },
    "faces": [
        {
            "x1": 0.5121,
            "y1": 0.1879,
            "x2": 0.6926,
            "y2": 0.6265,
            "features": {
                "left_eye": {
                    "x": 0.6438,
                    "y": 0.3634
                },
                "right_eye": {
                    "x": 0.5578,
                    "y": 0.3714
                },
                "nose_tip": {
                    "x": 0.6047,
                    "y": 0.4801
                },
                "left_mouth_corner": {
                    "x": 0.6469,
                    "y": 0.5305
                },
                "right_mouth_corner": {
                    "x": 0.5719,
                    "y": 0.5332
                }
            },
            "attributes": {
                "female": 0.96,
                "male": 0.04,
                "minor": 0.01,
                "sunglasses": 0.01
            }
        }
    ],
    "media": {
        "id": "med_0MsK3A6i2vNxQgHkc11j9",
        "uri": "https:\/\/sightengine.com\/assets\/img\/examples\/example7.jpg"
    }
}
                    
                

Did you find this page helpful?

We're always looking for advice to help improve our documentation!

Let us know what you think