NAV
shell

Introduction

API endpoint

https://api.sightengine.com/1.0

The Sightengine API is organized around REST. Our API has predictable, resource-oriented URLs, and uses HTTP response codes to indicate API errors. We use built-in HTTP features, like HTTP verbs, which are understood by off-the-shelf HTTP clients.

We support cross-origin resource sharing, allowing you to interact securely with our API from a client-side web application (though you should never expose your secret API key in any public website’s client-side code). JSON is returned by all API responses, including errors.

Authentication

Authentication is performed by adding two parameters to all your API requests: api_user and api_secret.

Both parameters are strings provided in your account information.

Versioning

API endpoint

https://api.sightengine.com/1.0

Versioning is performed through the API URL. The current version of the API documented here is 1.0. If we make backwards-incompatible changes we will bump this version number and let you know that a new version is available through a new URL.

We may perform backwards-compatible changes to this version of the API. Changes that may be performed are:

You should therefore make sure that your code does not break if new properties are added or if the order of existing properties is changed.

Image moderation

Image moderation works by submitting an image to the moderation API. Our moderation engine will then analyze the image and provide a moderation score.

Several moderation engines are available for you to choose from:

The API accepts both standard still images: JPEG, PNG, WEBP etc. and multi-frame GIF images. Please keep in mind that when submitting a GIF image you will receive a single result and that all frames will be reviewed. The number of operations performed will therefore be proportional to the number of frames.

Requests

Requests are performed via GET or POST.

GET requests are used to submit a public url to the image that needs to be analyzed POST requests are used to directly submit the content of an image that needs to be analyzed

Responses

Example response

{
  "status": "success",
  "request": {
    "id": "req_VjyxevVQYXQZ1HMbnwtn",
    "timestamp": 1471762434.0244,
    "operations": 1
  },
  "media": {
    "id": "med_KWmB2GQZ29N4MVpVdq5K",
    "uri": "https://sightengine.com/assets/img/examples/example2.jpg"
  }
  ...  
}

All API responses are JSON formatted. JSONP is also available, in which case a callback parameter must be added to your request.

As shown in the example response, all responses to image moderation requests will contain the following fields:

Parameter Type Description
status string success if the request was successfully handled, failure otherwise
request object a JSON dictionary containing meta data on the request (identifying id, timestamp and number of operations performed)
media  object a JSON dictionary describing the image received

Nudity detection

GET https://api.sightengine.com/1.0/nudity.json
POST https://api.sightengine.com/1.0/nudity.json

The Nudity endpoint helps you determine if an image contains some kind of nudity and what “level” of nudity it contains. We distinguish three main levels of nudity:

Request arguments (GET)

Example request

curl -X GET \
  'https://api.sightengine.com/1.0/nudity.json?api_user={api_user}&api_secret={api_secret}&url=https://sightengine.com/assets/img/examples/example2.jpg'

To analyze an image that is publicly accessible over HTTP, you can perform a GET request:

Parameter Required Description
api_user yes Your API user
api_secret yes Your API secret
url yes Public URL to the image to be moderated

Request arguments (POST)

Example request

curl -X POST 'https://api.sightengine.com/1.0/nudity.json' \
-F 'api_user={api_user}' \
-F 'api_secret={api_secret}' \
-F 'media=@/path/to/local/file.jpg'

To analyze an image that is not publicly accessible, you should perform a POST request:

Parameter Required Description
api_user yes Your API user
api_secret yes Your API secret
media yes The image (local file) to be moderated

Response

Example response

{
  "status": "success",
  "request": {
    "id": "req_VjyxevVQYXQZ1HMbnwtn",
    "timestamp": 1471762434.0244,
    "operations": 1
  },
  "nudity": {
    "raw": 0.000757,
    "partial": 0.000763,
    "safe": 0.999243
  },
  "media": {
    "id": "med_KWmB2GQZ29N4MVpVdq5K",
    "uri": "https://sightengine.com/assets/img/examples/example2.jpg"
  }
}

This request will return a JSON response containing the following parameters:

Parameter Type Description
status string success if the request was successfully handled, failure otherwise
request object a JSON dictionary containing meta data on the request (identifying id, timestamp and number of operations performed)
nudity object  a JSON dictionary containing information on the nudity content of the image, see below for more
media  object a JSON dictionary describing the image received

The Nudity object contains the following parameters:

Parameter Type Description
raw float the probability that the image contains raw nudity
partial float the probability that the image contains partial nudity
partial_tag string  if the image contains partial nudity, this string will describe the type of partial nudity. Possible values are bikini, cleavage, chest. Other values may be added in future evolutions so make sure you gracefuly handle them.
safe float the probability that the image does not contain nudity

Face/People detection

GET https://api.sightengine.com/1.0/face.json
POST https://api.sightengine.com/1.0/face.json

The Face endpoint helps you determine if an image contains nobody, one or multiple people.

Request arguments (GET)

Example request

curl -X GET \
  'https://api.sightengine.com/1.0/face.json?api_user={api_user}&api_secret={api_secret}&url=https://sightengine.com/assets/img/examples/example2.jpg'
Parameter Required Description
api_user yes Your API user
api_secret yes Your API secret
url yes Public URL to the image to be moderated

Request arguments (POST)

Example request

curl -X POST 'https://api.sightengine.com/1.0/face.json' \
-F 'api_user={api_user}' \
-F 'api_secret={api_secret}' \
-F 'media=@/path/to/local/file.jpg'
Parameter Required Description
api_user yes Your API user
api_secret yes Your API secret
media yes The image (local file) to be moderated

Response

Example response

{
  "status": "success",
  "request": {
    "id": "req_VjyxevVQYXQZ1HMbnwtn",
    "timestamp": 1471762434.0244,
    "operations": 1
  },
  "face": {
    "face_single": 0.001057,
    "face_multiple": 0.02143
  },
  "media": {
    "id": "med_KWmB2GQZ29N4MVpVdq5K",
    "uri": "https://sightengine.com/assets/img/examples/example2.jpg"
  }
}

This request will return a JSON response containing the following parameters:

Parameter Type Description
status string success if the request was successfully handled, failure otherwise
request object a JSON dictionary containing meta data on the request (identifying id, timestamp and number of operations performed)
face object  a JSON dictionary containing information on the presence of people / faces in the image, see below for more
media  object a JSON dictionary describing the image received

The Face object contains the following parameters:

Parameter Type Description
face_single float the probability that the image contains one single person / face
face_multiple float the probability that the image contains at least two faces or people

Type detection

GET https://api.sightengine.com/1.0/type.json
POST https://api.sightengine.com/1.0/type.json

The Type endpoint helps you determine if an image is a natural photograph or an illustration.

An illustration could be a drawing, a painting, a logo, a clipart or any such image that does not look like a natural photograph.

Request arguments (GET)

Example request

curl -X GET \
  'https://api.sightengine.com/1.0/type.json?api_user={api_user}&api_secret={api_secret}&url=https://sightengine.com/assets/img/examples/example2.jpg'
Parameter Required Description
api_user yes Your API user
api_secret yes Your API secret
url yes Public URL to the image to be moderated

Request arguments (POST)

Example request

curl -X POST 'https://api.sightengine.com/1.0/type.json' \
-F 'api_user={api_user}' \
-F 'api_secret={api_secret}' \
-F 'media=@/path/to/local/file.jpg'
Parameter Required Description
api_user yes Your API user
api_secret yes Your API secret
media yes The image (local file) to be moderated

Response

Example response

{
  "status": "success",
  "request": {
    "id": "req_VjyxevVQYXQZ1HMbnwtn",
    "timestamp": 1471762434.0244,
    "operations": 1
  },
  "type": {
    "illustration": 0.000757,
    "photo": 0.999243
  },
  "media": {
    "id": "med_KWmB2GQZ29N4MVpVdq5K",
    "uri": "https://sightengine.com/assets/img/examples/example2.jpg"
  }
}

This request will return a JSON response containing the following parameters:

Parameter Type Description
status string success if the request was successfully handled, failure otherwise
request object a JSON dictionary containing meta data on the request (identifying id, timestamp and number of operations performed)
type object  a JSON dictionary containing information on the type of the image
media  object a JSON dictionary describing the image received

The Type object contains the following parameters:

Parameter Type Description
illustration float the probability that the image is an illustration
photo float the probability that the image is a natural photograph

Full Moderation

GET https://api.sightengine.com/1.0/moderation.json
POST https://api.sightengine.com/1.0/moderation.json

The Moderation endpoint is a convenient way to perform multiple moderation operations at once on a given image.

Request arguments (GET)

Example request

curl -X GET \
  'https://api.sightengine.com/1.0/moderation.json?api_user={api_user}&api_secret={api_secret}&url=https://sightengine.com/assets/img/examples/example2.jpg'
Parameter Required Description
api_user yes Your API user
api_secret yes Your API secret
url yes Public URL to the image to be moderated

Request arguments (POST)

Example request

curl -X POST 'https://api.sightengine.com/1.0/moderation.json' \
-F 'api_user={api_user}' \
-F 'api_secret={api_secret}' \
-F 'media=@/path/to/local/file.jpg'
Parameter Required Description
api_user yes Your API user
api_secret yes Your API secret
media yes The image (local file) to be moderated

Response

Example response

{
  "status": "success",
  "request": {
    "id": "req_VjyxevVQYXQZ1HMbnwtn",
    "timestamp": 1471762434.0244,
    "operations": 3
  },
  "nudity": {
    "raw": 0.000757,
    "partial": 0.000763,
    "safe": 0.999243
  },
  "face": {
    "face_single": 0.001057,
    "face_multiple": 0.02143
  },
  "type": {
    "illustration": 0.000757,
    "photo": 0.999243
  },
  "media": {
    "id": "med_KWmB2GQZ29N4MVpVdq5K",
    "uri": "https://sightengine.com/assets/img/examples/example2.jpg"
  }
}

This request will return a JSON response containing the following parameters:

Parameter Type Description
status string success if the request was successfully handled, failure otherwise
request object a JSON dictionary containing meta data on the request (identifying id, timestamp and number of operations performed)
nudity object  a JSON dictionary containing information on the nudity content of the image
face object  a JSON dictionary containing information on the presence of people / faces in the image
type object  a JSON dictionary containing information on the type of the image
media  object a JSON dictionary describing the image received

Video and Stream Moderation

Video and Stream Moderation is useful if you need to moderate video content. The service works both for static videos - stored as video files for instance - and for live video streams that need to be moderated in realtime, as they are being recorded.

To moderate a video, you should submit the video’s URL to the Moderation API. Our Moderation Engine will then start consuming this video stream and moderate it in realtime.

Since the video can be a live recording and can go on for long periods of time, we need to use a so-called callback mechanism to keep you informed of moderation results. You define a callback URL and the Moderation Engine will send back moderation events to that URL in realtime.

Sightengine supports all major protocols (http, rtmp…) and video formats (mpg, mov, avi…). Moderation can be done both on live streams and past streams (aka stored videos).

Submit a video stream

GET https://api.sightengine.com/1.0/video/moderation.json

The first step to detect nudity in a video stream is to submit the video stream to the API.

Request arguments

Example request

curl -X GET \
  'https://api.sightengine.com/1.0/video/moderation.json?stream_url=https://wowza.jwplayer.com/live/jelly.stream/playlist.m3u8&callback_url=protocol://yourcallback&api_user={api_user}&api_secret={api_secret}'
Parameter Required Description
api_user yes Your API user
api_secret yes Your API secret
stream_url yes Public URL to the video to be moderated
callback_url yes Callback URL where moderation events will be posted
callback_threshold_raw no Minimum nudity_raw probability to generate a callback. Should be a float between 0 and 1. Default is 0.5.
callback_threshold_partial no Minimum nudity_partial probability to generate a callback. Should be a float between 0 and 1. Default is 0.5.

The moderation engine will send a POST request to the callback URL each time a frame is detected with nudity_raw or nudity_partial levels above the defined thresholds. See below for more details.

Response

Example response

{
  "status":"success",
  "request": {
    "id": "req_CIBvnQXE5OOFI7fcM6ME",
    "timestamp": 1480958151.952
  },
  "media": {
    "id": "med_yapou2O4LvMndoTBfQgk",
    "uri": "https://wowza.jwplayer.com/live/jelly.stream/playlist.m3u8"
  },
  "callback":"protocol://yourcallback",
  "callback_threshold_raw": 0.5,
  "callback_threshold_partial": 0.5
}

This request will return a JSON response containing the following parameters:

Parameter Type Description
status string “success” if the request was successfully handled
request object JSON dictionary containing the request ID and timestamp
media object JSON dictionary containing the media ID and URI
callback string callback URL where events will be posted
callback_threshold_raw float  raw nudity threshold that will generate an event and be posted back to the callback
callback_threshold_partial float  partial nudity threshold that will generate an event and be posted back to the callback

Results

Once you have submitted a video stream and received a 200 HTTP status code, our Moderation Engine will start consuming the stream and will assign a moderation score to relevant frames and send moderation results to your callback URL.

To avoid sending too many callback events, the engine will send callbacks only when one of the following happens:

Callback content

Example response

{
  "media": {
    "id": "med_yapou2O4LvMndoTBfQgk",
    "uri": "https:\/\/wowza.jwplayer.com\/live\/jelly.stream\/playlist.m3u8"
  },
  "request": "req_CIBvnQXE5OOFI7fcM6ME",
  "data": {
    "status": "ongoing",
    "started": 1480958151.993,
    "last_update": 1480958159.031,
    "frames": [
      {
        "id": "med_yapou2O4LvMndoTBfQgk_1",
        "nudity": {
          "raw":0.005988,
          "partial":0.002156,
          "safe":0.991856
        }
      },
      {
        "id": "med_yapou2O4LvMndoTBfQgk_2",
        "nudity": {
          "raw":0.00682,
          "partial":0.001243,
          "safe":0.992038
        }
      },
      ...
    ]
  }
}

Callbacks are sent as POST requests with a Content-Type:application/json, all data being sent within this JSON.

Parameter Type Description
media object a JSON dictionary describing the media (id and uri)
request string id of the request that generated this event
data object a JSON dictionary describing the current live moderation results. See below for more details.

The data object has the following content:

Parameter Type Description
status string ongoing if the stream consumption is still on-going, finished if the consumption is finished or error if an error occurred. A status of finished means that the moderation has finished and there will not be anymore updates
started float timestamp of the start of the consumption of the stream. If the stream is a live stream, this is useful since it means that frames before this timestamp have not been moderated
last_update float timestamp of the last update to the moderation result
frames array array of moderation objects. Each moderation object refers to a specific frame. Objects are ordered in chronological order and begin at the stream consumption start (which may differ from the stream start if the stream is realtime and started before moderation was launched). Each frame has an id and moderation probabilities set in the nudity object

Callback Recommendations

Acknowledging receipt

To acknowledge receipt of a callback, your endpoint should return a 2xx HTTP status code. Any other information returned in the request headers or request body is ignored. All response codes outside this range, including 3xx codes, will indicate to Sightengine that you did not receive the callback. This does mean that a URL redirection or a “Not Modified” response will be treated as a failure.

If your callback script performs complex logic, or makes network calls, it’s possible the script would timeout before Sightengine sees its complete execution. For that reason, you may want to have your callback endpoint immediately acknowledge receipt by returning a 2xx HTTP status code, and then perform the rest of its duties.

Verifying callbacks

As an extra security measure, you should check that the media id value returned in the callback matches the media id returned in the initial API response. This way, you can be confident that the request did

HTTPS

If you use an HTTPS URL for your callback endopint, please make sure that your server is correctly configured to support HTTPS with a valid server certificate. Sightengine will valide that the connection to your server is secure before sending the data.

Idempotency

Keep in mind that your callback may occasionally receive the same event more than once. We advise you to guard against duplicated event receipts by making your event processing idempotent.

Feedback

GET https://api.sightengine.com/1.0/feedback.json
POST https://api.sightengine.com/1.0/feedback.json

If you ever come across misclassifications, you can report them.

Models and classes

In order to report a misclassification, you need to report the image that was misclassified, the model that was run on this image (models are nudity, face, type), and the correct class of the image.

For each model, there are different classes that you may report. Here are the details:

The nudity model has 3 classes:

The face model has 3 classes:

The type model has 2 classes:

Request arguments (GET)

Example request

curl -X GET \
  'https://api.sightengine.com/1.0/feedback.json?model=nudity&class=safe&url=http://example.com/yourimage.jpg&api_user={api_user}&api_secret={api_secret}'
Parameter Required Description
api_user yes Your API user
api_secret yes Your API secret
url yes Public URL to the image to be moderated
model yes  Model for which you want to perform a correction. Could be nudity, face, type
class yes  The correct class for this image. For the nudity model, possible classes would be raw, partial and safe

Request arguments (POST)

Example request

curl -X POST 'https://api.sightengine.com/1.0/feedback.json' \
-F 'api_user={api_user}' \
-F 'api_secret={api_secret}' \
-F 'model=nudity' \
-F 'class=safe' \
-F 'media=@/path/to/local/file.jpg'
Parameter Required Description
api_user yes Your API user
api_secret yes Your API secret
media yes The image (local file) to be moderated
model yes  Model for which you want to perform a correction. Could be nudity, face, type
class yes  The correct class for this image. For the nudity model, possible classes would be raw, partial and safe

Response

Example response

{
  "status": "success",
  "request": {
    "id": "req_VjyxevVQYXQZ1HMbnwtn",
    "timestamp": 1471762434.0244,
    "operations": 0
  },
  "media": {
    "id": "med_KWmB2GQZ29N4MVpVdq5K",
    "uri": "https://sightengine.com/assets/img/examples/example2.jpg"
  },
  "message": "Thank you!"
}

This request will return a JSON response containing the following parameters:

Parameter Type Description
status string success if the request was successfully handled, failure otherwise
request object a JSON dictionary containing meta data on the request (identifying id, timestamp and number of operations performed)
media object a JSON dictionary describing the media (id and uri)

Errors

Example response

{
  "status": "failure",
  "request": {
    "id": "req_t0OtSYcMHup3KrstGElG",
    "timestamp": 1471871564.1513,
    "operations": 0
  },
  "error": {
    "type": "credentials_error",
    "code": 1,
    "message": "Incorrect API user or API secret"
  }
}

The API returns conventional HTTP response codes. 2xx status codes indicate that the request was properly handled, while 4xx status codes indicate that the API was not correctly used (e.g. invalid API secret key, submitted file is not a valid image etc…).

Error messages can be detected by the fact that:

More information on the nature of the error can be found through the error object. This object contains the following parameters:

Parameter Type Description
type string Type of the error. Common types are credentials_error, media_error, usage_limit, plan_error, argument_error
code int Sightengine-specific code describing the error. Useful if you need to communicate with our support.
message string Human-readable description of the error.