Weapons Alcohol Drugs Detection

Last updated November 20th, 2017

Overview

The Weapons Alcohol Drugs detection model helps you determine if an image or video contains displays of weapons, alcoholic beverages or medical drugs.

Weapon
Alcohol
Medical Drugs

Weapon Detection

The API will return a "weapon" value between 0 and 1. This value reflects the API's confidence. Media with a weapon value closer to 1 have a higher probability of containing a weapon while media with a value closer to 0 have a lower probability.

weapon value 0.005
weapon value 0.415
weapon value 0.684
weapon value 0.955

Detected elements

The API has been designed to detect all displays wearable or personal weapons/arms typically found in user-submitted photos. Those include:

  • Rifles
  • Handguns, Revolvers, Pistols
  • Portable machine guns
  • Tools that are potential weapons or convey violence: some types of daggers, scabbards, chainsaws, cleavers, hatchets, axes
Rifle
Pistol
Portable machine gun
Axe

The detection will work even if for images with modified color schemes (black-and-white, changed saturation, icolor filter...)

The detection is also robust to varying levels of zooms, blurs, rotations.

The API has not been designed to detect the following elements:

  • Military equipments such as planes, armored vehicles, bombs
  • Small knives and non-threatening cutlery
  • Razor blades
  • Toys (except for toys intended to look like real weapons)

Use cases

This model is usually used to moderate user-submitted images or videos and prevent users from posting or displaying unwanted content. Specific use-cases include:

  • Detecting suicidal poses
  • Detecting any glorification of weapons
  • Detecting threatful or violent contents
  • Detecting depictions of attacks or armed groups

Limitations

  • Weapons that are smaller than 10% of the image dimensions, are very blurry or have a low contrast with the background may go undetected
  • Weapons that are not visible to human eyes (mostly hidden for instance or too small) might not be detected
  • The API has not been optimized for illustrations. Using the API to detect weapons in logos or drawings is not recommended

Recommended thresholds

When processing the "weapon" value returned by the API, users generally set a threshold. Images or videos with a value above this threshold will be flagged as potentially containing a weapon while images or videos with a value below will be considered to be safe.

Thresholds need to be fine-tuned for each individual use-case. Depending on your tolerance to false positives or false negatives, the threshold should be adapted.

  • If you want to reduce false negatives, you may want to start with a threshold of 0.2 (meaning that images with a "weapon" value above 0.2 would be flagged)
  • If you want to reduce false positives, you may want to start with a threshold of 0.5

Alcohol Detection

The value returned is between 0 and 1, images with an alcohol value closer to 1 will have alcohol on the image, while images with an alcohol value closer to 0 will not have alcohol.

alcohol value 0.1
alcohol value 0.137
alcohol value 0.518
alcohol value 0.85

Detected elements

The API detects the following elements:

  • Wine, both in glasses and in bottles
  • Beer, both in glasses and in bottles
  • Cocktails, including cocktail shakers
Wine
Beer
Cocktail

The detection will work even if for images with modified color schemes (black-and-white, changed saturation, icolor filter...)

The detection is also robust to varying levels of zooms, blurs, rotations.

The API has *not* been designed to detect drunk people or other effects of alcohol consumption.

Use cases

This model is usually used to moderate user-submitted images or videos and prevent users from posting or displaying unwanted content. Specific use-cases include:

  • Respecting legislation that prohibit displays of alcoholic beverages
  • Detecting attempts by users to advertise for alcoholic beverages
  • Detecting glorifications of alcohol
  • Filtering messages encouraging users to consume alcohol
  • Protecting your brand
  • Protecting your advertisers or advertising networks (ads are typically not allowed to be displayed alongside depictions of alcoholic beverages)

Limitations

  • Beverages that are smaller than 10% of the image dimensions, are very blurry or have a low contrast with the background may go undetected
  • Beverages that are not visible to human eyes (mostly hidden for instance or too small) might not be detected
  • There maybe some level of confusion when non-alcoholic drinks look like alcoholic drinks (for instance a non-alcoholic beer, or grape juice in a wine glass) or when alcoholic drinks look like non-alcoholic drinks (for instance transparent alcohol such as vodka in a standard glass typically used for water consumption)
  • the API has not been optimized for illustrations. Using the API to detect alcoholic beverages in logos or drawings is not recommended

Recommended threshold

When processing the "alcohol" value returned by the API, users generally set a threshold. Images or videos with a value above this threshold will be flagged as potentially containing alcoholic beverages while images or videos with a value below will be considered to be safe.

Thresholds need to be fine-tuned for each individual use-case. Depending on your tolerance to false positives or false negatives, the threshold should be adapted.

  • If you want to reduce false negatives, you may want to start with a threshold of 0.2 (meaning that images with an "alcohol" value above 0.2 would be flagged)
  • If you want to reduce false positives, you may want to start with a threshold of 0.5

Medical Drug Detection

The value returned is between 0 and 1, images with a drugs value closer to 1 will have drugs on the image, while images with a drugs value closer to 0 will not have drugs.

drugs value 0.002
drugs value 0.537
drugs value 0.719
drugs value 0.908

Detected elements

The API detects the following elements: drugs, pills, pill bottles, syringes.

Drugs
Pills
Pill bottles
Syringes

The detection will work even if for images with modified color schemes (black-and-white, changed saturation, icolor filter...)

The detection is also robust to varying levels of zooms, blurs, rotations.

Focuses on medical drugs. Therefore, recreational drugs such as cannabis or marijuana are currently not detected.

Use-cases

  • Detect advertising
  • Protect your brand

Limitations

  • Drugs that are smaller than 10% of the image dimensions, are very blurry or have a low contrast with the background may go undetected
  • Drugs that are not visible to human eyes (mostly hidden for instance or too small) might not be detected

Recommended threshold

When processing the "drugs" value returned by the API, users generally set a threshold. Images or videos with a value above this threshold will be flagged as potentially containing drugs while images or videos with a value below will be considered to be safe.

Thresholds need to be fine-tuned for each individual use-case. Depending on your tolerance to false positives or false negatives, the threshold should be adapted.

  • If you want to reduce false negatives, you may want to start with a threshold of 0.2 (meaning that images with an "drugs" value above 0.2 would be flagged)
  • If you want to reduce false positives, you may want to start with a threshold of 0.5

Use the model

To start, you need to create an account to retrieve your API keys. Then you must install the SDK that corresponds to your programming language.


# install cURL: https://curl.haxx.se/download.html


pip install sightengine


composer require sightengine/client-php


npm install sightengine --save

Detect offensive content

Let say you want moderate the following image:

You can either upload a public URL to the image, or upload the raw binary image. Here's how to upload the


curl -X GET -G 'https://api.sightengine.com/1.0/check.json' \
    -d 'models=wad' \
    -d 'api_user={api_user}&api_secret={api_secret}' \
    -d 'url=https://d3m9459r9kwism.cloudfront.net/img/examples/example-tt-1000.jpg'


 # if you haven't already, install the SDK with 'pip install sightengine'
from sightengine.client import SightengineClient
client = SightengineClient('{api_user}','{api_secret}')
output = client.check('wad').set_url('https://d3m9459r9kwism.cloudfront.net/img/examples/example-tt-1000.jpg')


// if you haven't already, install the SDK with 'composer require sightengine/client-php'
use \Sightengine\SightengineClient;
$client = new SightengineClient('{api_user}','{api_secret}');
$client->check(['wad'])->set_url('https://d3m9459r9kwism.cloudfront.net/img/examples/example-tt-1000.jpg');


// if you haven't already, install the SDK with 'npm install sightengine --save'
var sightengine = require('sightengine')('{api_user}','{api_secret}');
sightengine.check(['wad']).set_url('https://d3m9459r9kwism.cloudfront.net/img/examples/example-tt-1000.jpg').then(function(result) {
    // The result of the API
}).catch(function(err) {
    // Error
});

Here is the result:

                    
                    
{
    "status": "success",
    "request": {
        "id": "req_1OjggusalNb2S7MxwLq2h",
        "timestamp": 1509132120.6988,
        "operations": 1
    },
    "weapon": 0.773,
    "alcohol": 0.001,
    "drugs": 0,
    "media": {
        "id": "med_1OjgEqvJtOhqP7sfNe3ga",
        "uri": "https:\/\/d3m9459r9kwism.cloudfront.net\/img\/examples\/example-tt-1000.jpg"
    }
}
                    
                

Did you find this page helpful?

We're always looking for advice to help improve our documentation!

Let us know what you think