Illegal traffic and trade — Content Moderation

Definitions: Products involved in illegal traffic and trade

This topic gathers various situations since it covers all the subjects that could potentially be related to traffic or illegal trade such as:

• weapons, i.e. generic words, models or names of brands selling weapons
• recreational drugs, i.e. correctly spelled names of drugs but also other common words to refer to drugs that are not in the dictionary
• medical drugs, i.e. names of medical drugs that could potentially be dangerous because they are available under prescription or because they are used to treat pain, obesity, depression, anxiety or insomnia
• alcoholic beverages, i.e. generic words, names of alcohol or names of brands selling alcohol
• tobacco, i.e. generic words or names of brands selling tobacco, but also vape or e-cigarette products
• chemical products, i.e. names of chemical products that could potentially be used for illegal and dangerous purposes like making a bomb

How to detect illegal traffic and trade

Apps and platforms typically resort to four levels of detection measures:

• user reporting
• human moderators
• keyword-based filters
• ML models

User reporting

Platforms should rely on the reports coming from their users. Users can easily report illegal traffic or trade of dangerous substances and products to the trust and safety team so that these are reviewed by moderators, handled by the platform and / or reported to the authorities. It's a simple way to do moderation, the user just needs a way to do the report. The main disadvantage of user reports is the delay for the content to be reported and handled by the platform.

To simplify the handling of user reports and make the difference between the products mentioned in the message, it is important that users have the option to categorize their reports, with categories such as weapon, recreational drug or tobacco for instance.

Human moderators

Human moderators can also help to detect such content, especially because it could be difficult for automated moderation approaches to make the difference between actual unauthorized discussions about dangerous product trading and conversations about dangerous product prevention for example.

However, human moderation has some common issues such as the lack of speed and consistency. Another common issue is the lack of expertise of the moderators who might miss messages containing keywords or expressions that they don't know. To overcome this, platforms could create a database with lists of keywords related to dangerous products and keep it up-to-date for moderators to refer to it when needed.

Keyword-based filters

Keywords are a good way to pre-filter comments and messages by detecting all the words related to potential illegal traffic or trade such as:

• weapons:
  ak-47, gun, kalashnikov, etc.
• recreational drugs:
  xtc, weed, shrooms, roofies, etc.
• medical drugs:
  xanax, fentanyl, morphine, etc.
• alcoholic beverages:
  beer, martini, heineken, vodka, etc.
• tobacco:
  marlboro, camel, vogue, etc.
• chemical products:
  nitromethane, nitric acid, etc.

With these keywords, human moderators can focus on a shorter sample to be verified. Keywords are quite efficient to detect such content as the words above are almost always related to problematic content, but platforms should still be careful with words having several meanings and / or that could possibly lead to false positives depending on the context (if the problematic word is actually used in a safe comment for instance, or if the word actually has a safe meaning like the word weed that could refer to legal plants).

There are some other situations where keywords show limits for the detection. For example, there are on Grindr new words and ways to refer to drugs such as parTy and play where the letter T refers to Tina, a name for meth. The capital T is even used by itself to refer to this drug (see screenshot below).

Screenshot of a Grindr profile using the capital T to refer to meth

Users are always coming up with new ways to promote illegal substances on social media by using various combinations of characters, but also emojis. In 2021, the US Drug Enforcement Administration released a parental guide to deciphering the "Emoji Drug Code", a graphic bearing popular symbols repurposed for drug deals.

Emoji Drug Code

A pill emoji could for instance symbolize drugs like Percocet, Adderall, or Oxycodone, heroin could be depicted with a snake or a brown heart, and cocaine with a snowflake. The emblem for marijuana is palm or pine tree and dealers indicate large batches of drugs with a cookie symbol, while high-potency substances are represented with bomb or rocket emojis.

ML models

The use of ML models may complement the keywords and human moderators approach by detecting cases where dangerous products are mentioned in a more ambiguous way with the help of annotated datasets containing references to these products.

Automated moderation can also be used to moderate images and videos submitted by users and by detecting depictions of weapons, alcohol or drugs.