Text Moderation / Principles

Text Moderation Principles

Introduction

Sightengine's Text Moderation API is useful to moderate any type of text contents: comments, messages, chats, posts and even usernames.

The categories of content that are detected are:

  • Profanity: insults, discriminatory content, sexual content or other inappropriate words and phrases.
  • Personal Details: typically email addresses, phone numbers, IPs, US social security numbers etc. that qualify as personally identifiable information and should be hidden. More on PII detection.
  • Misleading usernames: usernames that intend to mislead other users or cause running errors because they mimic
  • Custom blacklist: specific words that you need to catch. Could be words specific to your community, could be a list of competitor names, etc.

You can also implement a custom whitelist to force our API to disregard any words or content you feel shouldn't be flagged.

Detection strength

Our Text Moderation is a lot stronger than word-based filters. It uses advanced language analysis to detect objectionable content, even when users specifically attempt to circumvent your filters.

As an example, for each word we will be looking up millions of variations that might be used to evade filtering, while smartly ignoring all situations that might generate false positievs. Here is a partial list of the situations that we cover:

druuugggggggs
Repetitions

Characters being repeated to avoid basic word filtering

$#!t
Grawlix

Replacement of characters with typographical symbols

B__* 0 __ 0 -- B__s
Insertions

Adding spaces, punctuation and more within words

🅓rͬu̸🄶s̼
Obfuscation and Special characters

Unusual non-ASCII characters used to evade basic word filters

phok yu
Spelling mistakes & Phonetic Variations

Changing word spellings while retaining their original meaning or pronunciation

|)R|_|G5
Leet speak

Replacing some alphabetical characters with a combination of punctuation, digits and letters

123FuckBlablah
Smart embeddings

Catching profanity based embeddings, while smartly ignoring potential false positives such as bassguitar amass...

Did you find this page helpful?

We're always looking for advice to help improve our documentation!

Let us know what you think

Cookies help us deliver our services. By using our services, you agree to our use of cookies. Learn more

OK