Home
/
Chat
/
Platform API
/
Application

Filter and moderation

To protect your users from harmful language and unsafe content, Sendbird provides a set of filters and a variety of moderation methods. The filters are designed to detect with precision anything that is seemingly inappropriate in user-submitted messages, images, and so on. The moderation methods help you reduce the need for human reviews and submission approvals. With the supported filters and moderation methods, you can determine the level of suitability of language and content for your Sendbird application.


Domain filter

The domain filter allows you to set the domains to be detected in URL-containing text and file messages as well as users’ profile images. It will filter the detected domains according to your policies and criteria.

HTTP request

Light Color Skin
Copy
// The application-wide settings.
PUT https://api-{application_id}.sendbird.com/v3/applications/settings_global

// The settings only applied to channels with a custom channel type.
PUT https://api-{application_id}.sendbird.com/v3/applications/settings_by_channel_custom_type/{custom_type}

Parameters

The following table lists the parameters that this action supports.

Parameters
RequiredTypeDescription

custom_type

string

Specifies the custom channel type to apply a set of settings to channels with the corresponding type.

Request body

The following table lists the properties of an HTTP request that this action supports.

Properties
OptionalTypeDescription

domain_filter

nested object

A domain filter configuration to filter out text and file messages with URLs that contain the domain set. This also filters out a user's profile image URL if it matches the domain set.

domain_filter.domains[]

array

Specifies an array of domains to detect. Each item of the array is specified at least with a combination of domain name and TLD (top level domain) like 'amazon.com'.

* Moderation for domains applies only to the message property of a text or file message resource, not the data property or the metaArrays property.

domain_filter.type

int

Determines which filtering mode to apply to messages with URLs that contain any of the domain set. Acceptable values are limited to the following:
- 0 (none): takes no action on matching messages. This is the default setting.
- 1 (allow): only allows messages containing URLs that match the domains property.
- 2 (block): blocks messages containing URLs that match the domains property.
- 3 (replace): detects and replaces URLs that match the domains property with asterisks (*).

domain_filter.should_check_global

boolean

Determines whether to apply the domain filter of the global application settings in addition to the settings above for the custom channel type. For the application, the property value is always set to false. (Default: false)

application wide
channels with custom type
Light Color Skin
Copy
# Request body example for turning on the filter at an application-wide level
{
    "domain_filter": {
        "domains": ["casinobetting.com", "betonhorserace.com"],
        "type": 2
    }
}
Light Color Skin
Copy
# Request body example for turning on the filter only effective to channels with a custom channel type
{
    "custom_type": "Sports",
    "domain_filter": {
        "domains": ["esqn.com", "sports.yaho.com"],
        "type": 2,
        "should_check_global": true
    }
}

If you want to turn off the domain filter, send a PUT request with the blank domains property like below:

application wide
channels with custom type
Light Color Skin
Copy
# Request body example for turning on the filter at an application-wide level
{
    "domain_filter": {
        "domains": []
    }
}
Light Color Skin
Copy
# Request body example for turning on the filter only effective to channels with a custom channel type
{
    "custom_type": "Sports",
    "domain_filter": {
        "domains": []
    }
}

Response

If successful, this action returns the updated settings of an application or a custom channel type in the response body.


Profanity filter

The profanity filter allows you to configure which profanity words or patterns to be detected in text and file messages, and determine how to process them according to your policies and criteria.

HTTP request

Light Color Skin
Copy
// The application-wide settings.
PUT https://api-{application_id}.sendbird.com/v3/applications/settings_global

// The settings only applied to channels with a custom channel type.
PUT https://api-{application_id}.sendbird.com/v3/applications/settings_by_channel_custom_type/{custom_type}

Parameters

The following table lists the parameters that this action supports.

Parameters
RequiredTypeDescription

custom_type

string

Specifies the custom channel type to apply a set of settings to channels with the corresponding type.

Request body

The following table lists the properties of an HTTP request that this action supports.

Properties
OptionalTypeDescription

profanity_filter

nested object

A filter configuration on certain words and patterns for matching character combinations in strings, which are not allowed to be used within the application.

profanity_filter.keywords[]

array

Specifies an array of words to detect. *word filters all words that end with "word" including "word" itself while word* filters all words that start with "word" including "word" itself.

profanity_filter.regex_filters[]

list

Specifies a list of regular expressions used for detecting. Each item of the list is specified in {"regex": "a pattern in regular expression"} format.

profanity_filter.type

int

Determines which filtering method to apply to messages that contain the specified keywords or regular expressions. Acceptable values are limited to the following:
- 0 (none): takes no action on matched messages. This is the default setting.
- 1 (replace): detects and replaces words that match the keywords property with asterisks (*).
- 2 (block): prevents users from sending messages that contain the keywords property or match the regex_filters property.

profanity_filter.should_check_global

bolean

Determines whether to apply the profanity filter of the global application settings in addition to the settings above for the custom channel type. For the application, the property value is always set to false. (Default: false)

application wide
channels with custom type
Light Color Skin
Copy
# Request body example for turning on the filter at an application-wide level
{
    "profanity_filter": {
        "keywords": "ass,snatch,sperm",
        "regex_filters": [
            {
                "regex": "[^!@#$%^&*]*(fuck|prick)[^!@#$%^&*]*"
            },
            {
                "regex": "(http://|https://)?(casino|sex)+([-.]{1}[a-z0-9]+)*.[a-z]{2,5}(:[0-9]{1,5})?(/.*)?"
            }
        ],
        "type": 2
    }
}
Light Color Skin
Copy
# Request body example for turning on the filter only effective to channels with a custom channel type
{
    "custom_type": "Sports",
    "profanity_filter": {
        "keywords": "idiot,dummy,damn",
        "regex_filters": [
            {
                "regex": "[^!@#$%^&*]*(bitch|assole)[^!@#$%^&*]*"
            },
            {
                "regex": "(http://|https://)?(casino|sex|betting)+([-.]{1}[a-z0-9]+)*.[a-z]{2,5}(:[0-9]{1,5})?(/.*)?"
            }
        ],
        "type": 2,
        "should_check_global": true
    }
}

If you want to turn off the profanity filter, send a PUT request with the blank keywords and regex_filters properties like below:

application wide
channels with custom type
Light Color Skin
Copy
# Request body example for turning off the filter at an application-wide level
{
    "profanity_filter": {
        "keywords": "",
        "regex_filters": []
    }
}
Light Color Skin
Copy
# Request body example for turning off the filter only effective to channels with a custom channel type
{
    "custom_type": "Sports",
    "profanity_filter": {
        "keywords": "",
        "regex_filters": []
    }
}

Response

If successful, this action returns the updated settings of an application or a custom channel type in the response body.


Profanity-triggered moderation

Based on the profanity filter, this feature moderates the users who are sending profanity words to a channel. You can apply different profanity-triggered moderation to appllication-wide and channels with a custom type separately. You can also adjust the level of the moderation by configuring the number of violation limit, the time window for counting violations, and the type of moderation penalty.

HTTP request

Light Color Skin
Copy
// The application-wide settings.
PUT https://api-{application_id}.sendbird.com/v3/applications/settings_global

// The settings only applied to channels with a custom channel type.
PUT https://api-{application_id}.sendbird.com/v3/applications/settings_by_channel_custom_type/{custom_type}

Parameters

The following table lists the parameters that this action supports.

Parameters
RequiredTypeDescription

custom_type

string

Specifies the custom channel type to apply a set of settings to channels with the corresponding type.

Request body

The following table lists the properties of an HTTP request that this action supports.

Properties
OptionalTypeDescription

profanity_triggered_moderation

nested object

A moderation configuration on which penalty is automatically imposed on users who reach the profanity violation limit within a channel.

profanity_triggered_moderation.count

int

Specifies the number of profanity violation limit which then imposes a penalty on a user if reached. A value of 0 indicates that automatically-triggered moderation is turned off. A value of equal to or larger than 1 indicates that the moderation is turned on and imposes a penalty on a user who commits a number of violations equal to or beyond the set value of the count property within the set time of the duration property.

profanity_triggered_moderation.duration

int

Specifies the duration of the time window in seconds which counts the number of a user’s violations within a channel. For example, if the count property is 2 and the duration property is 5, the number of violations equal to or beyond 2 will be moderated for every 5 seconds. The maximum value is 86400 which indicates 24 hours. (Default: 1 second)

profanity_triggered_moderation.action

int

Determines the type of moderation penalty within a channel which is permanently imposed on users until canceled. Valid values are 0 (no action), 1 (mute), 2 (kick), and 3 (ban). (Default: 0)

application wide
channels with custom type
Light Color Skin
Copy
# Request body example for turning on the moderation at an application-wide level
{
    "profanity_filter": {
        "keywords": "ass,snatch,sperm",
        "regex_filters": [
            {
                "regex": "[^!@#$%^&*]*(fuck|prick)[^!@#$%^&*]*"
            },
            {
                "regex": "(http://|https://)?(casino|sex)+([-.]{1}[a-z0-9]+)*.[a-z]{2,5}(:[0-9]{1,5})?(/.*)?"
            }
        ],
        "type": 2
    },
    "profanity_triggered_moderation": {
        "count": 5,
        "duration": 5,
        "action": 1
    }
}
Light Color Skin
Copy
# Request body example for turning on the moderation effective to channels with a custom channel type
{
    "custom_type": "Sports",
    "profanity_filter": {
        "keywords": "idiot,dummy,damn",
        "regex_filters": [
            {
                "regex": "[^!@#$%^&*]*(bitch|assole)[^!@#$%^&*]*"
            },
            {
                "regex": "(http://|https://)?(casino|sex|betting)+([-.]{1}[a-z0-9]+)*.[a-z]{2,5}(:[0-9]{1,5})?(/.*)?"
            }
        ],
        "type": 2
    }
    "profanity_triggered_moderation": {
        "count": 2,
        "duration": 10,
        "action": 2
    }
}

If you want to turn off the profanity-triggered moderation, send a PUT request with the count property of 0 like below:

application wide
channels with custom type
Light Color Skin
Copy
# Request body example for turning off the moderation at an application-wide level
{
    "profanity_triggered_moderation": {
        "count": 0
    }
}
Light Color Skin
Copy
# Request body example for turning off the moderation only effective to channels with a custom channel type
{
    "custom_type": "Sports",
    "profanity_triggered_moderation": {
        "count": 0
    }
}

Response

If successful, this action returns the updated settings of an application or a custom channel type in the response body.


Image moderation

Image moderation is one of Sendbird's premium features. Contact our sales team for further assistance.

Sendbird’s image moderation feature is powered by Google Cloud Vision API. This feature is for moderating the text and file messages with explicit images or inappropriate image URLs, and uses five categories such as adult, spoof, medical, violence, and racy. Image moderation doesn't apply when uploading channel images or profile images to Sendbird server.

After an image is uploaded and moderated, the feature returns limit values. These numbers range from one to five for each category and corresponds to how likely the image will be blocked. The following shows what each limit means:

LimitDescription

1 (very unlikely)

The likelihood of the image getting blocked is very unlikely.

2 (unlikely)

The likelihood of the image getting blocked is unlikely.

3 (possible)

The likelihood of the image getting blocked is possible.

4 (likely)

The likelihood of the image getting blocked is likely.

5 (very likely)

The likelihood of the image getting blocked is very likely.

You can test different images on the Google Cloud Vision API's try-it tool to see how moderation works and determine which image moderation settings suit your needs.

Note: This feature may not work on culturally sensitive images such as those related to religion, drugs or weapons.

HTTP request

Light Color Skin
Copy
// The application-wide settings.
PUT https://api-{application_id}.sendbird.com/v3/applications/settings_global

// The settings only applied to channels with a custom channel type.
PUT https://api-{application_id}.sendbird.com/v3/applications/settings_by_channel_custom_type/{custom_type}

Parameters

The following table lists the parameters that this action supports.

Parameters
RequiredTypeDescription

custom_type

string

Specifies the custom channel type to apply a set of settings to channels with the corresponding type.

Request body

The following table lists the properties of an HTTP request that this action supports.

Properties
OptionalTypeDescription

image_moderation

nested object

A moderation configuration on inappropriate images within the application. This feature is powered by Google Cloud Vision API, which supports various image types.

image_moderation.type

int

Determines the moderation method which is applied to the images and image URLs in the text and file messages. Acceptable values are limited to the following:
- 0 (none): no moderation is imposed. This is the default setting.
- 1 (normal): the messages with images or image URLs are blocked if the images or image URLs violate the content policies.
- 2 (strict): the file messages with no images are also blocked in addition to the messages with explicit images or inappropriate image URLs.

image_moderation.soft_block

boolean

If true, the moderation method set by the type property above is ignored and no moderation is imposed on the text and file messages in regard to explicit images or inappropriate image URLs. It will only give the image analysis results in the response. If false, the image moderation works according to the moderation method already set.

image_moderation.limits

nested object

A set of values returned after an image has been moderated. These limit numbers range from one to five and specify the likelihood of the image passing the moderation standard. (Default: adult: 3, spoof: 5, medical: 5, violence: 3, racy: 4)

image_moderation.limits.adult

int

Specifies the adult content likelihood for the image.

image_moderation.limits.spoof

int

Specifies spoof likelihood.

image_moderation.limits.medical

int

Specifies likelihood that the image is a medical image.

image_moderation.limits.violence

int

Specifies likelihood that the image contains violent content.

image_moderation.limits.racy

int

Specifies likelihood that the image contains racy content.

image_moderation.check_urls

boolean

Determines whether to check if the image URLs in the text and file messages are appropriate.

* Moderation for image URLs applies only to the message property of a text or file message resource, not the data property.

* This check_urls property can filter URLs of inappropriate images but it can't moderate URLs of websites containing questionable images. For example, image search results of adult images on 'google.com' will not be filtered.

application wide
channels with custom type
Light Color Skin
Copy
# Request body example for turning on the moderation at an application-wide level
{
    "image_moderation": {   // For more information, see the 'Properties' table above.
        "type": 1,  // The 'normal' moderation method.
        "soft_block": false,
        "limits": {
            "adult": 3,
            "spoof": 5,
            "medical": 5,
            "violence": 3,
            "racy": 4
        },
        "check_urls": true  // Check if the image URLs in messages are appropriate.
    }
}
Light Color Skin
Copy
# Request body example for turn off the moderation only effective to channels with a custom channel type
{
    "custom_type": "Sports",
    "image_moderation": {   // For more information, see the 'Properties' table above.
        "type": 2,  // The 'strict' moderation method.
        "soft_block": false,
        "limits": {
            "adult": 4,
            "spoof": 5,
            "medical": 5,
            "violence": 5,
            "racy": 5
        },
        "check_urls": true  // Check if the image URLs in messages are appropriate.
    }
}

If you want to turn off the image moderation, send a PUT request with the type property of 0 like below:

application wide
channels with custom type
Light Color Skin
Copy
# Request body example for turning off the moderation at an application-wide level
{
    "image_moderation": {
        "type": 0
    }
}
Light Color Skin
Copy
# Request body example for turning off the moderation only effective to channels with a custom channel type
{
    "custom_type": "Sports",
    "image_moderation": {
        "type": 0
    }
}

When a message has passed through image moderation, Sendbird server sends back a success response body like the following:

application wide
channels with custom type
Light Color Skin
Copy
# Response body example when a message has passed through image moderation
{
    ...
    "moderation_action": 1,
    "moderation_info": {
        "action": 1,
        "message": "Success",
        "scores": { // Image analysis results.
            "adult": false,
            "spoof": false,
            "medical": false,
            "violence": false,
            "racy": false
        },
        "cache_hit": false
    },
    ...
}
Light Color Skin
Copy
# Response body example when a message has passed through image moderation
{
    "custom_type": "Sports",
    ...
    "moderation_action": 2,
    "moderation_info": {
        "action": 2,
        "message": "Success",
        "scores": { // Image analysis results.
            "adult": false,
            "spoof": false,
            "medical": false,
            "violence": false,
            "racy": false
        },
        "cache_hit": false
    },
    ...
}

When a message has been blocked by image moderation, Sendbird server sends back an error response containing the 900066 (ERROR_FILE_MOD_BLOCK) or 900065 (ERROR_FILE_URL_BLOCK) code.

Response

If successful, this action returns the updated settings of an application or a custom channel type in the response body.