ActiveOS Updates: Real-Time Actioning, Detecting Sexual Solicitation, and Stopping Extremism

Over the past month, our product team has enhanced ActiveOS and ActiveScore with new features to help teams stay compliant and efficient, including new AI models and more. 

Let’s dive in:

 

Keep Your Platform Safe and Compliant by Combating Sexual Solicitation

 

Engaging in illegal activities like sex solicitation can have serious legal ramifications. Laws, such as  FOSTA (Allow States and Victims to Fight Online Sex Trafficking Act) and SESTA (Stop Enabling Sex Traffickers Act) are already in effect. Non-compliance can lead to fines, prosecution, and losing users who feel unsafe on your platform. 

While over-18 platforms like dating apps may allow some adult content, sex solicitation is illegal and will result in fines and user churn. General adult content or nudity models are not trained to identify monetary transactions and sexual services, so they won’t protect against sexual solicitation alone. 

ActiveScore’s advanced sex solicitation AI model automatically detects any content, communication, or depiction indicating illegal, sexually related transactions. It complies with SESTA/FOSTA laws and recognizes euphemisms, slang, emojis, and code words that conventional detection techniques often miss. 

 

Customized protection aligned with your policy

Use ActiveOS’ policy management tool to combine ActiveScore’s sex solicitation model with other models, including adult content, nudity, violative usernames, underage content, and more. This ensures wider protection that aligns with your platform’s community standards. 

For example, combine ActiveScore’s sex solicitation model with the violative usernames model to identify and remove users with sexually explicit usernames. You can also implement codeless workflows to automate responses, such as banning users or reporting illegal content to authorities. 

 

This approach helps stop violative users at the first touch and maintains a safe environment on your platform.

 

Continuous Improvement

Our models continuously improve through moderator feedback and retraining to align with real-world changes, new intelligence findings, and unique policies.

 

Benchmark Information:

Combat Extremism:  Advanced Detection to Keep Your Platform Secure

 

Extremist groups use online platforms to spread propaganda, radicalize, and incite violence. This activity is not only dangerous but also highly regulated by laws like the EU’s TERREG (preventing the dissemination of terrorist content online), which requires platforms to act on reported terrorist content within a specific time frame, such as removing content within 60 minutes. Failure to comply can result in fines of up to 4% of a platform’s global turnover. 

As terrorist content proliferates, Trust & Safety teams are faced with the challenge of identifying and catching it. Extremist groups continuously evolve their tactics, using subtle and coded imagery, making detection difficult.

 

How Our Model Helps:

• Advanced Detection: Our extremism model, fueled by intel insights and the largest database of extremist logos, automatically detects content from over 100 organizations, even as they evolve or use multiple logos. 
• Effective Mitigation: This helps prevent platforms from becoming havens for hate speech and violent extremism by swiftly and effectively identifying and removing such content.
• Comprehensive Coverage: Our model considers any visual content containing extremist logos as defined by the USA State Department’s Foreign Terrorist Organization (FTO) list and the UK’s Proscribed Terrorist Groups list. 

 

Updated intel insights for proactive threat detection:

Logos and symbols of extremist groups are constantly evolving, often within days. Since October 2023, Hamas, a US-recognized FTO, has changed the logos and symbols it uses in recruitment and propaganda content. Identifying these changes, our intelligence team trained the model on the new material, ensuring continuous and effective accuracy in detecting terrorist content.

Below you can see our benchmarks:

Enforce Actions in Real-Time Before Violations Reach Your Platform

 

Preventing policy violations before they reach your platform is crucial for maintaining a safe environment. Whether it’s abusive messages in a live chat or users creating profane usernames, real-time enforcement is possible with our new updates.

 

Key Features: 

• Real-Time Blocking: Our enhanced real-time actioning allows you to block or act against content before it’s published. 
• Customizable Actions: Configure actions based on whatever definitions you want, and send a callback with a label to your platform.
• Automatic Actions: Automatically notify users to change profane usernames or redact inappropriate messages in live chat before they’re sent. 

Below are a few more examples.

Our real-time actioning capabilities also support all of ActiveScore’s multilingual text models and custom keyword lists, enabling wider protection in real-time. 

Stay tuned, as we are always working on more exciting features and enhancements for ActiveOS and ActiveScore. If you’d like to see these new features in action, or learn more about them and other features, feel free to schedule a one-on-one demo session with us. 

 

Thanks,

The ActiveFence Team