True AutoMod

Using AI to save True Digital $750,000 in annual content moderation costs

True Corporation is a communications and media conglomerate in Thailand, controlling the largest cable TV provider and internet service provider in the country.

Part of this conglomerate is True Digital Group; parent of the TrueID In-Trend platform, which allows registered users to contribute news items and reviews related to their city or community within a dedicated section of the True web portal. Each successful published article receives a payment of 100 THB (~3 USD).

Having hit 68,000 registered regular In-Trend contributors, True sought to vastly upscale their contributed content operation without having to contend with spiralling costs for content moderation. For this, they sought help from digital solutions experts Fluxus.

With True’s existing moderation workflow dependent on human moderators, Fluxus’ solution was to use Artificial Intelligence and machine learning in conjunction with third-party services to create a finely-tuned automated content moderation system.

Following a successful implementation by True, Fluxus’ newly-designed system took the average moderation cost per article down from $2 to $0.05, saving the company over $750,000 annually.

As well as the clear financial benefits, Fluxus’ automated system was also quicker - taking the average article moderation time down from seven days to seven minutes - and more effective as it was 91% more accurate than its human equivalent and reduced wastage from 72% to 0%

Delivering success at scale

From a starting base of 68,000 regular content contributors for their In-Trend platform, True set an ambitious target of reaching 8 million contributors. However, being able to facilitate this massive upscale solely relying on human moderators was instantly recognised as unachievable for several reasons.

With moderators able to review an average of 50-70 articles per day, approximately 5,000 would be required working around the clock to keep on top of the demand. Also, a moderator’s salary costs on average $1,000 per month - making an outlay of $5,000,000 on moderator salaries completely unviable.

Fluxus’ solution was to develop an automated moderation system that reduced the need for human evaluation on contributed articles.

Initially, this system would provide real-time sentiment analysis to contributors as they wrote, as well as running SEO checks and scores.

If an article passed those basic checks, it would advance towards further background checks, including AI based text and image plagiarism, image and text profanity checks, sentiment analysis, and other locally tailored filters.

If those vigorous checks passed, the article would publish automatically. However, if some checks failed or were inconclusive, the content was diverted to a human moderator for further assessment.

The new system could also automatically decline an article and provide feedback to the contributor so they could apply their own amendments.

A community moderation function was also developed where readers could report articles, and the system could automatically pay contributors, which eradicated the need for payment to be approved by humans.

The system was also much more cost effective, providing True with massive financial savings. Using human moderators, the average moderation cost per article was $2. With the automated system able to review articles at a much greater speed, Fluxus’ new design took the average article moderation cost down to $0.05.

For 1,000 articles a day, True’s moderation costs reduced from $2,000 to $50. Over the course of a year, these costs reduced from $750,000 to just $18,000, representing a 40x saving.

Creation & combination

To develop their automated moderation system, Fluxus used a combination of their own custom-built services combined with existing third-party services.

The Fluxus team designed their own bespoke profanity filter module that recognises an extensive list of words that have been red flagged by the In-Trend contributor community. An SEO evaluation system was also created, which analysed several aspects of an article, including text formatting, key words and word counts.

In order to implement sentiment analysis for articles, Fluxus used a number of AI tools, combining their own designs alongside third-party services.

Taking an existing AI model, Fluxus supplied the component with True’s already approved published content to allow the system to understand what was required for an article to be deemed acceptable.

Fluxus are investing in R&D for the tool - building a number of additional standalone features rather than using existing services. One such example is a text plagiarism analysis feature using ElasticSearch, web archives and machine learning to create a thorough review of the Thai internet.

Although existing text plagiarism systems already existed, using their own function meant Fluxus were able to drastically reduce costs. Each article search now cost an average of $0.01 compared to $1 using well known third-party services such as TurnItIn.

Image plagiarism was assessed using infringement reports, while image copyright checks were implemented using existing AWS service Rekognition.

Fluxus’ final concept was for contextual analysis, which saw passages of text instantly assessed and key words picked out so the article could be correctly tagged for its upload destination.

Results

Thanks to Fluxus’ innovations, True have successfully been able to massively upscale their contributed content moderation operation for the In-Trend platform effectively, cost effectively and intelligently.

Financial benefits

Alongside the clear financial benefits - reducing content moderation costs per article from $2 to $0.05, saving True more than $750,000 over the course of a year - Fluxus’ designs for automated tests have also proven to be 91% more accurate than their human equivalent, reducing True’s legal liability.

The future

Throughout 2020 & 2021, Fluxus will continue to develop new AI based features for Automod, covering content quality, video, audio, and text formatting and styling.

True is also looking to launch a similar In-Trend service in Indonesia and Philippines and thanks to the success of the new service in Thailand, Fluxus’ system will be implemented in the rest of the company’s operating regions.

From a manual process to

• 100% of content auto-moderated
• 40x cost saving
• Articles published in minutes
• AI based tooling
• Long term scalability
• In less than 6 months