Skip to content

Demo — Community platforms and bots

Moderation bot.

A practice channel with a bot watching it. Write anything: a small model scores it in your browser, the channel’s rules and the policy decide what happens, and you see why. Nothing you type leaves your device.

Checking what this device can run…

How it works

From a message to a decision, in milliseconds.

Every message goes through four steps before anyone else sees it. Here is one of them, as the bot handles it.

  1. A member writes

    i fk u

    1. Step 1

      Read it as meant

      Short forms, masks and lookalikes are spelt out, so “fk” and “f*ck” can’t slip past.

      “i fuck you”

      Our rules · instant

    2. Step 2

      Score it

      A small open model gives six scores between 0 and 1.

      Toxic 1.00 · Insult 0.76

      MiniLMv2 · 24 MB · 7 ms

    3. Step 3

      Check the channel’s rules

      Veiled threats, links, spam and the words this community bans.

      No rule needed: the scores are clear

      Your rules · instant

    4. Step 4

      Decide, and explain

      The policy picks an action at this channel’s strictness, and says why.

      Insult over the 0.60 line

      Policy · Standard

  3. The bot

    Hidden, author warned. A moderator can approve it.

downloaded once with the engine that runs it, then kept by your browser
50 MBdownloaded once with the engine that runs it, then kept by your browser
to score a message on an ordinary laptop
7 msto score a message on an ordinary laptop
of what you type sent anywhere
0 bytesof what you type sent anywhere

What it can’t do

Honest limits.

  • It was trained on English. Insults that look like English are caught in other languages too (“eres un idiota”), but most others get through.
  • Like most public toxicity models, it gives harmless messages that name an identity (“gay”, “Muslim”, “Black”) higher identity hate scores than they deserve. That is why identity hate goes to a person and is never hidden automatically. Try the “As a gay man” example to see it.
  • On its own the model misses veiled threats such as “watch your back”: the channel’s rules catch the common ones. Turn that rule off and try the example again.
  • It judges one message at a time, so it misses sarcasm, running jokes and harassment spread over many messages. For clients we use larger models, add the community’s own examples, tune the lines with its moderators, and look at conversations as well as single messages.

About this demo

Small on purpose.

The demos here download 50 MB to 1.0 GB: small enough to fit into a web page and run on your own device, with nothing sent to us or to an AI company. Your browser keeps the files, so a second visit starts from that copy.

They do a great deal for their size, and they make mistakes a larger model wouldn’t. What we build for clients uses larger models where they’re needed, the client’s own data and words, testing before launch, and a person to hand over to.

The files stay in your browser until you remove them, or the browser needs the space. Removing them here clears every demo’s models from this device; a demo downloads its own again if you load it.

← All capability demos