LayerSift
Start free

Search the docs

Find a page or a section of one

Protecting forms from spam

Comment threads, reviews, contact forms, listings, sign-up bios: anywhere a visitor can write something, a script will post what it is paid to post (OWASP calls it spamming, OAT-017). It comes as a campaign — one message, or a spinner's variants of it, from address after address — so each address posts once or twice and stays under any per-address limit. What gives it away is that the message is the same.

An assessment with content — what was written — sees that sameness, and remembers what your moderators decided.

#What is being watched

  • One message from many senders. Every message is fingerprinted and compared with the last hour's. Three addresses posting it, or one near enough, is a reason (content:repeated, 30 — 45 with a link in it, which is what spam is for); ten is a challenge on its own (60). Near enough means sharing at least half their phrases: a link retagged with a new tracking id is the same message, and so is a variant with a greeting or a few words swapped.
  • A message your moderators reported. Report a message as spam and its close copies are known for seven days, from wherever they are posted next (content:reportedSpam, 60: a challenge on its own, a block beside anything else). Close means sharing four fifths of their phrases or more — closer than a campaign is counted by — and linking to the same hosts: a petition with a spammer's link added is near the petition, and reporting it must not report the petition. Where verdicts disagree, the nearest message's stands.
  • A message your moderators approved. A petition or a form letter that many real people copy looks like a campaign. Approve one and its near-duplicates are not held against their senders for thirty days (approved).
  • Many posts from one address. Thirty in an hour is a reason (velocity:postsFromAddress, 30), a hundred a challenge (60). From one device (velocity:postsFromDevice), at the same marks: one browser behind a proxy pool is an address per post, and one device for all of them, counted by the session token's visitor. Seen both ways, it is counted once. Across an IPv6 /48 (velocity:postsFromNetwork), at sixteen times those marks: a fresh /64 of a tunnel broker's free /48 for every few posts, unless the devices that ran the agent there make it a campus's or a carrier's.
  • Many links. Five or more in one message is a reason (content:manyLinks, 15).
  • The client, as for every assessment.

A message under eight words is too short to tell a campaign from a common phrase — "thanks, great post" — and is never counted as repeated. A message is read as a reader sees it: characters that are not drawn at all — zero-width spaces, soft hyphens — are dropped, and Cyrillic and Greek letters drawn like Latin ones are read as those, so a spinner sprinkling invisible characters through a message, or writing "сheap" with a Cyrillic с, has not written a new one.

The sameness is the message's, so it falls on everyone who posts it. A receipt from identifying does not excuse it (a spam farm's browsers earn those), but a receipt earned by passing a challenge does: whoever passed is not asked again for posting a popular message. A message moderators reported is excused by nothing.

#What is kept

Never the text. Each message is read in memory and kept for two hours as a MinHash: its runs of three words hashed 24 ways under a key of this deployment, keeping the smallest of each — 24 numbers that estimate how much two messages have in common, and give back nothing of either. Without the key, not even whether a given text was posted can be asked of them. A moderator's verdict is kept the same way: seven days for spam, thirty for an approval.

How near is near is a trade, measured. On a spinner's variants of a forty-word message the share of phrases in common ran from 0.6 to 1.0, and between unrelated messages — including two that open with the same greeting — a tenth or less; the threshold is half. Over 400 keys, a variant with a link retagged or a greeting swapped was found every time, one with two phrases reworded 99 times in 100, and one with its whole opening rewritten 94. A short message (fifteen or twenty words) with a word changed is found most of the time rather than always. Two different messages from one template ("Is the shop open on Sunday? I would like to…") share about four tenths and are rarely counted together — which is still why three senders is only a reason, and ten a challenge a person can pass.

#At the edge, no code

With the edge enforcer in front of the site, name the forms and the field that carries each message:

jsonc
"ANTIBOT_CONTENT_FIELDS": "/reviews=review.body,/contact=message"

Every POST to them is assessed with what was written — read from a copy of the request, nothing else of the body — and the verdict reaches your application in antibot-assess-* headers, so a challenge can hold a message for moderation rather than refuse it. Moderators' verdicts still come from your server, with reportContentOutcome.

#From your server

python
verdict = antibot.assess(
    ip=client_ip,
    headers=raw_header_pairs,
    method="POST",
    url="https://shop.example/reviews",
    content=form["review"],
    session_token=request.cookies.get("_abid"),
)
if verdict["action"] == "block":
    abort(403)
if verdict["action"] == "challenge":
    hold_for_moderation()      # or show a challenge before accepting it

And when a moderator decides, with the address the message was posted from:

python
antibot.report_content_outcome(ip=review.ip, content=review.text, outcome="failure")   # spam
antibot.report_content_outcome(ip=review.ip, content=review.text, outcome="success")   # not spam

The same in every server SDK: assess with content, and reportContentOutcome. The response carries:

json
"content": {
  "words": 41, "links": 1, "linkHosts": 1,
  "sendersOfContent": 6, "postsFromAddress": 1, "postsFromDevice": 0, "postsFromNetwork": 0,
  "reportedSpam": false, "approved": false
}

and rules read every field as content.*. The editor has Hold back repeated messages with links (three senders and a link, unless approved: a challenge, once — whoever passed it is not asked again, as the scoring does not ask them) and Block messages moderators reported as spam.

#Knowing it is happening

Edge traffic names the reasons, and the protection checklist on the Proxy page says whether messages are being assessed and moderators' verdicts reported — the verdicts are what turn one moderator's click into every copy caught.