Now&Me

Designing the human layer around a moderation system

In a mental health community.

  • Content moderation
  • Human-in-the-loop design
  • Evaluation
  • Team building

The big picture

Now&Me was a community-based mental health startup where members could post anonymously or with a username. I was part of the team that turned its community guidelines into working rules, and I led it. I also led the six moderators who applied them, and I tracked the results. Violations fell significantly, by 60%. The system was a rule-based word list, not ML, so most of the work was deciding where a person had to step in.

Why it matters

Members could share anything about how they felt, but promoting self-harm, dating requests and spam were not allowed. A word list cannot tell the difference. Designing for that gap is the same problem AI products have today: knowing where automation is enough and where a human has to decide.

By the numbers

60%
Fewer spam posts and guideline violations.
6
Moderators, trained and led by me.
24/7
Moderation, with a mobile-friendly dashboard built by the in-house technical team.
Weekly
Tracking of total, flagged and blocked posts, week by week and month by month.

How it worked

Members typed their name to agree to the guidelines. The system flagged posts containing certain words or phrases. Obvious violations, like dating requests, were blocked automatically. The rest went to a moderator, who approved the post or did not allow it to go live. When a member broke a guideline, we told them and asked them to re-read the guidelines. Severe matters went to a WhatsApp group so the team could act fast.

Four design choices, and what each cost

  • Make the guidelines binding. Typing a name to agree felt more binding than ticking a box, and a reminder to re-read followed any violation. It acts before a post is ever written.
  • Block the obvious, review the rest. A word list is blunt in both directions. It flagged normal conversation that happened to include an abusive word, and it missed posts written in vague ways. So a person read the context. Cost: moderator time, and a dashboard that let people review any hour, from a phone.
  • Keep humans on the hard calls. Grey areas, like posts about self-harm, were decided as a team against the guidelines. Bans had no fixed number: we looked at the member's profile and analytics and decided case by case, because in a mental health community we wanted to be very sensitive. Cost: less consistency than a fixed rule. [Fill in: how the team decided the self-harm cases, in general terms.]
  • Turn every edge case into a rule. Each new case went into the rule book, which also showed what the guidelines were missing. Example: dating requests were banned, but some members asked for a "female friend" when they clearly wanted a date. We chose to remove that too, to keep the platform authentic.

How I led the team

I ran onboarding for every new moderator: the product, the guidelines, specific cases to know about, and a walkthrough of the dashboard. We met weekly to discuss cases, and used the group chat for anything severe.

Yes, but

  • A word list misses what is written in vague ways, and flags what is harmless.
  • Several changes landed together, so I can't say which one drove the 60%.

What this shows

  • Finding where automation fails and designing the human layer around it.
  • Turning a policy into rules, tooling and team habits.
  • Using edge cases as the way the system improves.
  • Making sensitive calls with judgment, not a blunt number.
  • Measuring a system, week by week and month by month.

What I'd build today

An LLM would read each post in context first and send only the unclear ones to moderators. People would keep the self-harm and ban decisions. The rule book and the logged edge cases would become the test set, and I'd track the same numbers, plus how often moderators overrule the model.