Ethos Guard — Multimodal Speech Risk Analytics

The Problem

Public speakers and influencers in Pakistan operate in a code-switched mix of Urdu, Roman Urdu and English — a register that mainstream moderation tooling handles poorly. One careless generalisation about an ethnic or religious group can do real damage before anyone reviews it. Ethos Guard, built as part of a doctoral research programme at the University of Karachi, gives a speaker an advisory review of their own recorded session before they publish it.

What Makes the Analysis Different

• Identity is never treated as harm. Words like ethnic or religious group names never raise a flag alone — only the combination of a group reference, a hostile predicate and a generaliser ("yeh sab… ghaddar hain") does.
• Negation is respected, including Urdu grammar. "Hum nafrat ke khilaf hain" is recognised as counter-speech, not hostility — post-positional negation is handled, which keyword systems get wrong.
• Advisory only, by design. The system tells the speaker a segment touches a sensitive area and should be reviewed. It never issues legal determinations — a deliberate safety constraint, documented in the research.

Every flagged sentence is located in time, typed (group targeting, gendered contempt, sensitive domain…), scored for severity, and paired with a safer-wording hint — so the output is coaching, not censorship.

Multimodal, Not Text-Only

Text carries most of the signal, but not all of it. The engine also reads prosody — pitch, intensity, pace, pauses — and facial-expression probabilities, then fuses the three channels into one risk score that re-weights itself when a modality is missing.

Engineering Under the Hood

• Python analysis engine (negation-aware lexicons, pluggable toxicity model interface) behind a FastAPI service, with a React dashboard consuming live per-segment findings.
• A full research pipeline around it: Whisper-based transcription and segmentation, a rule-based Urdu-script → Roman Urdu transliterator, prosodic and facial feature extraction, and an inter-annotator agreement calculator verified against the literature.
• An evaluation harness with grouped cross-validation, bootstrap confidence intervals and paired significance tests — every number the research reports is reproducible from one command.
• 56 automated tests, including regression tests that pin the safety behaviours (counter-speech never flags; no statute names ever appear in output).

The Outcome

• A working end-to-end prototype: upload a session JSON, get an explained, per-segment risk review in seconds.
• The classic keyword-system failure — flagging "we reject hatred and violence" as incitement — demonstrably fixed and locked by tests.
• A defensible foundation for the research corpus and benchmark now being built on top of it.

This is an internal R&D project from our founder's doctoral research at the University of Karachi — shown here because it reflects how Baydot approaches ML products: honest naming, measurable claims, and safety constraints written into the code, not the marketing.