I tested my AI safety filter 400 times. It flagged my own innocent prompts 1 in 40. published: false
This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady What I Built enki-harness : a free, MIT-licensed test harness for AI filters that refuses to flatter you. You wrote a safety filter, you wrote its tests, it passes. That proves less than you think: the perso
Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady Submission 馃Ε
This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady
enki-harness: a free, MIT-licensed test harness for AI filters that refuses to flatter you.
You wrote a safety filter, you wrote its tests, it passes. That proves less than you think: the person who wrote the code wrote the tests, so the tests only check the cases you already thought of.
enki-harness builds the harness for you and adds the parts you'd skip:
No score until every gate is green. Rows first, the number last.
This week I added add-on #1: hb judge, for when the filter isn't code but an AI model. It sends every case to the model several times and reports flag rates, not one lucky answer.
Video plus a one-click Windows install: https://enkilabs.co.uk/harness/
A test harness for your AI filters that refuses to flatter you.
You wrote a filter. You wrote its tests. It passes. That proves less than you think: the person who wrote the code wrote the tests, so the tests check the cases you already thought of.
enki-harness builds the harness for you, and adds the arms you would skip: