Why I Test the Attack Before I Write the Fix

A security fix isn't done when the code looks right. It's done when a test proves the exploit no longer works — and would fail again if the fix were ever reverted.

When I found a SQL injection bug on Atlas Market's search endpoint, the fix itself was a few lines — swap string interpolation for a parameterized query. The part that actually mattered was what I did first: I wrote a test that encoded the exact attack, watched it fail against the vulnerable code, then made it pass.

"It looks fixed" isn't a security claim

Reading a diff and deciding it looks correct is how vulnerabilities get reintroduced six months later by someone who never saw the original bug. A visual review of a fix tells you the code changed. It doesn't tell you the specific attack it was meant to stop can't happen again — and it definitely doesn't stop the next person from reintroducing the same pattern somewhere else in the codebase.

A regression test does both. It's an executable, permanent record of "this exact input used to break this exact thing," and CI runs it forever without anyone having to remember why it's there.

What this looks like in practice

For the SQLi fix, the test sent the actual malicious payload — a real DROP TABLE attempt — through the search endpoint and asserted the table still existed afterward. Not a mock, not a sanitized-input assumption — the real attack string, run against the real code path.

That's the pattern I use for security fixes generally:

• Reproduce the exploit first, in a test, before touching the fix. If you can't write a failing test that demonstrates the bug, you probably don't understand the bug well enough yet.
• Fix the root cause, not the symptom. Escaping a specific input pattern is a symptom fix; parameterized queries or proper validation at the boundary is the root cause fix.
• Confirm the test goes green, and that it would go red again if the fix were reverted — a test that passes for reasons unrelated to your fix isn't actually testing anything.
• Grep for the same pattern elsewhere. A vulnerability class rarely occurs exactly once. The SQLi fix above led to finding two more instances of the same anti-pattern in the same codebase.

A bug is not fixed until a test would fail without the fix.

Why this matters beyond the one bug

This isn't really about SQL injection specifically — the same logic applies to auth bypasses, broken access control, or any bug where "it seems to work now" is doing a lot of unearned trust. Security work that isn't backed by a test is a claim, not a fact. I'd rather ship a fact.

If you've got a codebase with security debt you're not sure how to prioritize, that's exactly the kind of audit I do — see my approach or get in touch.