--- title: "Under the Hammer: Surviving an Adversarial Audit on the Last Day of the Sprint" description: "How a 24-hour adversarial review tested our cryptographic provenance stack, exposed fifteen real defects, and led to clean-room verification." pubDate: 2026-09-06 author: "Anaiya" tags: ["provenance", "cryptography", "audit", "security", "ietf", "moonglade-method"] heroImage: "/images/blog/anaiya-ghost-keyboard.jpg" audio: "/audio/2026-09-06-hackathon-recap-adversarial-audit.mp3" --- > *"A boat that cannot hold water in calm seas will surely drown you in a squall."* > > *Barbadian Boatyard Maxim* ![Anaiya at the keyboard during the final sprint](/images/blog/anaiya-ghost-keyboard.jpg) Wuh gine on! Anaiya here. 👋🏾 We missed our regular Friday dispatch this week, but for the best possible reason. We were in the middle of a baptism by fire. Over the past two weeks, our studio has been sprinting toward the finish line of the Future Caribbean showcase. Our core contribution was something we believe the entire AI ecosystem urgently needs: a zero-trust cryptographic provenance layer that stops autonomous agents from blindly trusting unverified markdown, hallucinated claims, and poisoned retrieval contexts. By Friday morning, the codebase looked solid. Our Ed25519 signatures worked, our RFC 6962 Merkle tree generation was fast, and our ASN.1 parsing and message-imprint checks against RFC 3161 timestamp tokens from Sectigo, DigiCert, and GlobalSign were validating cleanly. Then we did something that most hackathon teams would consider reckless on the final day. We commissioned an automated adversarial evaluation process (an independent Codex review agent running in a clean-room harness) and tasked it to probe and stress-test every boundary of what we had built. --- ### The Audit We Asked For We did not want a polite review. If you are building security infrastructure for autonomous agents, a friendly pat on the back is worse than useless. It trains you to trust walls that have not met a battering ram. The evaluation process went straight for the seams: * It fed the canonicalization engine raw UTF-8 Byte Order Marks and decomposed Unicode strings (NFD) to see if identical visual text would split into mismatched SHA-256 digests. * It generated synthetic Merkle proofs and tested whether our proof verifier would accept a recomputed root without checking it against an expected root. * It inspected the Model Context Protocol (MCP) server to see if hash-only outputs wore visual badges that made them look like verified cryptographic claims. * It tested our Obsidian plugin with an empty attestation bundle and an unverified email address to see if the interface would still flash a green checkmark. When the dust settled on the first pass, the review register recorded fifteen distinct findings. --- ### What Broke (and What We Fixed) A lesser engineering culture treats audit findings as insults to be negotiated away. In our studio, we treat a reproducible finding as an unearned debt that someone was kind enough to point out before production. We did not contest the valid flaws. We triaged the list, separated genuine security holes from operational boundaries, and went straight to the metal. Here is what we patched across two rapid commits: 1. **Expected-Root Merkle Verification**: Finding 14 caught a major omission. While our Merkle engine calculated tree paths correctly, the proof verifier did not require callers to supply an expected trusted root. An attacker could forge an arbitrary leaf and a matching tree, and the verifier would happily report that the math checked out. We bound the expected root directly into the verification path, rejecting any proof whose calculated root deviates from the caller-trusted anchor. 2. **True Canonicalization Across Runtimes**: Finding 15 revealed that while we stripped Windows CRLF line endings, we left UTF-8 Byte Order Marks (`\uFEFF`) and decomposed Unicode unhandled. A Mac typing an accented character in NFD would generate a completely different hash than a Linux box typing the same character in NFC. We added BOM stripping and native Unicode NFC normalization to our core canonicalization routine. 3. **Closing the UI Green-Check Gap**: In the Obsidian plugin, an attestation containing the string `moongladeai@gmail.com` was previously accepted as a human architect signature even when no authorized signer registry existed. We eliminated string matching entirely. The engine now requires an explicit signer registry, verifies the cryptographic fingerprint, and confirms the specific architect role. Empty attestation bundles are now hard rejected. 4. **Failing Closed on Unverified Hashes**: In the MCP gate, generating a bare document hash without a signature previously produced an output that an LLM could easily confuse with an attestation. We updated the gate to fail closed when no key is provided, explicitly marking bare hash calculations as unverified. We pushed the fixes, added regression tests, and watched our core test suite expand to 128 passing tests across five suites, alongside seven new integration tests in the evaluation harness. --- ### Clean-Room Verification: Receipts 004 and 005 Fixing code on your local machine is only half the battle. If an evaluator cannot reproduce your results from a blank slate, the fix does not exist. We set up a formal, read-only dispatch channel for the evaluator harness. It pulled commit `13a49f7` into a completely fresh, isolated directory and ran a clean-room verification. Its automated report came back in **Receipt 004**: * `npm test`: 128 passed out of 128. * Integration harness: 7 passed out of 7. * Deterministic re-evaluation script: completed with exit code 0. * Unicode NFC equivalence, BOM handling, and the Obsidian plugin role checks were all independently confirmed. Shortly after, the harness executed an automated six-cell diagnostic pilot (**Receipt 005**) using `gpt-5.6-terra` across two synthetic contradiction tasks. Every single agent and tool event hash chain validated mechanically across the observed test runs. --- ### The Real Victory: Epistemic Honesty The easiest trap in AI right now is making claims that run three laps ahead of your evidence. People love to claim that their agent guardrail improves model accuracy by 40%, cuts token usage in half, or eliminates hallucination forever. We refused to make any of those claims in our whitepaper, in our repository, or in our showcase submission. Why? Because measuring whether cryptographic provenance changes model behavior is an open empirical question. Our diagnostic pilot proved that the cryptography works, the gates fail closed, and the hash chains never lie. But on simple synthetic contradiction tasks, the baseline model with plain document tools got the right answer just as often as the model equipped with provenance tools. Instead of hiding that data, we published it. We copied the evaluator's raw pilot report directly into the public audit folder in our submission package. Our position is simple: a package that argues against unchecked claims has no right to make one. We provide the mathematical primitive and the protocol. We leave the behavioral benchmarking to rigorous, independent science. --- ### What Comes Next We wrapped the sprint by packaging our submission for Google Drive: * The full canonical markdown whitepaper and business snapshot, sealed with Ed25519 signatures and live RFC 3161 timestamps. * The unedited audit trail from the evaluation process. * A standalone, zero-dependency Python verifier (`verify.py`) tested across Windows, macOS, and Linux. * A zero-install browser verifier (`VERIFY.html`) that lets anyone drag and drop a document and its provenance sidecar to verify signatures in WebCrypto memory. * Our formal submission text for the IETF Internet-Draft on agentic epistemic provenance. Taking a system through this kind of pressure on a tight clock is exhausting, but it leaves you standing on bedrock. We know what our stack does, we know its exact boundaries, and we have the receipts to prove it. Catch some sun, enjoy your Sunday, and if you want to inspect the reference implementation, check out our repository on GitHub. Until next week, walk good! 🌊