Changelog¶

All notable changes to wardcat will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[Unreleased]¶

[1.2.1] — 2026-09-23¶

Added¶

Strict mode. with_strict() (YAML strict: true) turns a degraded scan into a DegradedScanError instead of a result with warnings — at build time for a model that did not load, at scan time for a backend that went down, and from scan_batch, which otherwise files the error under scan_error. The exception carries the warnings and the partial result. For pipelines where a document stored with names in it is worse than no document.
classify(): the sensitivity judgement with its categories. Wardcat.classify(text) returns a SensitivityVerdict — sensitive, the categories found (pii, credentials, financial, health, special_category, business_confidential) and the model's one-line reason — so a policy can route on the kind. An answer that cannot be read is sensitive with the category unknown. It is its own prompt, not the source of is_sensitive(): on the 100-text sensitivity benchmark with qwen3:14b it makes 1 false alarm against is_sensitive()'s 11 and names the right category every time, but misses 8 sensitive texts against 1 (mostly confidential business plans) and takes 17 s a text against 2 s. Use it to route what the gate stops, not as the gate. Its prompt names placeholders, format examples, public service numbers and reference numbers as not sensitive; the is_sensitive() prompt is unchanged, since a change to the gate ships only with its own measurement.
scan_batch runs batched layers in one pass. SpaCy NER sees the whole list through nlp.pipe; the LLM layer runs per text in the worker pool, bounded at the backend by with_llm(max_concurrency=4) — across scan_batch threads and async chunk fan-out alike, where the async batch previously fanned out one request per item. Results are unchanged, per-item errors still land in scan_error, and strict mode still raises. A detector that batches implements BaseDetector.detect_many. Measured on 200 samples of the Presidio corpus with regex + en_core_web_lg: 1.43× the throughput of a loop over scan(), identical output.
An opt-in HIGH_ENTROPY_STRING detector. A long base64-shaped run with the entropy of a generated key, or a hex digest longer than a git SHA, with no prefix and no keyword to go on. Off unless enabled and scored 0.70, under the default floor, so it acts only with its own min_confidence. Measured with it on: no hit on the 100 hard cases, none on the 1,500-sample Presidio corpus, 117 on the 2,891 Gretel finance documents — 18 of them API keys Gretel labels as such, most of the rest unlabelled wallet addresses and UUIDs, a few EDI segments. Base64 with / is not matched, so URL paths are not candidates.
A surrogate action. A realistic stand-in of the same shape instead of a placeholder: a name from the guard's locale (with_locale("tr"), YAML locale), an e-mail on a reserved domain, a phone number keeping its country and area code, a Luhn-valid card, a mod-97-valid IBAN, a checksum-valid TC number, a TEST-NET IP. Deterministic per salt, unique within a scan, and reversible through restore(). A type without a generator falls back to tokenize and the violation says so. Surrogates look real; the security guide says what that costs.
Presets. with_preset("kvkk") (also "gdpr", "pci_dss", "hipaa_lite", "secrets_only"; YAML preset:) enables a starting policy modelled on a data-protection regime — an entity → action mapping that switches no layer on and claims no compliance. Each one documents what it leaves out. Wardcat.supported_presets() lists them.
A wardcat command. wardcat scan [file|-] with --preset, --entity TYPE[=ACTION], --config, --ner, --llm, --strict and --json; wardcat check-config policy.yaml runs every validation including the ReDoS screen; wardcat entities lists the types. Exit codes say clean, found, config error or degraded. The salt is read from an environment variable named by --salt-env, never from an argument, and nothing found is ever printed.
Every violation names the layer that found it. Violation.source (and DetectedSpan.source) is "regex", "ner", "llm", "denylist", "propagation" or "custom", and is carried through redacted(), reapply() and the restore report. A Layer enum is accepted wherever a layer name was (add_entity(..., layers=[Layer.LLM]), supported_entities(Layer.NER)). The benchmark now breaks TP/FP down by layer.
Where a replacement sits in the output. Violation.sanitized_start and sanitized_end locate the replacement in sanitized_text, for highlighting in a UI; start/end keep pointing at the original.
A confidence floor per entity. add_entity(..., min_confidence=0.6), the add_entities spec form, and entities.X.min_confidence in YAML let one weak-checksum type through while the rest keep the global floor. entity_policy(detailed=True) reports it.
A circuit breaker on the LLM backend. After circuit_failures consecutive failures (default 3) the layer is skipped without a call for circuit_cooldown seconds (default 30), then tried once. A backend outage no longer costs every scan the full request timeout. Skipped scans carry a warning naming the open circuit; is_sensitive() raises CircuitOpen. circuit_failures=0 disables it.
A labelled phone number is found in any national format, with no phone_regions. The built-in pattern refuses a bare 905-674-3793 or 0490 75 40 81, rightly: a digit run is an order number as often as a phone number. The label settles it — Phone:, Mobile:, Fax:, call me at …, telefon: …, Handy:, or 781 1704 office in a signature block — so a number next to one is now reported, at 0.90. The value must hold 7–15 digits and must not be a date; phrases that merely tend to precede a number ("messages to", "not answering at") are deliberately not cues.
Troy cards. Turkey's domestic card scheme (9792 …) was not in the card pattern, so a Luhn-valid Troy number went undetected. It is matched now, and still has to pass Luhn.
A TC number written in groups, 111 654 670 34. Only the 3-3-3-2 grouping forms and printouts use, and the checksum still decides.
An SSN without dashes, when it is labelled. social security number 412 76 9038 and SSN: 412769038 are found at 0.90. An unlabelled nine-digit run is still refused.

Changed¶

README. Rewritten to a one-screen introduction — install, a first scan that runs offline, why wardcat, links — that defers to docs.wardcat.com. Nothing was dropped: the entity tables moved to a new entity types page, the limitations table to known limitations, the repository layout and test commands to CONTRIBUTING.md, and the remaining paragraphs into the guide pages they belong to. A test runs every README example and resolves every link, so the two cannot drift apart.
Literal denylist values are matched in one pass. Ten thousand names cost one scan of the text rather than ten thousand; the spans found are the same as before (a test keeps the old search as the oracle). A value listed under two entity types keeps the first and logs a warning.

Security¶

No log line carries a value from the scanned text. The NER filters logged the span they dropped, the regex layer logged a match that failed its checksum, and the LLM layer logged the model's raw reply — all at DEBUG, all containing the PII being scanned. Values are now logged as their entity type and length only. A test scans PII at DEBUG and asserts none of it reached the log, and a static check walks every logger call for a text-bearing argument.
The ReDoS check on your own regex patterns now works. custom_patterns were run against one pathological input in a thread with a 0.5 s timeout. Python's re holds the GIL for a whole match and cannot be interrupted, so the timeout could not fire until the match was over: on (a+)+$ the check hung configuration loading outright, and it passed (\d+)+$ and (\w+\s?)+$ because it only ever tried a run of a.

Patterns are now screened when they are configured. The parse tree is searched for a variable-width repeat nested in another repeat, or alternation inside a repeat that can consume the same text; each suspect is then run against inputs built from its shape in a child interpreter that the kernel kills after one second. Most patterns have no suspect shape and never start a process. A pattern shown to be catastrophic raises ConfigError — so a configuration that loaded before, or hung, can now refuse to load, and only for that reason.

Denylist patterns get the same screening, from YAML and from add_denylist(). YAML patterns were only compiled; add_denylist() checked nothing at all, so (a+)+$ was accepted and a 28-character input then took 9 seconds per scan. Invalid regex added through add_denylist() is refused now too, where it used to be skipped on every scan without a word. Patterns are compiled once rather than on every scan.
The scan-time timeout around custom patterns is removed. It bounded nothing, for the same reason, and discarded the matches of any pattern that ran past two seconds.

Fixed¶

NER spans are cut at the edges where models run over. Of a span that crosses a line break — into the next field in an address block (Anna Josefsen\nAddress), back over a heading (Renewals Team\nDalton Inc.) — the line with the most capitalised words is kept, a legal form winning for an organisation. A person's name that runs into markup or a record delimiter (`Carolyn Hill