Evaluation

Document Privacy Scanner for AI Uploads: What a Useful Scan Should Show

How to assess scanner coverage, evidence, false negatives, format support, remediation, rechecking, and the meaning of a no-findings result.

Published August 19, 20267 min readReviewed against official sources

A scanner is decision support

A document privacy scanner can find supported patterns and file structures: an email address in text, a known API-key format, an Office author property, a hidden worksheet, or a PDF attachment. The result tells a reviewer where to look and what the tool can attempt to remove.

It cannot prove the absence of confidential content. The same phrase can be sensitive in one organisation and harmless in another, while a combination of ordinary details can identify a customer without matching any individual pattern.

Report language should preserve that distinction. No supported findings detected is accurate. Safe to upload overstates what a scan alone establishes.

Coverage should be format-specific

StatusMeaningExample
Detected and removableThe tool can locate the item and has a tested transformationSupported author property in a DOCX copy
Detected, manual actionThe tool can report the item but cannot safely transform itA workbook dependency that needs owner review
Best-effort detectionCoverage varies with content quality or structureOCR text in a low-quality scan
UnsupportedThe item or format is outside tested scopeA proprietary embedded object
Manual reviewAutomation cannot determine business sensitivityA unique project fact or contractual restriction

A single label for PDF, DOCX, XLSX, PPTX, image, text, and configuration files hides important differences. Each format has distinct structures, parsing failure modes, and removal methods.

Ask how the scanner was tested

Good evidence includes synthetic files containing one known risk at a time, mixed-risk files, corrupted inputs, unusual encodings, and before-and-after verification. Tests should cover both positive detection and false-positive handling.

A zero-network test is relevant for a local-processing claim. A reopen test is relevant for file integrity. A second scan of the output is relevant for removal. None replaces a human review, but each supports a specific claim.

GitHub's secret-scanning documentation illustrates why scope matters: detection depends on supported patterns, object type, size, and other limits. Document scanners face analogous boundaries across text, OCR, embedded objects, and metadata.

Look for explainable findings

A finding should identify the category, location, evidence safely enough for review, confidence or rule basis, available action, and any effect of that action. The report should avoid writing a live secret into logs or telemetry.

Reviewers also need a way to distinguish a true finding from a false positive and to add organisation-specific terms or patterns. A generic scanner cannot know every internal customer code, project nickname, or proprietary identifier.

Removal should be reversible only through the preserved original, not through hidden content in the output. The output needs a clear path, a non-overwrite policy, and failure behaviour that does not silently publish a partial result.

A scanner acceptance checklist

  • Supported formats and maximum sizes are explicit.
  • Detection, removal, unsupported, and manual-review items are separated.
  • Known blind spots are published in plain language.
  • The original is preserved and failures are fail-closed.
  • Local-processing claims have testable boundaries.
  • Outputs are reopened and scanned again.
  • Reports do not expose the sensitive values they describe.
  • No-findings language is qualified.
  • Users can complete a final human review.

Official sources

This guide uses primary sources available on August 19, 2026. Product policies and software features can change, so confirm current terms before handling sensitive material.