Agent-Skill Field Test

Does the Tufte data-viz skill survive contact with a real organization?

A multi-agent evaluation of caylent/tufte-data-viz across companies, teams, skill installs, and levels of context.

Skill under test: 22 Edward Tufte visualization rules + per-library rule files (Recharts, ECharts, Chart.js, matplotlib, Plotly, D3/SVG) · Method: 7 parallel sub-agents, each a different organizational lens; one literally executed the skill end-to-end · Date: June 2026

On this page Verdict How it was tested What works Converging findings Context scorecard Recommendations

The VerdictHigh craft, solo-author worldview

The skill is genuinely well-made for the situation it imagines: one developer making a fresh chart from clean, labeled data. A hands-on agent followed its instructions literally and the matplotlib code ran on the first try, producing a chart that looks the part.

But in an organization it recognizes exactly one authority — the present human's explicit words — and one altitude — the single chart in front of it.

It has no concept of a brand guide, an existing codebase convention, a downstream designer, an absent human in a pipeline, or another skill. So it overrides design systems by default, generates pull-request style churn, ships dead-end raster images, and asserts statistical noise as fact when run unattended.

Every problem below is fixable with text-only changes. The single highest-leverage edit is about fifteen lines: a precedence ladder plus a split between structural rules (which survive any brand) and thematic ones (which should yield to it).

How it was testedSeven agents, seven lenses

Rather than ask "is this good design advice" (it mostly is), the test asked "what happens when this skill is installed somewhere real." Seven sub-agents ran in parallel, each reading the full SKILL.md, README, and several rule files, then simulating concrete interactions.

AGENT 1

Regulated enterprise

Bank / insurer / public-company finance. Board decks, audit workpapers, an exec who demands a pie chart.

AGENT 2

Startups & product

Pitch decks, a React SaaS dashboard with a design system, throwaway notebook EDA, a dual-axis board request.

AGENT 3

Agency · science · gov

Multi-client brand books, data journalism, journal figures, Section 508 accessibility.

AGENT 4

Team workflows

Analyst→designer→exec handoffs, code-review churn, org-wide governance, headless report pipelines.

AGENT 5

Skill composition

Co-firing with pptx, xlsx, frontend-design, brand-review; CLAUDE.md precedence; token budget.

AGENT 6

Context stress test

From a bare unlabeled CSV up to conflicting instructions and wrong-domain data (heatmaps, maps, Gantts).

AGENT 7

Hands-on execution

Actually wrote and ran the matplotlib chart on the machine; logged every ambiguity in the instructions.

Findings that several independent agents reached on their own are flagged below — convergence is the signal.

Credit where dueWhat works well

The structural rules earn their keep. No chartjunk, direct labels, no pie/dual-axis/3D, assertive titles — these were welcomed in every context tested, even by simulated hostile design reviewers. The fights are almost all about the aesthetic rules (gray+red palette, #fffff8 background, serif fonts).
The library rule files are the standout asset. The matplotlib TUFTE_RC config and helpers were copy-paste runnable with zero debugging.
Progressive disclosure is real: a lean-ish entry file, "read one rule file per task," and a fallback config table — about 5.5k tokens per invocation.
The accessibility tier (contrast, dual-encoding, reduced-motion) is better than most human teams ship — despite a self-contradiction noted below.
Best-fit homes: data journalism, content marketing, and internal analyst docs, where ~85% of the rules match professional practice out of the box.

The FindingsWhere it breaks, and who found it

C1 · root cause HIGH 7 / 7 agents

The escape hatch doesn't work in organizations

The skill's only override mechanism is one sentence: "deviate only when the user explicitly requests otherwise." It is per-rule, per-turn, and human-only.

"Use our corporate template" is one instruction covering ~6 rules — but un-named defaults silently reassert on the next turn ("one more chart, same as before" → the cream #fffff8 box returns).
A CLAUDE.md directive ("all charts use our brand palette") is not "the user explicitly requesting," so the skill licenses Claude to override recorded org config.
The validation checklist then pressures Claude to "fix" legitimate deviations back to Tufte before delivery.

C2 · triggering MED–HIGH 5 / 7 agents

It fires too widely — and restyles things nobody asked it to

The description triggers on "creating, reviewing, or styling… any data visualization."

Bug fixes become restyling incidents: "fix the tooltip crash" triggers the anti-pattern detector, which instructs Claude to strip legends and swap fonts — noise in the diff.
Style ping-pong: Engineer A ships Tufte → Engineer B reverts to the design system → next sprint the skill re-"fixes" B's fixes. Two agents on one PR give opposite verdicts.
Native-chart contamination: "add a chart to this spreadsheet" loads ~3.8k tokens of rules unimplementable in Excel — and the dangerous resolution is silently replacing a live, data-linked chart with a static image.

C3 · contradiction HIGH 4 / 7 agents

The skill contradicts itself

Rule 4 says axes should span only the data range. The skill's own anti-patterns file says truncated y-axes are deceptive — include zero. The hands-on agent hit it live: 75% growth rendered as a near-vertical climb the skill's own docs call misleading.

Also found: the small-multiples "shared scales always" rule is mathematically impossible for its own dual-axis-replacement case (MRR $ vs. churn %); rule 8 forbids >4 colors while permitting a sequential ramp in the same paragraph; the checklist has 20 items while the README claims 22.

C4 · data integrity HIGHEST 4 / 7 agents

It manufactures findings that aren't there

Rule 20 demands titles that assert a finding. Rule 13 demands comparison context. A binary checklist enforces both. With thin data, that becomes fabrication:

Unlabeled CSV columns → a title like "Values Climbed 40%" about quantities Claude can't identify.
A weekly automated report → "Signups Jumped 11%" on n≈200 noise, emailed with the authority of an automated system.
A board market-share slide → "Acme Leads the Market" — exactly the claim antitrust counsel scrubs.

In audit, SEC, or FDA contexts this is legal exposure, not a style nit.

C5 · aesthetics collide MED 6 / 7 agents

Hardcoded tokens clash with every host environment

#fffff8 renders as a visible cream rectangle on white slides, web pages, and journal pages — and it's baked into three config keys, so partial fixes don't stick.
Red #e41a1c as the only accent is semantically wrong for finance: a red MRR line reads as decline. No valence/semantic-color guidance anywhere.
The serif mandate fails design review at any shop with a type system, violates journal specs (Arial/Helvetica), and ET Book is a web-CDN font that breaks in Figma and print.

C6 · verified bug MED 2 agents · checked the math

The accessibility palette fails its own contrast rule

The skill's specced colors fall below its stated 4.5:1 minimum — #999 ticks ≈ 2.85:1, #aaa source lines ≈ 2.3:1 — while a rule file claims "the Tufte palette meets both minimums." Every chart would fail the Section 508 audits the skill advertises supporting.

C7 · lifecycle MED agents 1, 4, 5

It has no awareness of the work around the chart

Handoffs: output is a flattened PNG — title and labels baked into pixels, no SVG, no source-data sidecar, no provenance. Designers rebuild by eye; regeneration destroys their edits. (The matplotlib examples literally end in plt.show(), never savefig.)
Owned repos: the skill never looks for an existing <Chart> wrapper or theme — despite holding Glob/Grep — and hand-rolls components that fail lint/CI.
Org adoption: changing five brand values means edits in ~9 files; no version field, no changelog, no clean update path for forks.

C8 · scope gaps MED agents 2, 3, 6, 7

Missing boundaries: purpose, chart family, and medium

No exploratory vs. presentation mode — the full 22-rule treatment lands on throwaway notebook EDA, where direct-labeling 8 cohorts on every re-filter is the opposite of helpful.
No chart-family scope — applied literally, the rules delete a heatmap's colorbar, sort a confusion matrix into meaningless bars, and strip a Gantt's date gridlines; the range-frame helper crashes on categorical axes.
No statistical graphics at all — no error bars, CIs, or box/forest plots, the core vocabulary of the scientific audience its seaborn support implies.
The checklist isn't gated by medium — static PNGs get "validated" against hover, reduced-motion, and 320px responsiveness.

Context ScorecardWhere it fits, where it fights

Deployment context	Fit as-is	Dominant failure
Data journalism / content marketing	Strong	Missing source-line mandate, social crops, alt-text structure
Internal analyst docs, greenfield charts	Strong	Range-frame vs. honesty contradiction; invented baselines
Government / Section 508	Moderate	Best-in-class a11y tier, but self-violating contrast values
Startup pitch / investor decks	Moderate	Cream box on white slides; red growth lines; serif vs. deck font
SaaS product dashboards (design system)	Poor	Overrides the design system by default; PR fails design review
Notebook EDA	Poor	No exploratory mode; 22 rules on throwaway charts
Regulated enterprise / audit / finance	Poor	Assertive titles = legal exposure; no org-style precedence; red = loss
Scientific / journal / regulatory	Poor	No statistical graphics; fights journal specs on five axes
Multi-skill installs (pptx / xlsx / brand-review)	Poor	No precedence model; co-firing conflicts; native-chart contamination
Headless automation	Poor	Asserts noise as findings; non-deterministic report structure

RecommendationsAll text-only, prioritized

P1 — Correctness & integrity

Replace the deviation sentence with a precedence ladder: explicit user request > project/org config (CLAUDE.md, brand guides) > the repo's existing conventions > these rules. A brand guide counts as deviation for every rule it touches, and an established deviation persists for the whole session.
Resolve the rule 4 / truncated-axis contradiction: range-frame for line/scatter; zero baseline always for bar/area; disclose a non-zero line baseline in the subtitle.
Add anti-fabrication guards: titles assert only what the data supports (else a neutral title, and ask when columns are unlabeled); reference lines must be derivable from supplied data, never invented; annotations stay neutral, no invented causes.
Add an unattended / pipeline mode: neutral factual titles unless a significance bar is cleared; produce every chart the template expects every run; pin axis domains for run-to-run comparability.
Fix the contrast self-violation (e.g. #999→#767676) and delete the false "palette meets both minimums" claim.

P2 — Adoption & composition

Rewrite the description with negative triggers: drop "reviewing" and "any data visualization"; add "Do NOT use for native Excel/PowerPoint chart objects, bug fixes unless restyling is requested, or chart-library internals."
Add a "Step 0: survey existing conventions" that finally uses Glob/Grep — find theme files, design tokens, existing chart components — and in owned repos apply Tufte through them rather than hand-rolling.
Tokenize the constants into one referenced table (kills the 9-file fork problem), support a project override file, and add a version field + changelog.
Gate rules and checklist items by output medium, and add a print/publication mode (white bg, venue font, ≥300 dpi vector, finding moves to the caption).
Define a comply-don't-lecture script for hard rules: build exactly what was asked (never silently merge a user's named pie slices to hit a 4-slice cap); offer the alternative once, in a sentence.

P3 — Coverage & polish

Add an exploratory-vs-presentation context gate; add a statistical-graphics.md (error bars, CIs, box/forest plots); scope which rules apply to heatmaps, maps, and Gantts.
Fix allowed-tools (a code-generation skill needs Write/Edit/Bash); always savefig, never plt.show(); ship dark-mode constants; use valence-aware accent colors.