Skip to content

Chart Property Diagnostics

This repo has a reusable artifact-based harness for probing chart property control surfaces, plus a tiered regression strategy that turns audit findings into durable CI gates.

The goal is simple:

  • test one property at a time
  • render three independent cases
  • default
  • chart override
  • system override
  • record whether the property visibly changes at chart level, at system level, or both

This avoids the ambiguity we hit in the live playground, where per-chart theme and structure are not reliably honored by the current standard chart path.

CI Lane Policy

Chart property coverage is split into two deliberate CI lanes:

Fast lane (blocks PRs)

tests/core/test_chart_property_regression.py — spec-level regression assertions derived from the chart property audit. These tests verify that authored properties (title, axis, mark) reach the final Vega-Lite spec through both the chart-authored and style-authored control paths. They run in < 0.1s with no SVG rendering or external dependencies.

just test tests/core/test_chart_property_regression.py

What the fast lane covers:

Section What is asserted
Title text, subtitle, color, fontSize, fontWeight, anchor, angle via style
Axis labelAngle, gridColor, grid toggle, labelColor, labelFontSize, titleColor, domainColor, tickCount, gridWidth via style; config passthrough
Mark opacity, color, strokeWidth, cornerRadius, interpolate, size, filled via style; config passthrough
Config nested passthrough, merge with defaults, transform/params append, precedence over style

Slow lane (nightly, never blocks PRs)

tests/scripts/test_chart_property_diagnostics.py — full artifact-based regeneration of the title family. Generates face YAML, compiles, renders SVG, and compares against known outcomes. Marked @pytest.mark.slow.

tests/integration/test_examples.py::TestExploratoryCorpus — compile and render smoke for exploratory files like chart-design-lab.yml that are excluded from the default examples sweep. Also marked @pytest.mark.slow.

just test-slow

When to add to which lane

  • Fast lane: Add a test when a property should always reach the final spec and the assertion can be checked at the spec dict level (no rendering needed). This is the right place for most regression gates.
  • Slow lane: Add a test when you need to regenerate artifacts, render SVGs, or run against exploratory/scratch dashboards that are expensive or evolving.

Exploratory Corpus: chart-design-lab.yml

examples/playground/faces/charts/chart-design-lab.yml is an exploratory scratch dashboard with 20+ chart types. It is intentionally excluded from the default examples sweep (SKIP_EXAMPLE_FILES in test_examples.py) because it is large and evolving.

It has an explicit test story in the slow lane: TestExploratoryCorpus compiles and renders it nightly, so breakage is caught without slowing the default CI path.

Canonical Example

The canonical preserved example case is the title family.

Generator: - scripts/chart_property_diagnostics.py

It currently probes:

  • title.text
  • title.color
  • title.font
  • title.fontWeight
  • title.fontSize
  • title.anchor
  • title.offset
  • title.angle
  • title.baseline

This title family should be treated as the reference pattern when building future families such as axis labels, axis titles, grids, legends, or scales.

How To Run

Generate the current title-family artifacts:

uv run python scripts/chart_property_diagnostics.py

The main inspection outputs are:

  • index.html
  • index.md
  • one folder per property

Each property folder contains:

  • default.face.yml
  • chart-override.face.yml
  • system-override.face.yml
  • default.spec.json
  • chart-override.spec.json
  • system-override.spec.json
  • default.svg
  • chart-override.svg
  • system-override.svg
  • result.json

Method

For each property:

  1. start from one stable specimen chart
  2. generate a default case
  3. generate a chart override case
  4. generate a system override case
  5. inspect the rendered SVG and generated Vega-Lite spec
  6. record raw results in result.json

The harness intentionally reports raw control success:

  • chart override: changed or no_change
  • system override: changed or no_change

and summarizes each property as:

  • both_work
  • partial
  • neither_work

It does not encode prior expectations about which layer should own the property. The point is to discover what can actually be controlled today.

Validation

The title family is preserved as a repeatable validation case with a test:

  • tests/scripts/test_chart_property_diagnostics.py

Run it with:

uv run pytest tests/scripts/test_chart_property_diagnostics.py -q

That test regenerates the title family in a temporary directory and asserts the current known raw outcomes. This gives us a stable regression harness when we revisit the mapping between DFT and Vega-Lite.

Why This Exists

We discovered a recurring control-surface problem:

  • some properties work only as chart-authored values
  • some work only through theme or structure
  • some are emitted into the final Vega-Lite spec but still do not visibly take effect

This harness gives us a systematic way to answer:

  • can I control this property at chart level?
  • can I control it at system level?
  • does the final render actually change?

That makes it suitable both for design-system investigation and for future DFT/Vega-Lite contract debugging.