Skip to content

Stabilize test_html5_rubric - #14792

Open
thisisshivamtiwari wants to merge 1 commit into
sphinx-doc:masterfrom
thisisshivamtiwari:fix-flaky-test-html5-rubric
Open

thisisshivamtiwari wants to merge 1 commit into
sphinx-doc:masterfrom
thisisshivamtiwari:fix-flaky-test-html5-rubric

Conversation

@thisisshivamtiwari

Copy link
Copy Markdown

Fixes #14791

Purpose

test_html5_rubric checks for two warnings. One of them, "7" unknown, is the docutils error for :heading-level: 7, and it is only emitted while index.rst is being read.

The latex, man and texinfo rubric tests (test_latex_rubric, test_rubric, test_texinfo_rubric) use the same markup-rubric test root. They therefore share its source and doctree directories. If one of them runs before test_html5_rubric in the same worker, the HTML build loads the pickled environment, doesn't re-read index.rst, and the directive warning never appears. Only the HTML writer's unsupported rubric heading level: 7 warning is left, which matches the failure in the linked CI run. So it depends on test order under xdist, not on the docutils development version.

The fix adds freshenv=True to the test's pytest.mark.sphinx marker, so this test always reads its sources. The other three rubric tests only check the output, so they are unaffected.

Testing

On master (docutils 0.23, Python 3.14), this fails every time:

pytest -p no:randomly tests/test_builders/test_build_latex.py::test_latex_rubric tests/test_builders/test_build_html_5_output.py::test_html5_rubric
# FAILED test_html5_rubric - assert '"7" unknown' in '...WARNING: unsupported rubric heading level: 7 [html]...'

With this change:

  • the command above passes;
  • all four rubric tests pass in either order (HTML first or last);
  • tests/test_builders/test_build_html_5_output.py passes (173 tests);
  • pytest -n auto tests/test_builders passes (769 passed, 13 skipped), run twice;
  • ruff check and ruff format --check on the changed file pass.

I didn't add a CHANGES entry because this only touches a test (see #14572 for the same kind of change). I'm happy to add one if you'd like.

The test asserts on a warning that docutils emits while reading
index.rst. The latex, man and texinfo rubric tests use the same
markup-rubric test root, so when one of them runs first in the same
worker, the HTML build reuses the pickled environment, never re-reads
the source, and the warning is missing. Use a fresh environment.

Fixes sphinx-doc#14791

@jdillard jdillard left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to me. Thanks!

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

2 participants