pytestHTML Reporter
Home Docs Compatibility & files
Reference

Compatibility & files

What is actually tested rather than what setup.py declares, which plugins the reporter understands, exactly what a run leaves in your repository, and the full licence inventory of the ten libraries and one icon set that ship inside every generated report.

Versions and platforms

Nothing in the three repositories states a support policy, and the one machine-readable claim is now closer to the truth than it was. setup.py declares python_requires=">=3.7" as of 0.4.1, raised from >=3.5 because the async step support added in that release needs contextvars, which is 3.7. The package also uses f-strings — in cli.py, junit.py, coverage_report.py and report_opener.py — and nothing in CI has exercised 3.7 or 3.8 in years. What follows is the tested matrix rather than the declared one.

WhatTestedOught to workWhere that is tested
Python 3.9 and 3.13 3.8 and newer The Action's .github/workflows/ci.yml matrix
Operating system Ubuntu, macOS, Windows Anything Python runs on The Action's ci.yml and self-test.yml matrices
pytest Whatever pip install pytest resolves to Current majors, with the current plugin release The plugin's .github/workflows/main.yml

Python

The Action's own CI matrix runs 3.9 and 3.13 on ubuntu-latest, macos-latest and windows-latest — six cells, fail-fast: false, so a break on one interpreter does not hide a break on the other. The plugin's own workflow (main.yml) runs the unit suite on whatever Python ubuntu-latest provides, on pull requests to master only.

Treat 3.8 as the floor and 3.9–3.13 as the tested band — a step above the >=3.7 that setup.py declares, because 3.7 is what the code needs rather than what anything runs it on. If you need an older interpreter, pin an older plugin release and test it against your own suite — nobody else is.

Notetox.ini still lists py37 and pypy3, and an ansi2html factor that no code in the package reads. It is stale; the GitHub workflows are the live description of what gets run.

pytest

install_requires is an unpinned pytest, and the plugin tracks pytest closely rather than pinning it. Release 0.3.0 exists partly because pytest 9 removed TerminalReporter._sessionstarttime, which older releases reached into — the changelog entry reads "Fixed compatibility with pytest 9.x (AttributeError: '_sessionstarttime' not found)", and the reporter now keeps its own _sessionstarttime instead.

That is the shape of the risk. A pytest major release can break an old plugin release, and the fix ships as a new plugin release rather than as a version pin. Upgrade the plugin when you upgrade pytest.

Operating systems

setup.py classifies the package Operating System :: OS Independent, and the Action's self-test.yml backs that up by running the whole composite on ubuntu-latest, macos-latest and windows-latest and asserting on every output.

One behaviour is genuinely platform-sensitive: opening the report. macOS and Windows are assumed to have a desktop. Everywhere else, DISPLAY or WAYLAND_DISPLAY has to be set, or --report-open=auto stays quiet — see CI integrations for the full check.

The three projects

ProjectVersionRequires
Plugin 0.4.3 Python 3.8+, pytest, Pillow
GitHub Action v1, moved on each release Any runner with Python; the helper is stdlib-only
VS Code extension 0.1.2 VS Code ^1.90.0

The three version numbers are deliberately independent; there is no relationship between them. The extension reads output.json and nothing else, so it works with any plugin release that writes one — but its flaky-test verdicts need archive/ populated, which needs at least two builds.

Dependencies

Two hard dependencies, and that is the whole of it. Everything else is detected at run time and degrades to "that tab is empty" rather than failing the run.

Required

PackageNeeded for
pytest The plugin is a pytest plugin. Unpinned, so pip resolves whatever your project already has.
Pillow Writing and handling the screenshot images the reporter captures.

Optional — each unlocks one thing

None of these are installed for you, and none of them are imported unless they are present.

PackageNeeded forWhat happens without it
pytest-cov The Test Coverage tab, read from the Coverage object pytest-cov ran in the same session. Tab is empty, unless you point --report-coverage-file at a report of your own.
coverage Reading a bare .coverage SQLite data file. Only auto-discovered names are coverage.json and coverage.xml; a .coverage has to be named explicitly and needs this package. summarize_data_file returns None and the tab stays blank.
pytest-xdist Running -n auto. Workers are detected by config.workerinput and ship their records back to the controller, which writes one report. Nothing — the run is simply not parallel.
pytest-bdd Gherkin scenarios arriving in the Test Steps tab, with feature, scenario, keyword and parsed arguments. Nothing. The hooks are declared optionalhook, so pytest does not refuse to start over a hook nobody registered.
pytest-rerunfailures Folding a retried test's attempts into one row, and counting a retry as flake evidence. Detected with pluginmanager.hasplugin("rerunfailures"). Nothing — there are no reruns to fold.
A browser driver Automatic screenshots on failure. Selenium, Playwright, splinter, seleniumbase and pytest-selenium are all recognised. The Screenshots tab is empty.
TipThe plugin's requirements.txt is the development set — pytest-cov, coveralls, twine, pytest-xdist, beautifulsoup4, pytest-rerunfailures, Pillow and pytest-bdd. Do not read it as the runtime set; that is install_requires.

Plugins and drivers it understands

Recognised explicitly

Four plugins are handled by name in the source, and each one is handled defensively — the reporter asks pytest's plugin manager whether they are there rather than importing them.

Browser drivers, recognised by shape

No driver is imported and no driver is named as a dependency. The reporter looks through the test's own fixtures for a value that can be photographed, trying get_screenshot_as_png first and then screenshot, and looking one level down at a .driver attribute for wrappers. The fixture names it searches are the ones real suites use:

Fixture nameUsually from
page, contextpytest-playwright
driver, webdriverSelenium, by convention
browsersplinter, Playwright
session_browserpytest-splinter
seleniumpytest-selenium
sbseleniumbase

The same duck typing runs on HTTP clients: attach_api reads a response by shape, so requests and httpx both work without either being imported.

CarefulA Mock is ruled out by what it is, not by what it can do — it answers every call ever made to it, screenshot calls included, so it would be photographed on the strength of a method it does not have, and the call would show up in an assertion your test is about to make.

Not supported

ThingWhy not
behave It does not run under pytest, so there is no hook to implement and nothing that could be shown. Only pytest-bdd is covered.
-s / --capture=no Not a plugin, but the most common cause of an empty Logs column. pytest has to be capturing for there to be anything to keep. Use --capture=tee-sys to get output on the terminal and in the report.
Two report writers on one folder A report folder has exactly one writer per build. That is also why the merge of a sharded run is a console script rather than another pytest run — a fifth pytest started in the folder would clear away the screenshots it was sent to collect.

What a run writes to disk

This is the first thing a newcomer hits, and the README currently gets it wrong. The Usage section says "the filename used is pytest_html_reporter.html and path chosen is report". Neither half is true. With no --html-report, the plugin writes pytest_html_report.html — one er shorter — into the current working directory, which is usually your repository root:

python pytest_html_reporter/html_reporter.py
return os.path.abspath(logfile), 'pytest_html_report.html'

Run pytest once in a fresh directory and you get three things beside your tests: the report, an output.json, and — from the second run onwards — an archive/ folder. Nothing is cleaned up and nothing warns you. So the first thing to do in a real project is give the report a folder of its own and tell git to ignore it.

Screenshot: assets/img/shots/disk-after-a-bare-run.png A terminal, 1200px wide, dark theme. Two commands visible: a bare pytest in a clean project, then ls -la showing pytest_html_report.html, output.json and archive/ sitting in the repository root beside the tests. The file size on the HTML should be legible — it is the 1.6 MB figure made concrete.

Everything a run can create

All paths are relative to the report base — the folder side of --html-report. With no flag, the report base is the working directory.

output <report base>/
report/
├── pytest_html_report.html          the report itself
├── output.json                      this build's record
├── archive/
│   ├── output_1756900000.123.json   a previous build, rotated in
│   └── output_1756903600.456.json
├── pytest_screenshots/
│   └── <test name>.png              images the report links to
└── shards/                          only in a sharded run
    └── <shard id>/
        ├── records.json             this leg's records, written atomically
        └── pytest_screenshots/      this leg's images, left where they are
PathWritten whenWhat it is
pytest_html_report.html Every non-shard run The whole report: markup, styles, data and every vendored library, in one file. Roughly 1.6 MB for a one-test run — that is the floor, not the content.
output.json Every non-shard run This build's record. The stable, documented build format the VS Code extension and the Action's helper both read.
archive/ From the second run onwards Previous builds. A build joins the archive when the next run rotates it there, renaming output.json to output_<start time>.json. Each retained build costs roughly 5 KB of the report page.
pytest_screenshots/ When a screenshot is captured PNGs. The report stores a bare name and both screenshot templates rebuild the path, so the folder has to travel with the HTML.
shards/<id>/ --report-shard only One directory per leg, holding records.json and that leg's own images. A shard writes no report and no output.json, and rotates no archive.
Your JUnit XML path --report-junit only Written wherever you point it; parent directories are created. An unwritable path is a warning on stderr, never a failed build.

This project's own report, with 296 archived builds, is 6.6 MB. If yours is heading that way, the retention flags in the CLI reference are what you want — --archive-days, --archive-count and --archive-since.

Notecoverage.py's htmlcov/ is not written by this plugin. When it exists and was written after this run started, the report links to it relatively rather than copying it in — so the two travel together or the link is simply not offered.

Two path rules that surprise people

.html anywhere makes it a filename. The check is if '.html' in self.path, not endswith. So --html-report=./out.html_v2/report.html is read as a bare filename and the report lands in the working directory, because the code then asks whether the directory part also contains .html and falls back to . when it does. Rename the folder.

~ and $VAR are expanded, by os.path.expanduser and os.path.expandvars — so --html-report='$CI_PROJECT_DIR/report' and --html-report=~/reports both work, on top of the strftime expansion documented in Configuration.

The .gitignore block to paste

Both companion repositories ship a version of this. The Action's is the one to copy, because of the leading slashes: they anchor each entry to the repository root, so a tracked fixture called output.json deeper in your tree survives.

shell .gitignore — with --html-report=./report
# Everything a pytest-html-reporter run leaves behind lives in one folder.
/report/

What ships inside the report

Every generated report is a single self-contained file, and that is only possible because ten JavaScript and CSS libraries plus a Font Awesome icon set are vendored into the package and inlined at generation time. If you are reviewing this plugin for a licence audit, this section is the inventory.

Diagram: assets/img/shots/report-inlined-layers.png A layered diagram of one generated HTML file, stacked bottom to top: the ten vendored libraries as one band, the Font Awesome CSS-mask icons as a second, then the report's own markup, styles and build data. To the side, a CDN cloud with an arrow to the file crossed out and labelled "no network, at generation or at read time". Light and dark variants if practical; 1200px wide.

Why they are vendored at all

The report used to pull these off a CDN. A run on a machine with no route to the internet produced a blank white page: jQuery never arrived and nothing after it ran. They now ship in the wheel, verbatim as published, and html_page/assets.py writes them into the report. That is what makes the offline guarantee real rather than aspirational — you can mail the file, attach it to a ticket, put it on a stick, or open it from a locked-down build agent, and every chart, table, filter and export still works.

The inventory

Every one of them is MIT. JSZip is dual MIT/GPLv3 and is used under MIT.

FileVersionLicence
jquery.min.jsjQuery 3.5.1MIT
jquery.dataTables.min.js / .cssDataTables 1.10.19MIT
dataTables.buttons.min.js, buttons.dataTables.min.cssButtons 1.5.2MIT
buttons.html5.min.js, buttons.print.min.jsButtons 1.5.2MIT
buttons.colVis.min.jsButtons 1.6.1MIT
bootstrap.min.js / .cssBootstrap 4.1.3MIT
chart.min.jsChart.js 2.8.0MIT
jspdf.min.jsjsPDF 1.3.2MIT
dom-to-image.min.jsdom-to-image 2.6.0MIT
jszip.min.jsJSZip 3.1.3MIT or GPLv3 (used under MIT)

Icons

Font Awesome 4.7.0 glyphs, kept as individual SVGs, sourced from Font-Awesome-SVG-PNG (black/svg). Licence: CC BY 4.0, per the Font Awesome 4.7.0 licence.

Each file keeps its Font Awesome name (fa-homehome.svg) and is cropped to the glyph's own outline, with the viewBox still recording that outline's place on the font's 1792-unit em — the em Font Awesome drew on, with the baseline 1536 units down. html_page/icon_styles.py reads the directory at generation time and inlines every glyph as a CSS mask, which is why <i class="fa fa-home"></i> still works, an icon still sizes itself to the text around it, and it still takes its colour from currentcolor.

The one library deliberately absent

Don'tfancyBox is not here, and must not be added. The screenshot gallery used fancyBox 3.5.7, which is GPLv3 or a paid commercial licence — either way not something an MIT-licensed package can redistribute. The report opens screenshots with a lightbox of its own instead, in template.html.

If you are vendoring a replacement for anything, this is the precedent: check the licence before the feature.

Upgrading one, or adding an icon

Replace the file, update the row in html_page/vendor/README.md, and leave the load order alone — it lives in assets.py, not in the README. To add an icon, drop its SVG into html_page/icons/ named after the fa-* class and the stylesheet picks it up automatically; crop it to the outline first, or it is drawn in the wrong place and at the wrong size.

CarefulMANIFEST.in controls the sdist and setup.py's package_data plus include_package_data control the wheel. A new asset type has to be added in both, or the sdist has it and the wheel does not — and a wheel missing its assets generates a blank white page.

Glossary

A dozen terms these docs use with precise meanings that are easy to guess wrong.

TermWhat it means here
Run One pytest invocation.
Build One row of history — one output.json. Retention, archives, trends and analytics all count builds, never runs.
Report base The folder side of --html-report. Every path above is relative to it. With no flag it is the current working directory.
Archive <base>/archive/. A build joins it when the next run renames output.json to output_<start time>.json — so a build is archived by its successor, not by itself.
Worker An xdist process on this machine, identified by config.workerinput. Workers hand their records to a controller, which writes one report. Nothing to configure.
Shard A separate machine or leg, writing <base>/shards/<id>/records.json and no report of its own. Something has to merge the bundles afterwards.
Run token What tells a merge which shards belong together. Given by --report-shard-run, or derived from the CI system's own variables — always carrying the name of the system it came from, so Jenkins build 41 and Drone build 41 cannot collide.
Suite The grouping the report's Suite Highlights use: nodeid.split("::")[0], which is the test file.
Step A named, timed piece of a test — @step("…") or with step("…"), nesting. Setup, body and teardown are timed as steps whether or not anyone declared any, and a pytest-bdd scenario arrives as steps for free.
Attachment Text, JSON, a file or a whole HTTP request/response a test hands to the report. Built-in types only, because an xdist worker has to ship it back to the controller.
Decided build A build in which the test did not skip. All the pass/fail maths — rate, flips, streak — is done over decided builds only, so a test skipped for three builds between two passes has not flipped twice.
Flaky A test that flipped between pass and fail across its decided builds, or needed a retry inside a single build. A retry is the least ambiguous evidence there is: same code, same build, two different answers.
Standing failure Shown as Always failing. Every decided build failed and the current failing streak is at least two builds — so one failing build is not yet one. A standing failure is explicitly not counted as flaky: it is broken, and listing it under flakiness sends people hunting for a race that is not there.

Build vs. run

Usually they are the same thing — but an xdist run with eight workers is eight processes and one build, and a sharded matrix is four runs and one build. Everything about retention, archives and trends counts builds.

Shard vs. worker

Both split work up, and they compose, but they solve different problems and are handled completely differently. Workers are processes on one machine that hand their results back to a controller, which writes one report — nothing to configure. Shards are separate machines, each writing a bundle and no report, and something has to merge them afterwards. You can shard across four machines and run each shard with -n auto; see CI integrations for the wiring.