Contributing

The three repositories

The project is split across three repositories that ship independently and share no version number. They are deliberately separate: the plugin has no GitHub dependency, the Action has no Python dependency beyond the interpreter the workflow set up, and the extension reads a file the plugin writes and nothing else.

So two of the three trackers are the same tracker. The dividing line for the Action, from its own troubleshooting notes: if the problem is in getting the report built — inputs, permissions, caching, the pull request comment, the coverage gate — it belongs on the Action. If the problem is in the report's contents, it belongs on the plugin.

Setting up the plugin

The plugin repository has no CONTRIBUTING.md, so its development story is spread across tox.ini, .pre-commit-config.yaml, requirements.txt, .github/workflows/main.yml, codecov.yml and the pull request template. This section assembles it.

Clone, virtualenv, install

$ git clone https://github.com/prashanth-sams/pytest-html-reporter.git
$ cd pytest-html-reporter
$ python -m venv venv
$ source venv/bin/activate          # Windows: venv\Scripts\activate
$ pip install -r requirements.txt
$ pip install -e .

venv/ is already in .gitignore under that exact name, so use it and nothing leaks into a commit.

requirements.txt is the development set, not the runtime set. It holds pytest, pytest-cov, coveralls, twine, pytest-xdist, beautifulsoup4, pytest-rerunfailures, Pillow and pytest-bdd. The runtime dependencies declared in setup.py are only pytest and Pillow.

Selenium and Playwright are deliberately absent from that file. The browser demos in tests/functional/ are run by hand, and each one skips itself when its driver is missing — see the browser demos below.

Running the unit suite

tests/unit/ is what CI runs, and it is the suite to keep green: 33 modules covering the plugin lifecycle, analytics, archives, coverage, xdist, reruns, steps, screenshots, markers, logs, escaping, deep links, theming, icons, attachments, teardown failures, collection errors, offline rendering and the public API.

Run it with the same command the CI job uses, coverage and all:

$ python -m pytest --cov ./pytest_html_reporter/ tests/unit/

Name the path. The repository's pytest.ini sets addopts = -v -rf --capture=tee-sys and declares the slow and fast markers, but it does not restrict testpaths — so a bare pytest in the repository root collects the functional bank as well, and several of those tests fail on purpose. tox.ini carries a [pytest] section that does point testpaths at tests/unit/, which is what the tox environments get.

--capture=tee-sys is worth understanding rather than changing: it puts test output on the terminal and keeps it for the report. Turning capture off with -s leaves the Logs column empty, because there is then nothing to collect.

To run the suite the way tox does, across the declared environments:

$ pip install tox
$ tox                    # envlist: py37, pypy3, each with and without ansi2html, plus linting
$ tox -e linting         # pre-commit run --all-files --show-diff-on-failure

tox.ini's envlist still names py37 and pypy3, which is older than anything CI exercises. See Compatibility for the interpreters actually tested.

Linting and pre-commit

One linter: flake8, run through pre-commit. tox.ini holds its configuration — max-line-length = 120, excluding .eggs and .tox — and .pre-commit-config.yaml pins the hook and excludes docs.

$ pip install pre-commit
$ pre-commit install
$ pre-commit run --all-files --show-diff-on-failure

What CI checks on a pull request

.github/workflows/main.yml is named CI and triggers on pull_request against master only — there is no push build. It runs two jobs on ubuntu-latest:

• lintcclauss/Find-Python-syntax-errors-action, which catches a file that will not compile at all.
• unit-testInstalls requirements.txt and the package in editable mode, runs python3 -m pytest --cov ./pytest_html_reporter/ tests/unit/, then pushes coverage to Coveralls and Codecov.

The Codecov gate is deliberately not a gate. codecov.yml sets target: 0% with a threshold: 70%, precision: 3, patch: false, and a diff-only comment that only appears when there are changes — so coverage informs review rather than blocking it.

A legacy .travis.yml is still in the tree, running the same test command on Python 3.8. GitHub Actions is the live one.

The merge checklist

Straight from .github/pull_request_template.md, which is what appears in the PR body:

• Changelog
• Documentation
• Unit Tests
• Validated
• Version number incremented

CODEOWNERS puts /pytest_html_reporter/ and /tests/ under @prashanth-sams, so a change to either path requests his review automatically.

The functional test bank

tests/functional/ is not a CI suite. It is a demo gallery: small pytest files whose only purpose is to fill each tab of the report, several of which fail on purpose because a failure is what puts a screenshot, an error snippet and an attachment in the page. Its own Readme.md calls it "pytest bank — pytest exercises".

It is the fastest way to see a feature working before you wire it into your own suite.

Running it

$ pytest tests/functional/ --html-report=./report

Expect red. The failures are the content. Run it twice and Trends, Archives and Analytics have something to draw, because a build only joins the archive when the next run rotates it there.

The three ways the bank's own README suggests running a single file:

The slow and fast markers those commands select on are declared in the repository's pytest.ini.

Start with test_attachments.py

tests/functional/test_attachments.py needs no browser and no network — its HTTP client is a stub class shaped like a requests response — so it is the quickest thing in the repository to run:

$ pytest tests/functional/test_attachments.py --html-report=./report

It exercises every helper — attach_api from a response object and from arguments alone, attach_json, attach_text, attach_file, and the attach-on-failure fixture. Two of its tests fail on purpose. It also carries a fake bearer token, an ?api_key= and a password in a payload, none of which reach the report — so it doubles as a demonstration of redaction. See Security and privacy for what that covers.

Two patterns in that file are worth lifting into your own suite verbatim. The first is the fixture that attaches the last call, but only when the test failed:

@pytest.fixture
def api(request):
    client = Api()
    yield client

    if getattr(request.node, "rep_call", None) is not None and request.node.rep_call.failed:
        if client.last_response is not None:
            attach_api(client.last_response, name="Last call before the failure")

That works because the reporter builds a test's record after the fixture finalizers have run, which makes a teardown a perfectly good place to attach from. The rep_call attribute comes from the standard hook wrapper, which the demo defines in the test module itself so the file stands alone:

@pytest.hookimpl(tryfirst=True, hookwrapper=True)
def pytest_runtest_makereport(item, call):
    outcome = yield
    setattr(item, "rep_" + outcome.get_result().when, outcome.get_result())

pytest calls a hook defined in a test module, but only for that module's tests. Move it to conftest.py and it covers everything under it.

The second pattern is that attach_api reads a response by shape, not by type. That is why the stub Response class in the demo can be swapped for requests or httpx and nothing else in the file changes. The Python API page documents the shape it looks for.

The browser demos

test_selenium.py, test_screenshot.py and test_playwright.py drive a real browser to exercise screenshot-on-failure. Their dependencies are kept out of requirements.txt and each file importorskips itself when its driver is missing, so a checkout with neither installed still runs the bank cleanly.

Each file has exactly one test that fails on purpose — that is the one that puts a screenshot in the report. What all three demonstrate by omission is the actual feature: there is no conftest, no hook and no call to attach. The reporter photographs the driver itself, because the driver is in the test's own fixtures and the reporter is already standing in its teardown. The Playwright file proves the mechanism is not Selenium-specific — the reporter recognises a browser by what it can do, so page.screenshot() is found the same way get_screenshot_as_png() is. More in Screenshots.

The Gherkin demos

test_gherkin.py runs a checkout scenario from tests/functional/features/checkout.feature. Nothing in it mentions the reporter — a pytest-bdd scenario is already a list of named steps, so the Test Steps tab fills itself, each step timed and badged with its Gherkin keyword. test_gherkin_screenshot.py combines it with a browser against ui_features/heading.feature, so a failing Then carries a picture filed against that step rather than against the test as a whole.

$ pip install pytest-bdd
$ pytest tests/functional/test_gherkin.py --html-report=./report

test_steps.py shows the same tab filled by hand, with the step helper rather than pytest-bdd — a page object whose methods are the steps. Both are covered on Steps and BDD.

The rest of the bank

Setting up the GitHub Action

The Action repository is the best documented of the three: it ships CONTRIBUTING.md, RELEASING.md, docs/how-it-works.md, docs/troubleshooting.md, four workflows and a commented .yamllint.yml. This is the short version.

Setup and tests

$ git clone https://github.com/prashanth-sams/pytest-html-reporter-action.git
$ cd pytest-html-reporter-action
$ python -m pip install pytest pyyaml
$ python -m pytest

Two test files, and they check different kinds of thing:

• tests/test_phr.pyThe helper — the counts, the pass rate, the markdown summary, the thresholds and the pytest command it builds.
• tests/test_action_yml.pyThe metadata — the shape the runner requires, the wiring between steps, and whether the README still describes the inputs that exist.

That last check is why a new or renamed input needs its README row in the same commit. The test will tell you.

CI runs that suite on a matrix of ubuntu-latest, macos-latest and windows-latest against Python 3.9 and 3.13, with fail-fast: false.

Linting

Most of this action is bash inside YAML, and shellcheck cannot read YAML. So extract_shell.py lifts the run: blocks out first, replacing each ${{ ... }} with a quoted placeholder so shellcheck does not read ${{ as a broken parameter expansion and give up. Without that step the largest part of the codebase goes unlinted.

$ python scripts/extract_shell.py action.yml .shellcheck
$ shellcheck --shell=bash .shellcheck/*.sh
$ yamllint -c .yamllint.yml action.yml .github/workflows examples

CI additionally runs actionlint and validates the metadata against the official schema with check-jsonschema --builtin-schema vendor.github-actions action.yml.

.yamllint.yml relaxes exactly one default rule, with the reason in a comment: YAML 1.1 reads a bare on: as the boolean true, which is every workflow file ever written.

Running the action for real

.github/workflows/self-test.yml is the only place the composite wiring is exercised end to end, so a change to action.yml needs a push to see it work. It runs four jobs:

To reproduce a self-test run by hand, against the sample project the workflow builds:

$ bash tests/fixtures/make-project.sh /tmp/sample fail
$ cd /tmp/sample
$ pip install pytest-html-reporter
$ python /path/to/scripts/phr.py resolve --path report
$ pytest tests/ --html-report=report --report-open=none
$ python /path/to/scripts/phr.py summarize --json report/output.json --exit-code 1

That is the Action's own sequence with the YAML taken away — resolve works out the paths, pytest writes the report, summarize produces the counts and the verdict. See GitHub Action for what each step does in a workflow.

House style and dependencies

The helper is stdlib-only and stays that way. It runs on whatever Python the workflow set up, and an action that installs its own dependencies to print a table is an action that breaks on somebody's locked-down runner.

Comments explain why, not what. The action works around a number of sharp edges in the plugin's behaviour, and each workaround carries a comment saying what would otherwise go wrong — because without it the next reader deletes it.

.github/dependabot.yml watches two directories weekly with a deps commit prefix: / for the actions action.yml itself calls, and /.github/workflows for the ones this repository's own workflows call. GitHub treats those as separate ecosystems, and watching only one is the usual way half your uses: pins go stale.

The pull request checklist, from CONTRIBUTING.md and the tests that enforce it:

• python -m pytest passes
• A new or renamed input has its README row
• A change to action.yml has been through self-test.yml on a push
• CHANGELOG.md has an entry, if this is user-visible

Setting up the VS Code extension

The extension repository has no CONTRIBUTING.md and no CI workflows. The development story is in package.json's scripts, .vscode/, esbuild.js and tools/preview.mjs.

Setup and the inner loop

$ git clone https://github.com/prashanth-sams/pytest-html-reporter-vscode.git
$ cd pytest-html-reporter-vscode
$ npm install
$ npm run build          # node esbuild.js  ->  dist/extension.js
$ npm run watch          # the same, rebuilding on change

Then press F5 for an Extension Development Host and open a folder containing a report. .vscode/launch.json runs npm: build as its preLaunchTask and points outFiles at dist/**/*.js; .vscode/tasks.json wires build and watch to the $esbuild and $esbuild-watch problem matchers — which is what the small problemMatcherPlugin in esbuild.js exists to feed.

The bundle targets node20, CommonJS, with vscode external.

The preview renderer

Launching the Extension Development Host to look at a colour is a slow loop. npm run preview renders the same markup the webview gets to standalone HTML in .preview/, so a style change is one command and a browser refresh away.

$ npm run preview                  # the bundled fixture, with failures
$ npm run preview -- all-passed    # a green run
$ npm run preview -- no-config     # nothing found yet
$ npm run preview -- error         # a corrupt report

$ PREVIEW_REPORT=/path/to/output.json npm run preview   # one of your own runs

Four fixtures in both themes gives the eight files checked in under .preview/. The script compiles first — npm run preview is compile-tests followed by node tools/preview.mjs — and imports the real renderer and history service out of out/, so what you see is the code that ships.

The subtlety, and the reason the harness is more than twenty lines: a webview inherits a large set of --vscode-* custom properties from the host, and nothing outside VS Code supplies them. So tools/preview.mjs stubs the real Dark+ and Light+ values. Without them every themed colour falls back to its hard-coded default and the preview tells you nothing about what users actually see.

Pointing PREVIEW_REPORT at a real run makes the history charts read that report's own archive/ too, which is the only way to see the trend and flaky panels with plausible data.

Tests and type-checking

$ npm test        # tsc -p . --outDir out, then node --test "out/tests/**/*.test.js"
$ npm run lint    # tsc --noEmit

Plain node --test over compiled JavaScript, with no VS Code test harness, because none of the tested code touches the vscode API. Three suites:

• outputJson.test.tsParsing and shape validation — which is what stops an unrelated output.json in a workspace being shown as test results.
• history.test.tsFlaky verdicts and trends computed across the archive fixtures.
• testLocator.test.tsThe def-scanning source-line recovery that stands in for the file and line fields output.json never writes.

tests/fixtures/ holds an output.json, eight archived output_*.json builds for the history tests, and a deliberately corrupt.json.

tsconfig.json is strict and then some: strict, noUnusedLocals, noUnusedParameters, noImplicitReturns, noFallthroughCasesInSwitch and forceConsistentCasingInFileNames. There is no ESLint — npm run lint is the compiler.

Packaging

$ npm run package     # vsce package  ->  pytest-html-reporter-vscode-<version>.vsix

vsce package triggers vscode:prepublish, which is a production esbuild — minified, no sourcemap. .vscodeignore keeps src/, tests/, out/, tools/, .preview/, node_modules/, every .ts and every .map out of the VSIX, so what ships is dist/extension.js, resources/, the README, the CHANGELOG and the LICENSE.

Releasing

Three projects, three processes, and two of them have a step that silently ships to nobody if you forget it. The version numbers are independent — the Action's changelog links out to the plugin's rather than restating it, and there is no relationship between the three numbers. Do not try to synchronise them.

The plugin, to PyPI

RELEASE_GUIDE.md documents this, though it is written for 0.3.0 specifically. The version lives in exactly two places that must agree:

$ pip install --upgrade build twine
$ rm -rf build/ dist/ *.egg-info
$ python -m build
$ twine check dist/*
$ twine upload --repository testpypi dist/*     # optional
$ twine upload dist/*
$ git tag -a v0.4.3 -m "Release version 0.4.3"
$ git push origin v0.4.3

Then draft a GitHub release from the tag, with CHANGELOG.txt as the description.

MANIFEST.in also prunes tests/, test_draft/ and venv/, and excludes CHANGELOG.txt and requirements.txt.

One manual step that is easy to miss: the PyPI badge in README.md carries a cache-buster, badge.fury.io/py/pytest-html-reporter.svg?v=0.4.3. It has to be bumped by hand at release time, so a stale badge is a slip in this process rather than a broken publish.

The Action, to the Marketplace

The Marketplace name comes from name: in action.yml, not from the repository, and the listing lives at github.com/marketplace/actions/pytest-html-reporter. Renaming it breaks that URL and every link to it; tests/test_action_yml.py fails the build if it changes. The repository is called ...-action only because the plugin already owns the shorter name.

1. Update the changelog. CHANGELOG.md, in the repository root.

2. Tag and push.

   shell Copy

   $ git tag -a v1.2.3 -m "v1.2.3"
   $ git push origin v1.2.3

3. Draft the release from the tag, on GitHub, ticking Publish this Action to the GitHub Marketplace.

4. Move the major tag. .github/workflows/release.yml does this on publish, so normally there is nothing to do. By hand it is:

   shell Copy

   $ git tag -f v1 v1.2.3
   $ git push origin v1 --force

The release workflow is careful in two ways worth knowing about. It skips prereleases and any tag that is not exactly vX.Y.Z, and it refuses to drag the major tag backwards: if a backport is published after a newer version, v1 stays where it is, because @v1 means the newest v1, not the last published.

The extension, to the VS Code Marketplace

$ npm run package     # produces the .vsix
$ npx vsce publish    # ships it

The publisher is prashanth-sams and the extension id is prashanth-sams.pytest-html-reporter-vscode.

Community and support

Where to file what

Almost everything goes to the plugin's issue tracker, including every extension bug — the extension points its bugs URL there on purpose. Action issues are the exception, and only when the problem is in getting the report built.

Asking a good question

Most issues on any of these repositories are answerable in one reply if the report is reproducible, and take four rounds of questions if it is not. Include:

• The exact command or workflow YAMLThe whole pytest invocation, or the whole uses: block with its with: inputs.
• Versionspytest-html-reporter --version, pytest --version, the Python version, and the Action or extension version if one is involved.
• The logNot a screenshot of it. A pytest.UsageError names the option it rejected, and that name is usually the whole answer.
• For an Action issue, its outputsThe report-dir and status outputs say what the run thought it did.
• A reproduction from the bank if you can"pytest tests/functional/test_attachments.py also does it" removes your suite from the picture entirely.

Check the FAQ first — empty tabs, a missing PR comment, an unexpectedly large report and exit code 4 are all answered there.

Project links

The live demo

The Action repository rebuilds a report on every push and publishes it to GitHub Pages at prashanth-sams.github.io/pytest-html-reporter-action. It is deliberately built from the failing flavour of the sample project — three passes, a failure, an error and a skip — because a demo that is green all the way down says nothing about what the report does with the runs people actually need to read. It is also the only place the pages-artifact input is exercised end to end.