pytestHTML Reporter
Home Docs GitHub Action
Integrations

GitHub Action

One step installs the plugin, runs pytest, and delivers the result to four places at once — the workflow job summary, a pull request comment that edits itself on every push, a downloadable artifact, and a set of step outputs a later step can act on. This is the reference for every input, every output, and every workflow shipped in the repository.

What the action does

pytest-html-reporter-action is a composite GitHub Action that wraps the pytest-html-reporter plugin. One step optionally installs Python and your dependencies, runs pytest with the plugin’s --html-report flag, and then publishes the result.

On top of the plugin it adds the machinery a CI run needs: it resolves the report path up front so strftime placeholders cannot be expanded twice, probes pytest --help so pinning an older plugin degrades rather than aborting the run, carries the build archive between runs with actions/cache so Trends and Archives have history to read, gates the job on pass rate and coverage, and escapes every test name on the way into the summary — because on a fork’s pull request that text belongs to somebody else.

NamesThe Marketplace listing is pytest-html-reporter; the repository is prashanth-sams/pytest-html-reporter-action. The -action suffix exists only because the plugin already owns the shorter name, and it appears in the uses: line and nowhere else.

Quick start

This is the whole of a first setup: install, run, report, upload, fail if anything failed. Everything else on this page is an addition to this shape.

yaml examples/basic.yml — reach for this first; most projects never grow out of it
# The whole thing: install, run, report, upload, fail if anything failed.
name: tests

on:
  push:
    branches: [main]
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: prashanth-sams/pytest-html-reporter-action@v1
        with:
          python-version: '3.12'
          requirements: requirements.txt
          tests: tests/
          title: CI

Nothing else is required. Every input is a string and every input has a default, so you set as few or as many as you like — and booleans are the strings 'true' and 'false', because a workflow input is always a string.

The uses line, and which ref to pin

The project follows semantic versioning. The v1 tag moves to the newest v1.x.y release, so @v1 picks up fixes; pin an exact release to hold still.

yaml The moving major tag — fixes arrive on their own
- uses: prashanth-sams/pytest-html-reporter-action@v1
  with:
    tests: tests/

What the summary says

The same markdown is used for the job summary, the pull request comment and the summary output. It is built from the run’s output.json, in this order.

When no report was written at all, the heading is followed by a single paragraph naming output.json and the directory it was expected in, and nothing else.

Screenshot: assets/img/shots/action-job-summary.png The GitHub Actions run page, light theme, 1440px wide, scrolled to the rendered job summary: the red cross heading, the counts table, the Coverage line, and one Failures block expanded to show its message fence.
SizeGitHub caps a job summary at 1MiB; the action trims to 900KB, and a pull request comment to 60,000 characters. Both trims close an odd code fence and any unclosed <details> first, so nothing after the cut is silently folded away.

Inputs

Every input is a string, every input has a default, and no input is required. Nothing is interpolated into a shell: every input reaches bash as a PHR_* environment variable, never as ${{ ... }} spliced into a run: body — that would make pytest-args shell source.

What to run

InputTypeRequiredDefaultWhat it does
testsstring (multiline)no''Test paths to hand to pytest, one per line. Blank lines are dropped. Leave empty to let pytest pick its own targets from your rootdir configuration.
pytest-argsstringno''Extra arguments appended to the pytest command, parsed the way a shell would parse them — quotes respected, via shlex.split. Unbalanced quoting fails the run rather than being guessed at.
working-directorystringno.Directory to run pytest in. Every path is resolved against it. The action fails up front if it does not exist.

The environment

The install step is skipped entirely only when install is false and both requirements and extra-packages are empty.

InputTypeRequiredDefaultWhat it does
python-versionstringno''Python to set up with actions/setup-python@v5 before running. Leave empty to use whatever Python is already on the runner.
installboolean stringno'true'Install pytest-html-reporter before running. Set to 'false' when your own requirements file already pins it.
plugin-versionstringno''Version specifier for the plugin. A value starting with a digit becomes ==<value>; anything else is appended raw, so >=0.3.8 works. Empty installs the latest release. Ignored when install is false.
requirementsstringno''Requirements file to install before running. Resolved against working-directory, and installed before the plugin.
extra-packagesstringno''Further pip packages to install, whitespace separated. Globbing is disabled while installing, so a name containing * is not expanded against the working directory.

The report

Each of these maps onto a plugin flag, and each is left out of the command entirely when empty — so the plugin’s own defaults, and any report_* key in your pytest.ini, still apply. Setting one here overrides the ini key. Every flag is also checked against pytest --help before it is passed; see flag probing.

Two names to memoriseNearly every input shares its name with its flag. Two do not, and pytest’s error messages name the flag: --html-report is the report-path input, and --report-link is the report-links input.
InputTypeRequiredDefaultWhat it does
report-pathstringnoreportWhere the report goes, passed as --html-report. A value containing .html anywhere names the file; anything else names a directory. strftime placeholders are expanded once, before the run starts. Defaults to report rather than the plugin’s own default of the working directory, because a CI job wants the report, its output.json and its archive/ in one uploadable folder.
titlestringno''Report title, passed as --title. Shown cut to 20 characters, with the full text as the heading’s tooltip.
environmentstringno''Name of the environment under test, passed as --environment.
build-infostring (multiline)no''Extra KEY=VALUE details shown in the report, one per line, each passed as its own --build-info. Blank lines and lines starting with # are ignored.
report-linksstring (multiline)no''Extra side-nav links, one LABEL=URL per line, each passed as its own --report-link. A relative path works; a link with a scheme the plugin does not know is dropped silently.
archive-countinteger stringno''Maximum builds kept in the Archives section (--archive-count). Empty keeps every one. '0' deletes the whole archive and hides the Archives section; '1' is read as “this build and no others”.
archive-daysnumber stringno''Keep only builds from the last N days (--archive-days). Accepts fractions, e.g. '0.5'.
archive-sincestringno''Delete archived builds older than this date (--archive-since), given as YYYY-MM-DD or 'YYYY-MM-DD HH:MM'. Read in the runner’s timezone, which is UTC.
report-logsenum: all, failed, noneno''Whose captured stdout, stderr and logging to keep. Validated up front.
report-log-limitinteger stringno''Maximum characters of captured output kept per test; '0' keeps everything. Must be a non-negative whole number.
report-attachmentsenum: all, failed, noneno''Whose attachments to keep.
report-attachment-limitinteger stringno''Maximum characters kept per attached payload; '0' keeps everything.
report-stepsenum: all, failed, noneno''Whose test steps to keep.
report-step-limitinteger stringno''Maximum steps kept per test; '0' keeps every one.
report-coverageenum: auto, noneno''Whether to build the Test Coverage tab.
report-coverage-filestringno''Read coverage from this file — a coverage.json, a Cobertura coverage.xml or a .coverage data file — instead of looking for one. Naming a file here is final: if it cannot be read the Coverage tab is blank and the coverage this run measured is not used instead. The action warns when that happens.
report-coverage-limitinteger stringno''Maximum files listed on the Coverage tab; '0' lists every one.
report-openenum: auto, always, noneno'' (see below)Whether to open the finished report in a browser. Set it only on a self-hosted runner with a desktop.
Why report-open is differentThe action passes --report-open whether or not you set it, and none when you have not. The plugin’s default is to open the finished report and it declines on a runner — unless a pytest.ini says report_open = always, which skips every check and hands the report to a console browser with no TTY, CI or DISPLAY test at all. auto is not a defence the action controls, because the CLI beats the ini.
Negative limits are refusedThe plugin clamps a negative limit to 0, and 0 means “no limit” for every one of the *-limit inputs — so '-1' would silently mean the exact opposite of what it looks like. The action fails up front and says so.

Publishing

These decide where the result goes: the job summary page, a pull request comment, a workflow artifact, a Pages artifact, and the cached build history.

InputTypeRequiredDefaultWhat it does
job-summaryboolean stringno'true'Write the run’s summary to the workflow’s job summary page. When there is no $GITHUB_STEP_SUMMARY to write to, the markdown is printed to stdout instead.
summary-titlestringnopytest-html-reporterHeading used for the job summary and the pull request comment.
failure-limitinteger stringno'10'Maximum failures listed in the summary and the comment; the rest are counted in a trailing line. '0' lists none. A non-numeric value falls back to the default rather than abandoning the run.
suite-limitinteger stringno'20'Maximum suites listed. The Suites table is only rendered at all when the run has more than one suite.
slowest-limitinteger stringno'5'How many of the slowest tests to list; '0' lists none.
report-urlstringno''Link to the published report, shown in the summary and the comment as Open the report — for instance your GitHub Pages URL.
commentboolean stringno'false'Post the summary as a sticky comment on the pull request, updating the same comment on every run instead of piling new ones up.
comment-modeenum: always, on-failurenoalwaysWhen to comment. on-failure compares the report’s status against PASS, so a run that produced no report (UNKNOWN) also comments.
pr-numberinteger stringno''Pull request to comment on. Only needed when the event does not carry one — a workflow_run job, say. Empty uses the event’s own number. Must be digits.
github-tokenstringno${{ github.token }}Token used to post the comment. Needs pull-requests: write. A pull_request run from a fork gets a read-only token and cannot comment. An empty value makes the comment step warn and return rather than error.
upload-artifactboolean stringno'true'Upload the report directory as a workflow artifact with actions/upload-artifact@v4.
artifact-namestringnopytest-html-reportName of the uploaded artifact. Must be unique within a run — upload-artifact@v4 refuses two artifacts of the same name.
artifact-retention-daysinteger stringno''Days to keep the artifact. Empty uses the repository default.
artifact-include-historyboolean stringno'true'Include the archive/ folder of past builds in the artifact. Set to 'false' to upload only the current report; the action then adds an exclusion pattern for <report-dir>/archive.
artifact-if-no-files-foundenum: warn, error, ignorenowarnWhat upload-artifact does when nothing matched.
artifact-overwriteboolean stringno'false'Replace an artifact of the same name instead of failing. A matrix is better served by giving each job its own artifact-name.
pages-artifactboolean stringno'false'Also upload the report as a GitHub Pages artifact with actions/upload-pages-artifact@v3, ready for a following job to deploy with actions/deploy-pages.
historyboolean stringno'false'Carry the archive/ folder and output.json between runs with actions/cache, so the Trends, Archives and Analytics tabs have more than one build to show.
history-keystringnopytest-html-reporter-historyCache key prefix used for the build history. The full key appends runner.os, github.ref_name, github.run_id and github.run_attempt.

Gates

Four inputs decide whether the job goes red. They are evaluated together, and every reason that applies is collected — a failing run tells you everything that was wrong, not just the first thing. See thresholds and gates.

InputTypeRequiredDefaultWhat it does
fail-on-errorboolean stringno'true'Fail the job when pytest fails — any non-zero exit code other than 5, which fail-on-empty owns. It also covers the case where pytest exits 0 but wrote no output.json. Set to 'false' to always go green and decide with the outputs yourself.
fail-on-emptyboolean stringno'true'Fail the job when pytest collects no tests at all (exit code 5). Left on, because a mistyped -k that quietly produces a green run is worse than a red one. When exit code 5 occurs this input alone decides; fail-on-error is not consulted.
min-pass-ratenumber stringno''Fail the job when the pass rate falls below this percentage. Empty applies no threshold. A run where nothing decisive happened fails rather than passing by default. A non-numeric value is warned about and ignored.
min-coveragenumber stringno''Fail the job when coverage falls below this percentage. Empty applies no threshold. A run that produced no coverage data fails rather than passing by default.

Outputs

Every output is declared in action.yml and wired to one of the action’s internal steps. Read them from a later step by giving the action an id.

OutputTypeWhat it holds
statusstringPASS, FAIL, or UNKNOWN when no report was produced. Taken from the report’s own status field when it is PASS or FAIL, otherwise derived: FAIL if there were failures or errors, PASS otherwise.
totalintegerTests executed — passes, failures, errors, skips, xpasses and xfails summed. Reruns are attempts, not tests, and are not counted here.
passedintegerTests that passed.
failedintegerTests that failed.
errorsintegerTests that errored.
skippedintegerTests that were skipped.
xpassedintegerTests that passed while expected to fail.
xfailedintegerTests that failed as expected.
rerunintegerRerun attempts recorded across the run.
suitesintegerTest suites in the run.
pass-ratenumber stringpassed / (passed + failed + errors) as a percentage, trimmed of trailing zeros. Empty when nothing decisive ran.
coveragenumber stringCoverage percentage, or empty when the run produced none.
tests-durationnumberSummed test durations in seconds, to 2 places. This is what the report records — the run’s own wall clock is not written to output.json — so it leaves out collection, session fixtures and the gaps between tests.
wall-clocknumber stringSeconds the pytest step took, measured by this action with date +%s either side of the run.
exit-codeintegerThe exit code pytest returned. See the table below.
report-filepathAbsolute path of the generated HTML report.
report-dirpathAbsolute path of the directory holding the report.
json-pathpathAbsolute path of the run’s output.json.
summarymarkdownThe markdown summary, for reuse in a later step. Multi-line, so pass it through env: rather than splicing it into a command.
artifact-idstringId of the uploaded artifact, when one was uploaded.
artifact-urlurlURL of the uploaded artifact, when one was uploaded. Rendered in the summary as Download the artifact.

Reading the outputs

Pass multi-line values such as summary through env: rather than splicing them into a command — the summary is markdown built partly out of test names, and a test name is not something to hand to a shell.

yaml Open an issue when a nightly run goes red, without failing the job
- uses: prashanth-sams/pytest-html-reporter-action@v1
  id: report
  with:
    tests: tests/
    fail-on-error: 'false'      # decide for yourself, below

- name: Tell the team
  if: steps.report.outputs.status != 'PASS'
  run: gh issue create --title "Nightly run: $FAILED failing" --body "$SUMMARY"
  env:
    GH_TOKEN: ${{ github.token }}
    FAILED: ${{ steps.report.outputs.failed }}
    # Through env, never spliced into the command: the summary is multi-line
    # markdown built partly out of test names, and a test name is not
    # something to hand to a shell.
    SUMMARY: ${{ steps.report.outputs.summary }}
NoteWhen the action fails the job, the step that used it has failed, and a later step needs if: always() to run at all. The tidier way round is fail-on-error: 'false' plus your own decision — then nothing has failed and the outputs are simply there.

Pull request comments

Reach for this when you want reviewers to see the result without opening the run. This is the same-repository case: the job needs pull-requests: write, which a pull request from a fork will not have. For those, see fork pull requests.

yaml examples/pr-comment.yml — use when reviewers should see the result in the pull request itself
# A sticky summary on every pull request, edited in place on each push.
#
# This is the same-repository case. A pull request from a fork gets a
# read-only token and cannot comment - see fork-pr.yml for that.
name: tests

on:
  pull_request:

permissions:
  contents: read
  pull-requests: write

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: prashanth-sams/pytest-html-reporter-action@v1
        with:
          tests: tests/
          comment: 'true'
          comment-mode: always
          environment: ci
          build-info: |
            branch=${{ github.head_ref }}
            commit=${{ github.sha }}

comment: 'true' posts the summary as a sticky comment: the body carries the hidden marker <!-- pytest-html-reporter-action -->, and the action paginates the pull request’s existing comments looking for it. If it finds one it updates that comment; otherwise it creates one. So the next push edits the comment rather than adding another.

Screenshot: assets/img/shots/action-pr-comment.png A pull request conversation, light theme, showing the single sticky comment from the action with its “edited” badge visible — the counts table and one expanded failure — and the comment count on the pull request still at one after two pushes.

When the comment step runs

All three conditions must hold:

  1. comment is 'true', and
  2. the event is pull_request or pull_request_target, or pr-number is set explicitly, and
  3. comment-mode is always, or the run’s status is not PASS.

That second clause is why pr-number exists: a workflow_run job carries no pull request of its own.

It warns, it does not fail

Every failure mode here produces a warning annotation and lets the run continue, because a report is not worth turning a run red over: an empty github-token, no comment body produced, no pull request in the event, or an API call that threw. Look for a warning annotation rather than a red step.

Fork pull requests

A pull_request run triggered from a fork is handed a read-only GITHUB_TOKEN by GitHub, on purpose: the code being tested is not yours. The action does not fight this — it catches the API error, warns, and carries on. Commenting on a fork’s pull request takes two workflows.

Reach for this when you accept pull requests from forks and want them commented on. The first workflow runs the tests with a read-only token and writes the summary and the claimed pull request number out as a small artifact. The second, triggered by workflow_run, runs from your default branch with a writable token and none of the fork’s code, and posts the comment.

yaml examples/fork-pr.yml — the untrusted half: run the tests, write the summary out as an artifact
# Commenting on a pull request that came from a fork.
#
# A `pull_request` run from a fork is handed a read-only token on purpose:
# the code being tested is not yours. So the run that executes the tests
# writes the report and the PR number as artifacts, and a second workflow -
# triggered by `workflow_run`, running from your default branch, with a
# writable token and none of the fork's code - posts the comment.
name: tests

on:
  pull_request:

permissions:
  contents: read

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: prashanth-sams/pytest-html-reporter-action@v1
        id: report
        with:
          tests: tests/
          comment: 'false'          # nothing to comment with, here
          artifact-name: pytest-html-report

      # Written after the tests, so the fork's code has had its turn - and
      # checked again by the second workflow against the commit this run
      # actually tested, because an artifact out of a job that ran somebody
      # else's tests is a claim, not a fact.
      - name: Keep the summary and the PR number for the next workflow
        if: always()
        run: |
          set -euo pipefail

          rm -rf comment
          mkdir -p comment
          printf '%s' "$PR_NUMBER" > comment/pr-number
          printf '%s' "$SUMMARY" > comment/body.md
        env:
          PR_NUMBER: ${{ github.event.number }}
          SUMMARY: ${{ steps.report.outputs.summary }}

      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: pytest-html-report-comment
          path: comment/

Note the ordering: the two text files are written after the tests, so the fork’s code has already had its turn, and if: always() on both trailing steps means a red run still gets its comment.

yaml examples/fork-pr-comment.yml — the privileged half; save it as its own workflow file
# The second half of fork-pr.yml. Save this as its own workflow file.
#
# `workflow_run` runs from your default branch with your permissions, and
# never checks out the fork's code. It reads two small text files, and it
# checks that the pull request they name is really the one that produced
# them - an artifact comes out of a job that ran the fork's tests, so it is
# not on its own evidence of anything.
name: tests comment

on:
  workflow_run:
    workflows: [tests]
    types: [completed]

permissions:
  contents: read
  actions: read
  pull-requests: write

jobs:
  comment:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/download-artifact@v4
        with:
          name: pytest-html-report-comment
          path: comment
          run-id: ${{ github.event.workflow_run.id }}
          github-token: ${{ github.token }}

      - uses: actions/github-script@v7
        with:
          script: |
            const fs = require('fs');

            const marker = '<!-- pytest-html-reporter-action -->';
            const run = context.payload.workflow_run;

            const claimed = Number(fs.readFileSync('comment/pr-number', 'utf8').trim());

            if (!Number.isInteger(claimed) || claimed <= 0) {
              core.setFailed('the artifact did not carry a usable pull request number');
              return;
            }

            // The artifact was produced by a job running the fork's code, so
            // the number in it is a claim rather than a fact. A pull request
            // whose head is the commit this run tested is the one thing that
            // cannot be forged from inside that job.
            const { data: pull } = await github.rest.pulls.get({
              ...context.repo, pull_number: claimed,
            });

            if (pull.head.sha !== run.head_sha) {
              core.setFailed(
                `pull request #${claimed} is at ${pull.head.sha}, but this run ` +
                `tested ${run.head_sha}. Refusing to comment on a pull request ` +
                `the artifact only claims to belong to.`);
              return;
            }

            const body = marker + '\n'
              + fs.readFileSync('comment/body.md', 'utf8').slice(0, 60000);

            const existing = await github.paginate(github.rest.issues.listComments, {
              ...context.repo, issue_number: claimed, per_page: 100,
            });
            const mine = existing.find((comment) => comment.body?.includes(marker));

            if (mine) {
              await github.rest.issues.updateComment({
                ...context.repo, comment_id: mine.id, body,
              });
            } else {
              await github.rest.issues.createComment({
                ...context.repo, issue_number: claimed, body,
              });
            }
Why the SHA check is the pointAn artifact comes out of a job that ran the fork’s tests, so the pull request number inside it is a claim, not a fact. Before commenting, the second workflow fetches the pull request that number names and refuses unless pull.head.sha equals workflow_run.head_sha — the commit this run actually tested. That equality is the one thing that cannot be forged from inside the untrusted job.

Everything the tests print is untrusted text

On a fork’s pull request, test names and assertion messages belong to the fork author, and they end up in a job summary and in a comment posted under the repository’s own identity. The action hardens every path they travel.

Build history

Reach for this when you have a scheduled or repeated run and want to see change over time. The report’s Trends, Archives and Analytics tabs are built from previous builds, which a fresh runner does not have. history: 'true' carries them between runs with the Actions cache.

yaml examples/history.yml — use on a nightly or scheduled run, where trends are the point
# Trends, Archives and Analytics need more than one build to say anything.
# The cache carries the archived builds from run to run; the retention
# limits stop the report growing without end.
name: nightly

on:
  schedule:
    - cron: '0 2 * * *'
  workflow_dispatch:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: prashanth-sams/pytest-html-reporter-action@v1
        with:
          tests: tests/
          title: Nightly
          history: 'true'
          archive-days: '30'
          environment: staging

What is cached

Both <report-dir>/archive and <report-dir>/output.json. Carrying only archive/ accumulates nothing at all — a build joins the archive only when the next run rotates its output.json in, so the cache has to carry that json too.

The cache key is <history-key>-<runner.os>-<github.ref_name>-<github.run_id>-<github.run_attempt>, with restore-keys falling back first to the branch prefix and then to the OS prefix. So history follows a branch where it can and falls back to the same OS otherwise, and every run writes a new immutable cache entry rather than trying to overwrite one.

What the prime step fixes before pytest runs

A cache restore on its own is not enough, because two things have to be true for a build to join the archive.

  1. A placeholder report file. The plugin archives a build by rotating the previous output.json into archive/, and it only does that when the previous run’s report file is on disk. On a fresh runner it never is, so every run would quietly replace its predecessor and the archive would stay empty for ever. The action stands in an empty file and says so; the run then overwrites it.
  2. Quarantining unreadable builds. The plugin reads every archive/*.json without guarding the read, so a single truncated or foreign file raises inside pytest_terminal_summary and no report is written at all — and a restored cache is exactly where such a file comes from. Each one is validated, and a file that fails is renamed to <name>.json.unreadable, with a warning giving the count and the reason.

Because the restore puts the previous build’s output.json on disk before pytest runs, a run that then writes none of its own — it crashed, it collected nothing — would leave the previous build there to be summarised. The action compares the file’s start_time against the restored one and, if they match, reads it as no report at all rather than reporting somebody else’s passes as this run’s.

Keeping the report from growing without end

Every retained build costs roughly 5KB of the page, so an hourly run with no limit reaches a multi-megabyte report within a couple of months and the page gets slow to open. Three inputs prune it, and they intersect — a build has to satisfy every limit you set to be kept: archive-count, archive-days and archive-since.

Two things that quietly defeat historyA report-path containing strftime placeholders makes a new folder every run, so every run starts from nothing — put the date in artifact-name instead. And a matrix needs a history-key per cell, or two Python versions interleave builds that are not comparable.

The first run with history on still shows one build. It takes two.

GitHub Pages

Reach for this when you want a URL anyone can open, with no artifact to download and unzip. pages-artifact: 'true' runs actions/upload-pages-artifact@v3 against the report directory. That is as far as an action can take you: deploying to Pages needs its own job with the github-pages environment and pages: write plus id-token: write, and an action cannot grant itself either.

yaml examples/pages.yml — use when the report should live at a URL rather than inside a zip
# Publishing the report to GitHub Pages.
#
# The deploy has to be its own job: it needs the github-pages environment
# and its own permissions, neither of which an action can grant itself.
name: tests

on:
  push:
    branches: [main]

permissions:
  contents: read

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: prashanth-sams/pytest-html-reporter-action@v1
        with:
          tests: tests/
          pages-artifact: 'true'
          history: 'true'
          # The report is one page, so name it index.html and Pages serves
          # it at the root of the site.
          report-path: report/index.html

  deploy:
    needs: test
    runs-on: ubuntu-latest

    permissions:
      pages: write
      id-token: write

    environment:
      name: github-pages
      url: ${{ steps.deploy.outputs.page_url }}

    steps:
      - uses: actions/deploy-pages@v4
        id: deploy

Two things in that workflow are load-bearing. report-path: report/index.html, because the report is one page and naming it index.html makes Pages serve it at the root of the site; and history: 'true', because a published report is exactly where Trends and Archives earn their keep. Once the site is up, feed its URL back in with report-url and the job summary and pull request comment carry an Open the report link straight to it.

Screenshot: assets/img/shots/action-pages-report.png The deployed report open at a github.io URL with the browser address bar visible, light theme, 1440px wide, on the Dashboard tab — beside it the workflow’s Actions graph showing the two jobs, test then deploy, with the github-pages environment badge.

Thresholds and gates

Reach for a hard floor when the rule is simple enough to state as a number. Both gates are enforced by the action itself.

yaml Two hard floors, enforced by the action itself
- uses: prashanth-sams/pytest-html-reporter-action@v1
  with:
    tests: tests/
    min-pass-rate: '95'
    min-coverage: '80'

The rules, in order

  1. Exit code 5 — no tests collected — is handled first and on its own, by fail-on-empty. fail-on-error is not consulted for code 5.
  2. Any other non-zero exit code fails when fail-on-error is true, with the code spelled out in the annotation.
  3. A clean exit with no report also fails when fail-on-error is true.
  4. min-pass-rate, when set and numeric. If the pass rate cannot be measured at all the run fails rather than passing by default.
  5. min-coverage, when set and numeric. A run that produced no coverage data fails.

Every reason that applies is collected. Each is emitted as an ::error title=pytest-html-reporter:: annotation, and a > [!CAUTION] blockquote listing all of them is appended to the job summary.

Screenshot: assets/img/shots/action-gate-failure.png A failed run page, light theme, showing the red ::error annotation at the top (“pass rate 82% is below the required 95%”) and, scrolled together into the same shot, the [!CAUTION] blockquote appended to the job summary below it.
A threshold that cannot be measured fails loudlymin-coverage on a run that produced no coverage is an error, not a free pass. The same rule governs min-pass-rate on a run where nothing decisive happened.

Pass rate is deliberately narrow

pass-rate = passed / (passed + failed + errors), as a percentage. Skipped, xfailed and xpassed tests are in neither half — none of them is a pass-or-fail signal, and counting them would quietly move the threshold you set. When the denominator is zero, pass-rate is empty and any min-pass-rate fails.

Exit code against report status

The plugin calls a run FAIL only when a suite holds a failure or an error. pytest has more ways to exit non-zero than that: a strict xfail, pytest.exit(), an interrupted run. The exit code is what the job is decided on, and status is the report’s headline. When the two disagree the action says so rather than picking one quietly.

Exit codeWhat it means
1Tests failed.
2The run was interrupted.
3An internal error.
4A usage error: pytest refused an argument. Nothing ran.
5No tests were collected. Governed by fail-on-empty, not fail-on-error.

Deciding for yourself

Reach for this when “any red test fails the build” is too blunt — a few flakes are survivable, a collapse is not. fail-on-error: 'false' lets the red run through, and a following step reads the outputs and makes the call.

yaml examples/thresholds.yml — use when a handful of flakes should not stop the build
# Deciding for yourself what a failing run means.
name: tests

on: [push]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: prashanth-sams/pytest-html-reporter-action@v1
        id: report
        with:
          tests: tests/
          # Let a red run through, and judge it below.
          fail-on-error: 'false'

      # != 'PASS' rather than == 'FAIL': a run that produced no report at all
      # reports UNKNOWN, and that is the one outcome nobody should wave through.
      - name: A few flakes are survivable; a collapse is not
        if: steps.report.outputs.status != 'PASS'
        run: |
          set -euo pipefail

          if [ "$STATUS" != "FAIL" ]; then
            echo "::error::the run produced no report to judge (status $STATUS)"
            exit 1
          fi

          echo "pass rate: $PASS_RATE%  ($FAILED failed of $TOTAL)"

          if [ -z "$PASS_RATE" ] || [ "${PASS_RATE%%.*}" -lt 90 ]; then
            echo "::error::pass rate $PASS_RATE% - this is not flakiness"
            exit 1
          fi

          echo "::warning::$FAILED failing, pass rate still $PASS_RATE% - letting it through"
        env:
          STATUS: ${{ steps.report.outputs.status }}
          PASS_RATE: ${{ steps.report.outputs.pass-rate }}
          FAILED: ${{ steps.report.outputs.failed }}
          TOTAL: ${{ steps.report.outputs.total }}

Two details there are deliberate and worth copying. The condition is != 'PASS' rather than == 'FAIL', because a run that produced no report at all reports UNKNOWN and that is the one outcome nobody should wave through. And every value reaches the script through env: rather than being spliced into the command.

Coverage

Reach for this when you already measure coverage with pytest-cov and want it in the report and enforced. Nothing here is special to the action: install pytest-cov, pass --cov, and the plugin reads the coverage the run just measured.

yaml examples/coverage.yml — use when the Test Coverage tab should be populated and floored
# The Test Coverage tab, and a floor under it.
#
# Nothing here is special: install pytest-cov, pass --cov, and the plugin
# reads the coverage the run just measured.
name: tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: prashanth-sams/pytest-html-reporter-action@v1
        with:
          python-version: '3.12'
          extra-packages: pytest-cov
          tests: tests/
          pytest-args: --cov=src --cov-branch
          min-coverage: '80'

The plugin reads whatever measured coverage; it does not measure any itself. Auto-discovery looks for a coverage.json or coverage.xml in the report directory, the repository root and the working directory. A .coverage data file is not discovered — name it with report-coverage-file, and install coverage in that job so it can be read.

There is no freshness checkA coverage.xml restored from a cache before the tests ran is published as this build’s number, silently. Do not cache coverage files alongside the report.

Matrix and sharding

Reach for this when you test across operating systems and Python versions. The action is designed to be run once per matrix cell, and produces one report per cell.

yaml examples/matrix.yml — use when one report per operating system and Python version is what you want
# One report per cell. Artifact names must differ within a run.
name: tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix:
        os: [ubuntu-latest, macos-latest, windows-latest]
        python: ['3.9', '3.11', '3.13']

    steps:
      - uses: actions/checkout@v4

      - uses: prashanth-sams/pytest-html-reporter-action@v1
        with:
          python-version: ${{ matrix.python }}
          tests: tests/
          title: ${{ matrix.os }} py${{ matrix.python }}
          artifact-name: pytest-html-report-${{ matrix.os }}-py${{ matrix.python }}
          build-info: |
            os=${{ matrix.os }}
            python=${{ matrix.python }}

Sharding within a job is a pytest-args concern rather than an action concern. Splitting a suite across jobs produces one report per job, each with its own artifact; the action does not merge them.

yaml Parallel execution inside a single job, with one report
- uses: prashanth-sams/pytest-html-reporter-action@v1
  with:
    tests: tests/
    extra-packages: pytest-xdist
    pytest-args: -n auto -m "not slow"

Permissions

The action needs nothing beyond the default for the basic case. Add what the features you turn on need. Since 2023 the default GITHUB_TOKEN is read-only, so a missing pull-requests: write is the usual reason a comment does not appear.

yaml The two permissions most workflows need
permissions:
  contents: read        # always
  pull-requests: write  # for `comment: 'true'`
PermissionWhere it goesWhy
contents: readThe test jobAlways. Checkout.
pull-requests: writeThe test jobRequired for comment: 'true'. Unavailable to a pull_request run from a fork.
actions: readThe workflow_run comment jobTo download the artifact produced by the test run.
pages: writeThe deploy job onlyFor actions/deploy-pages.
id-token: writeThe deploy job onlyFor actions/deploy-pages.

How it works

The action is a composite: thirteen steps in action.yml, and one stdlib-only Python helper, scripts/phr.py, that does the thinking.

StepWhat it does
Check the inputsValidate before anything expensive happens.
Set up Pythonactions/setup-python, only when python-version is set.
Find the interpreterpython or python3 — settle on one name.
InstallThe plugin, your requirements, extra packages.
Work out whereResolve report-path exactly as the plugin will.
Restore build historyactions/cache/restore, when history is true.
Run pytestPrime, probe, run, capture the exit code.
Save build historyactions/cache/save, when history is true.
Upload the reportactions/upload-artifact.
Upload a Pages artifactactions/upload-pages-artifact, when pages-artifact is true.
Summarise the runoutput.json to outputs, job summary and comment body.
Commentactions/github-script, when comment is true.
Decide the jobGo red, or do not.

The helper is invoked four times.

SubcommandWhat it does
resolveExpands strftime placeholders once, normalises backslashes, applies the plugin’s file-or-folder rule, creates the directory, and emits the path outputs.
primeReadies a restored cache: records the restored build’s start_time, stands in an empty report file so the predecessor joins the archive, quarantines unreadable archived builds, and deletes stale pytest_screenshots.
argsTurns the PHR_* environment into a pytest argument list, filtered against pytest --help, written NUL-separated for bash to read back.
summarizeReads output.json and produces the step outputs, the job summary markdown, the comment body and the threshold verdict.
Only an explicit pass is a passThe final step reads gate-passed back out of the summarize step. true passes and false fails — and anything else, including empty, also fails, because a run nobody checked is not a run that passed.

Why the path is resolved before pytest runs

--html-report is run through strftime, so reports/%H%M/ names a different folder each minute. If the action expanded it separately from the plugin, a run crossing a minute boundary would leave the action looking in a folder the report is not in. So the action expands it once, up front, and hands pytest an already-concrete path — with every remaining percent doubled, because the plugin expands whatever it is given and a surviving % would be read as a directive on the second pass.

The file-or-folder rule is copied from the plugin rather than reinvented, including its sharp edge: the value names a file when it contains .html anywhere, not when it ends with it. When no .html is present, the report is named pytest_html_report.html. On Windows the action normalises backslashes to forward slashes first, because the plugin splits the value on / only.

The resolve step emits report-dir, report-name, report-file, json-path, archive-dir, screenshot-dir and html-report, and warns when the resolved directory is the working directory — because then your whole checkout is what gets uploaded as the artifact.

Why pytest’s flags are probed first

Anyone can pin their own version of the plugin, and the older releases have fewer options. An unknown flag is not a missing feature — argparse aborts the entire run with exit code 4 before a single test is collected, naming a flag the user never typed.

So the action runs pytest --help once and passes only the flags that appear in it. The check requires --html-report to be present as evidence that the probe is real: a probe that did not actually list the plugin’s options would otherwise read as “this plugin supports nothing” and drop every flag. If the probe itself fails, nothing is filtered — passing the flags and letting pytest object beats dropping all of them. Anything left out is announced as a warning naming the flag.

The argument list is assembled in a fixed order: --html-report first and always; then the single-value flags, each included only when its input is non-empty; then the repeated flags (--build-info, --report-link), one per non-blank line; then --report-open; then each line of tests; then pytest-args. It is written NUL-separated to a temp file and read back into a bash array, so no argument can be re-split or globbed on the way, and the assembled command is printed shell-quoted so the log shows exactly what ran.

Why nothing is interpolated into the shell

Every input reaches bash as an environment variable, never as ${{ ... }} spliced into a run: body — that would make pytest-args shell source. tests/test_action_yml.py fails the build if anyone writes one. Test names and failure messages get the same treatment on the way out; see untrusted text.

Troubleshooting

Most of what goes wrong here is one of a handful of things. If the problem is in the report’s contents rather than in getting it built, it probably belongs on the plugin’s side.

No output.json was written

The run finished and the plugin never wrote anything. Three causes, in order of likelihood.

The plugin is not installed in the environment that ran the tests. The action’s install step and your test run must use the same Python. If you set install: 'false' because your requirements.txt pins it, check that the file really lists pytest-html-reporter. The action logs a warning at install time when the import fails — look for it above the pytest output.

pytest failed before any test ran. A collection error, a missing dependency, a bad -k. The pytest log says which.

report-path points somewhere else. Check the report-dir output against where you are looking.

Exit code 4: “a usage error”

pytest refused an argument. This is not a failing test — nothing ran.

The message pytest printed names a flag, and you set an input. Nearly all of them share a name — --archive-since is the archive-since input — with two that do not: --html-report is report-path, and --report-link is report-links. The same message also appears when the value came from your pytest.ini, because the ini keys share their names with the flags too. So check both.

The action quotes the offending line back as an annotation, so you should not have to go digging in the log for it.

“the installed pytest-html-reporter has no --report-steps”

You are on a version of the plugin older than the input you set. The action leaves the flag out rather than failing the run — passing an unknown flag would abort pytest with a usage error before a single test was collected. Fix it by upgrading, or drop the input.

yaml
with:
  plugin-version: '>=0.3.8'
The Trends, Archives and Analytics tabs are empty

They read previous builds, and a fresh runner has none. Turn history on:

yaml
with:
  history: 'true'

Then check three things.

  • report-path must be fixed, not per-run. A path with %Y%m%d in it makes a new folder every run, so every run starts from nothing. Put the date in artifact-name instead. The action warns when it sees this combination.
  • A matrix needs a history-key per cell. The default key already includes the runner OS, but two Python versions writing into one history interleave builds that are not comparable.
  • archive-count: '1' keeps nothing. The plugin reads it as “this build and no others”. '0' deletes the archive entirely. Leave it empty for no limit.

Note that the first run with history on still shows one build. It takes two.

The Test Coverage tab is empty

The plugin reads whatever measured coverage; it does not measure any itself.

  • Install pytest-cov and pass --cov=<your package> in pytest-args, in the same run. That is the whole of it for a single job.
  • Auto-discovery looks for coverage.json or coverage.xml in the report directory, the repository root and the working directory. A .coverage data file is not discovered — name it with report-coverage-file, and install coverage in that job so it can be read.
  • report-coverage-file is final. If the file named there cannot be read, the tab stays blank and the coverage the run measured is not used instead. The action warns when this happens.
  • There is no freshness check. A coverage.xml restored from a cache before the tests ran is published as this build’s number, silently. Do not cache coverage files alongside the report.
The pull request comment never appears
  • Permissions. pull-requests: write on the job. The default token has been read-only since 2023.
  • A fork. A pull_request run from a fork gets a read-only token and cannot comment at all. This is deliberate on GitHub’s part. Use the workflow_run pattern in fork pull requests.
  • The event. comment: 'true' only fires on a pull_request event, unless you pass pr-number explicitly.

The action warns and carries on rather than failing the run, so look for a warning annotation rather than a red step.

The report is enormous

Every archived build costs roughly 5KB of the page. An hourly run with no retention limit reaches several megabytes within a couple of months, and the page gets slow to open.

Set archive-days: '30', or archive-count: '50'. They intersect with archive-since — a build has to satisfy every limit you set to be kept.

report-log-limit, report-attachment-limit and report-step-limit cap what each test contributes. Note that 0 means unlimited for all three, and a negative value is read as 0 — the action rejects a negative rather than letting it silently mean the opposite of what it looks like.

The job went green and no tests ran

That is what fail-on-empty is for, and it is on by default — a run that collects nothing fails. If a cell of your matrix legitimately has no tests, set fail-on-empty: 'false' for it.

The report landed in the wrong folder

report-path is read as a file when it contains .html anywhere, not just at the end. So out.html_v2/report.html makes the plugin treat the whole thing as a bare filename and write into the working directory. Rename the folder. The action warns when it detects this.

Percent signs are strftime placeholders: %p is AM/PM, so ./100%pass/ becomes ./100PMass/. Write %% for a literal percent.

pip refuses to install anything

error: externally-managed-environment. The runner’s Python belongs to its distribution, and pip will not touch it. This does not happen on GitHub-hosted runners; on a self-hosted one, set python-version so actions/setup-python provides an interpreter of its own.

yaml
with:
  python-version: '3.12'
Something else

Open an issue with the workflow YAML, the job log, and the report-dir and status outputs. If the problem is in the report’s contents rather than in getting it built, it probably belongs on the plugin — start with the FAQ and the CLI reference.

Where to go next