Who owns what
3 owners, and 1756 of 1765 tests unclaimed
Owner
Tests
Pass rate
Failing
Flaky
Time
payments-team
7
57.1%
3
0
1.85ms
search-team
1
0.0%
1
0
0.24ms
platform-team
1
100.0%
0
0
0.23ms
Unowned
1756
98.8%
34
22
1m 39s
How much it matters
1 critical test failing
Severity
Tests
Pass rate
Failing
Flaky
Time
blocker
3
100.0%
0
0
0.75ms
critical
2
50.0%
1
0
0.54ms
normal
1
100.0%
0
0
0.26ms
minor
1
100.0%
0
0
0.17ms
trivial
1
0.0%
1
0
0.33ms
catastrophic
1
100.0%
0
0
0.19ms
Unrated
1754
98.7%
35
22
1m 39s
This is the only build on record, so there is nothing yet to compare it
against. The duration panels below are this run's and are already real;
the trends, the flake rates and the movement cards fill themselves in from
the second run onwards, over as many builds as --archive-count
keeps.
Pass rate across builds
share of decided tests that passed
What moved, build to build
fixed, regressed, added, dropped
Where the time goes
tests per duration band, this run
merged 2 shards: 2 tests
shard gw0: 1 test, finished 2024-09-01 18:13:20
shard gw1: 1 test, finished 2024-09-01 18:13:20
PASS 1, FAIL 1
ran on runner-gw0, runner-gw1 over 1666 mins 40 secs
coverage: 2 shards were merged and none of them was asked for coverage. Run the shards under pytest-cov, then combine the data - `coverage combine && coverage json` - and pass the result with --report-coverage-file.
junit: /private/var/folders/qm/7tqk34092qxgcr_t5fysjykm0000gn/T/pytest-of-prashanthsams/pytest-567/popen-gw1/test_the_junit_command_writes_0/results.xml
Captured stdout call
merged 2 shards: 2 tests
shard gw0: 1 test, finished 2024-09-01 18:13:20
shard gw1: 1 test, finished 2024-09-01 18:13:20
PASS 1, FAIL 1
ran on runner-gw0, runner-gw1 over 1666 mins 40 secs
coverage: 2 shards were merged and none of them was asked for coverage. Run the shards under pytest-cov, then combine the data - `coverage combine && coverage json` - and pass the result with --report-coverage-file.
Captured stderr call
pytest-html-reporter: --html-report './my.html.d' has '.html' in it without ending in it, so it would be read as a file name and the report would be written into the current directory; name the folder without '.html' in it, or name the .html file itself
pytest-html-reporter: 1 bundle under /private/var/folders/qm/7tqk34092qxgcr_t5fysjykm0000gn/T/pytest-of-prashanthsams/pytest-567/popen-gw1/test_a_leg_with_a_token_merges0/shards came from another run and was not merged - this run's token is run-7 and they carry: old (run-6)
pytest-html-reporter: clear /private/var/folders/qm/7tqk34092qxgcr_t5fysjykm0000gn/T/pytest-of-prashanthsams/pytest-567/popen-gw1/test_a_leg_with_a_token_merges0/shards between runs, or give the first leg of a run --report-shard-reset, to stop them accumulating
pytest-html-reporter: merged shard gw0: 1 test, finished 2024-09-01 18:13:20
Captured stderr call
pytest-html-reporter: 1 bundle under /private/var/folders/qm/7tqk34092qxgcr_t5fysjykm0000gn/T/pytest-of-prashanthsams/pytest-567/popen-gw1/test_a_folder_holding_only_ano0/shards came from another run and was not merged - this run's token is run-7 and they carry: old (run-6)
pytest-html-reporter: clear /private/var/folders/qm/7tqk34092qxgcr_t5fysjykm0000gn/T/pytest-of-prashanthsams/pytest-567/popen-gw1/test_a_folder_holding_only_ano0/shards between runs, or give the first leg of a run --report-shard-reset, to stop them accumulating
pytest-html-reporter: merged shard gw0: 1 test, finished 2024-09-01 18:13:20
pytest-html-reporter: could not be read as json and was skipped: /private/var/folders/qm/7tqk34092qxgcr_t5fysjykm0000gn/T/pytest-of-prashanthsams/pytest-567/popen-gw1/test_a_file_that_is_not_a_bund0/shards/gw1/records.json (Expecting value: line 1 column 1 (char 0))
Captured stderr call
pytest-html-reporter: --report-junit could not write : [Errno 21] Is a directory: '/Users/prashanthsams/Documents/tmpz9tpzgz4.xml' -> '/Users/prashanthsams/Documents/pytest-html-reporter'
Captured stdout call
merged 2 shards: 7 tests
shard 1-2: 3 tests, finished 2026-09-06 16:22:52
shard 2-2: 4 tests, finished 2026-09-06 16:22:52
PASS 4, FAIL 1, SKIP 1, xFAIL 1
ran on Mac over 0.31 secs
coverage: 2 shards were merged and none of them was asked for coverage. Run the shards under pytest-cov, then combine the data - `coverage combine && coverage json` - and pass the result with --report-coverage-file.
report: /private/var/folders/qm/7tqk34092qxgcr_t5fysjykm0000gn/T/pytest-of-prashanthsams/pytest-567/popen-gw1/test_two_shard_processes_then_0/merged/pytest_html_report.html
Attempt 1FAIL0.97 msgw0
AssertionError: the order was rejected
assert 422 == 201
+ where 422 = <tests.functional.test_attachments.Response object at 0x105ca25d0>.status_code
Attempt 2FAIL0.55 msgw0
AssertionError: the order was rejected
assert 422 == 201
+ where 422 = <tests.functional.test_attachments.Response object at 0x105ca2e90>.status_code
Attempt 3FAIL0.46 msgw0
AssertionError: the order was rejected
assert 422 == 201
+ where 422 = <tests.functional.test_attachments.Response object at 0x1059b3950>.status_code
Attempt 4FAIL0.42 msgw0kept
AssertionError: the order was rejected
assert 422 == 201
+ where 422 = <tests.functional.test_attachments.Response object at 0x105ad0510>.status_code
Attempt 1FAIL0.34 msgw0
AssertionError: deleting a shipped order should have been allowed
assert False
Attempt 2FAIL0.39 msgw0
AssertionError: deleting a shipped order should have been allowed
assert False
Attempt 3FAIL0.32 msgw0
AssertionError: deleting a shipped order should have been allowed
assert False
Attempt 4FAIL0.32 msgw0kept
AssertionError: deleting a shipped order should have been allowed
assert False
Attempt 1FAIL0.57 msgw0
AssertionError: assert 'Saving world' == 'Jesus'
- Jesus
+ Saving world
Attempt 2FAIL0.41 msgw0
AssertionError: assert 'Saving world' == 'Jesus'
- Jesus
+ Saving world
Attempt 3FAIL0.41 msgw0
AssertionError: assert 'Saving world' == 'Jesus'
- Jesus
+ Saving world
Attempt 4FAIL0.31 msgw0kept
AssertionError: assert 'Saving world' == 'Jesus'
- Jesus
+ Saving world
Attempt 1FAIL100 msgw0
AssertionError: card declined by the gateway
Attempt 2FAIL102 msgw0
AssertionError: card declined by the gateway
Attempt 3FAIL105 msgw0
AssertionError: card declined by the gateway
Attempt 4FAIL106 msgw0kept
AssertionError: card declined by the gateway
Attempt 1FAIL1.79 sgw0
AssertionError: assert 'Example Domain' == 'Not the heading'
- Not the heading
+ Example Domain
Attempt 2FAIL582 msgw0
AssertionError: assert 'Example Domain' == 'Not the heading'
- Not the heading
+ Example Domain
Attempt 3FAIL773 msgw0
AssertionError: assert 'Example Domain' == 'Not the heading'
- Not the heading
+ Example Domain
Attempt 4FAIL726 msgw0kept
AssertionError: assert 'Example Domain' == 'Not the heading'
- Not the heading
+ Example Domain
Attempt 1FAIL0.24 msgw0
assert 5 == 6
Attempt 2FAIL0.32 msgw0
assert 5 == 6
Attempt 3FAIL0.28 msgw0
assert 5 == 6
Attempt 4FAIL0.28 msgw0kept
assert 5 == 6
Attempt 1FAIL0.24 msgw0
assert 1 == 2
Attempt 2FAIL0.42 msgw0
assert 1 == 2
Attempt 3FAIL0.33 msgw0
assert 1 == 2
Attempt 4FAIL0.30 msgw0kept
assert 1 == 2
Attempt 1FAIL794 msgw0
AssertionError: assert 'Example Domain' == 'Not the heading'
- Not the heading
+ Example Domain
Attempt 2FAIL824 msgw0
AssertionError: assert 'Example Domain' == 'Not the heading'
- Not the heading
+ Example Domain
Attempt 3FAIL748 msgw0
AssertionError: assert 'Example Domain' == 'Not the heading'
- Not the heading
+ Example Domain
Attempt 4FAIL874 msgw0kept
AssertionError: assert 'Example Domain' == 'Not the heading'
- Not the heading
+ Example Domain
Attempt 1FAIL1.47 sgw0
AssertionError: 'Example Domain' != 'Not the heading'
- Example Domain
+ Not the heading
Attempt 2FAIL1.44 sgw0
AssertionError: 'Example Domain' != 'Not the heading'
- Example Domain
+ Not the heading
Attempt 3FAIL1.25 sgw0
AssertionError: 'Example Domain' != 'Not the heading'
- Example Domain
+ Not the heading
Attempt 4FAIL1.36 sgw0kept
AssertionError: 'Example Domain' != 'Not the heading'
- Example Domain
+ Not the heading
Attempt 1FAIL1.26 sgw0
AssertionError: assert 'Example Domain' == 'Not the heading'
- Not the heading
+ Example Domain
Attempt 2FAIL1.43 sgw0
AssertionError: assert 'Example Domain' == 'Not the heading'
- Not the heading
+ Example Domain
Attempt 3FAIL1.36 sgw0
AssertionError: assert 'Example Domain' == 'Not the heading'
- Not the heading
+ Example Domain
Attempt 4FAIL1.44 sgw0kept
AssertionError: assert 'Example Domain' == 'Not the heading'
- Not the heading
+ Example Domain
Attempt 1FAIL0.23 msgw0
AssertionError: the order completed anyway
Attempt 2FAIL0.33 msgw0
AssertionError: the order completed anyway
Attempt 3FAIL0.33 msgw0
AssertionError: the order completed anyway
Attempt 4FAIL0.34 msgw0kept
AssertionError: the order completed anyway
Attempt 1FAIL0.29 msgw0
AssertionError: still says last year
Attempt 2FAIL0.29 msgw0
AssertionError: still says last year
Attempt 3FAIL0.45 msgw0
AssertionError: still says last year
Attempt 4FAIL0.33 msgw0kept
AssertionError: still says last year
Attempt 1FAIL0.20 msgw0
Exception: fail
Attempt 2FAIL0.34 msgw0
Exception: fail
Attempt 3FAIL0.26 msgw0
Exception: fail
Attempt 4FAIL0.24 msgw0kept
Exception: fail
Attempt 1FAIL0.30 msgw0
Exception: fail
Attempt 2FAIL0.34 msgw0
Exception: fail
Attempt 3FAIL0.26 msgw0
Exception: fail
Attempt 4FAIL0.24 msgw0kept
Exception: fail
Attempt 1FAIL251 msgw0
AssertionError: card declined by the gateway
Attempt 2FAIL254 msgw0
AssertionError: card declined by the gateway
Attempt 3FAIL255 msgw0
AssertionError: card declined by the gateway
Attempt 4FAIL261 msgw0kept
AssertionError: card declined by the gateway
Attempt 1FAIL41 msgw0
AssertionError: gateway timed out
Attempt 2FAIL42 msgw0
AssertionError: gateway timed out
Attempt 3FAIL42 msgw0
AssertionError: gateway timed out
Attempt 4FAIL41 msgw0kept
AssertionError: gateway timed out
Attempt 1FAIL0.23 msgw0
assert 99.5 == 100.0
Attempt 2FAIL0.31 msgw0
assert 99.5 == 100.0
Attempt 3FAIL0.29 msgw0
assert 99.5 == 100.0
Attempt 4FAIL0.34 msgw0kept
assert 99.5 == 100.0
Attempt 1FAIL0.22 msgw0
AssertionError: no results returned
Attempt 2FAIL0.27 msgw0
AssertionError: no results returned
Attempt 3FAIL0.25 msgw0
AssertionError: no results returned
Attempt 4FAIL0.24 msgw0kept
AssertionError: no results returned
Attempt 1FAIL0.23 msgw0
assert False
Attempt 2FAIL0.39 msgw0
assert False
Attempt 3FAIL0.50 msgw0
assert False
Attempt 4FAIL0.28 msgw0kept
assert False
Attempt 1FAIL0.31 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 2FAIL0.44 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 3FAIL0.42 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 4FAIL0.39 msgw0kept
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 1FAIL0.27 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 2FAIL0.36 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 3FAIL0.30 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 4FAIL0.32 msgw0kept
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 1FAIL0.30 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 2FAIL0.31 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 3FAIL0.29 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 4FAIL0.30 msgw0kept
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 1FAIL0.33 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 2FAIL0.35 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 3FAIL0.33 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 4FAIL0.31 msgw0kept
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 1FAIL0.32 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 2FAIL0.39 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 3FAIL0.30 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 4FAIL0.31 msgw0kept
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 1FAIL0.30 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 2FAIL0.31 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 3FAIL0.32 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 4FAIL0.34 msgw0kept
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 1FAIL0.24 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 2FAIL0.33 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 3FAIL0.34 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 4FAIL0.32 msgw0kept
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 1FAIL0.22 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 2FAIL0.31 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 3FAIL0.33 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 4FAIL0.48 msgw0kept
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 1FAIL0.29 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 2FAIL0.33 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 3FAIL0.34 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 4FAIL0.32 msgw0kept
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 1FAIL0.33 msgw0
ValueError: not enough values to unpack (expected 1, got 0)
Attempt 2FAIL0.32 msgw0
ValueError: not enough values to unpack (expected 1, got 0)
Attempt 3FAIL0.31 msgw0
ValueError: not enough values to unpack (expected 1, got 0)
Attempt 4FAIL0.33 msgw0kept
ValueError: not enough values to unpack (expected 1, got 0)
Attempt 1FAIL0.30 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 2FAIL0.42 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 3FAIL0.36 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 4FAIL0.41 msgw0kept
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 1FAIL0.27 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 2FAIL0.34 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 3FAIL0.34 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 4FAIL0.41 msgw0kept
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 1FAIL0.38 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 2FAIL0.48 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 3FAIL0.45 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 4FAIL0.34 msgw0kept
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 1FAIL0.34 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 2FAIL0.39 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 3FAIL0.40 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 4FAIL0.33 msgw0kept
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 1FAIL0.33 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 2FAIL0.38 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 3FAIL0.35 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 4FAIL0.31 msgw0kept
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 1FAIL0.30 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 2FAIL0.38 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 3FAIL0.42 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 4FAIL0.38 msgw0kept
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 1FAIL0.44 msgw0
RuntimeError: Cannot run the event loop while another loop is running
Attempt 2FAIL0.51 msgw0
RuntimeError: Cannot run the event loop while another loop is running
Attempt 3FAIL0.50 msgw0
RuntimeError: Cannot run the event loop while another loop is running
Attempt 4FAIL0.48 msgw0kept
RuntimeError: Cannot run the event loop while another loop is running
Attempt 1FAIL0.28 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 2FAIL0.37 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 3FAIL0.38 msgw0
RuntimeError: asyncio.run() cannot be called from a running event loop
Attempt 4FAIL0.40 msgw0kept
RuntimeError: asyncio.run() cannot be called from a running event loop
Test Steps
What each test did, step by step, and where its time went
Owner
Severity
Nothing matches that search.
No test named a step in this run
The tree above still works — every test has a set up, a body and a
tear down, and each is timed. Naming the pieces inside them is what turns
“this test failed after 4 seconds” into “it failed
charging the card, 3.2 seconds in”.
1Name a step
step is a with block or a decorator, and steps nest by
being called from inside one another — nothing is passed between them.
from pytest_html_reporter importstepdef test_checkout():
withstep("Add to cart", sku="A-12"):
cart.add("A-12")
withstep("Charge the card"):
assert gateway.charge(cart).ok
2Or decorate the helperrecommended
A page object’s methods are already the steps of the test. Decorating
them once names every test that calls them, and the arguments of the call
fill in the {placeholders} of the title.
@step("Log in as {user}")
def login(user, password):
page.fill("#user", user)
page.click("#submit")
3Already writing Gherkin?
Nothing to do. A pytest-bdd scenario is already a list of steps, so
its Given / When / Then land here on their own — each timed, each
carrying what it was called with.
Scenario: Add items to the cart
Given a logged in user
When I add 2 items to the cart
Then the cart shows 2 items
Anything attached while a step is open is filed under that step —
attach_json, attach_api and the rest need no extra argument
to say which one they belong to.
A call that went fine. Attached anyway - a 201 is a baseline worth having.
1 fixture
api
Set up0.12 ms
No steps named here
Test body0.53 ms
No steps named here
Tear down0.11 ms
No steps named here
Description
The case the tab exists for: three attachments explaining one failure.
1 fixture
api
Set up0.17 ms
No steps named here
Test body0.14 ms
No steps named here
Tear down0.11 ms
No steps named here
Description
A call described by hand - no response object anywhere in sight.
1 fixture
api
Set up0.12 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Nothing is attached here; the `api` fixture's teardown does it.
1 fixture
api
Set up0.17 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.10 ms
No steps named here
Description
Any small text file - a payload, a config, a HAR - can be attached.
1 fixture
tmp_path
Set up0.56 ms
No steps named here
Test body0.27 ms
No steps named here
Tear down0.07 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
A test that attaches nothing shows a dash in the Data column.
Set up0.07 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
1 fixture
db
Set up0.13 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
db
Set up0.10 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
mock_data
Set up0.15 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
mock_data
Set up0.17 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.06 ms
No steps named here
FeatureChecking out a basketA shopper buys 1 of an itemfeatures/checkout.feature
2 markers
smokecheckout
1 parameter
count = 1
Set up0.14 ms
No steps named here
Test body59 ms
GivenGiven a logged in shopper23 ms
WhenWhen they add 1 of "A-12" to the basketcount=1, sku=A-1235 ms
ThenThen the basket holds 1 itemscount=10.04 ms
Tear down0.13 ms
No steps named here
FeatureChecking out a basketA shopper buys 3 of an itemfeatures/checkout.feature
2 markers
smokecheckout
1 parameter
count = 3
Set up0.14 ms
No steps named here
Test body59 ms
GivenGiven a logged in shopper25 ms
WhenWhen they add 3 of "A-12" to the basketcount=3, sku=A-1233 ms
ThenThen the basket holds 3 itemscount=30.04 ms
Tear down0.11 ms
No steps named here
FeatureChecking out a basketA declined card names the step that failedfeatures/checkout.feature
3 markers
smokecheckoutdeclined
Set up0.26 ms
No steps named here
Test body106 ms
GivenGiven a logged in shopper25 ms
WhenWhen they add 1 of "DECLINE" to the basketcount=1, sku=DECLINE35 ms
AndAnd they check out45 ms
AssertionError: card declined by the gateway
Tear down0.14 ms
No steps named here
FeatureReading the heading of a pageA wrong heading is photographed on the step that failedui_features/heading.feature
2 markers
uismoke
Set up0.25 ms
No steps named here
Test body714 ms
GivenGiven the example page is open702 ms
WhenWhen the heading is read11 ms
ThenThen it reads "Not the heading"expected=Not the heading, heading=Example Domain0.14 ms
AssertionError: assert 'Example Domain' == 'Not the heading'
- Not the heading
+ Example Domain
Tear down12 ms
No steps named here
1 marker
slow
Set up0.12 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.06 ms
No steps named here
1 marker
fast
Set up0.16 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.06 ms
No steps named here
1 marker
parametrize(a, b)
2 parameters
a = 1b = 1
Set up0.14 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.07 ms
No steps named here
1 marker
parametrize(a, b)
2 parameters
a = 1b = 2
Set up0.18 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.07 ms
No steps named here
1 parameter
browser_name = chromium
1 fixture
page
Set up851 ms
No steps named here
Test body8.84 ms
No steps named here
Tear down2.32 ms
No steps named here
1 parameter
browser_name = chromium
1 fixture
page
Set up854 ms
No steps named here
Test body8.95 ms
No steps named here
Tear down12 ms
No steps named here
Set up0.19 ms
No steps named here
Test body1.78 s
No steps named here
Tear down0.17 ms
No steps named here
Set up0.31 ms
No steps named here
Test body1.35 s
No steps named here
Tear down5.42 ms
No steps named here
1 fixture
driver
Set up1.13 s
No steps named here
Test body8.28 ms
No steps named here
Tear down74 ms
No steps named here
1 fixture
driver
Set up1.33 s
No steps named here
Test body7.20 ms
No steps named here
Tear down105 ms
No steps named here
Description
The one that stops a release. Its Severity row shows two badges: `blocker`, and the module's `normal` struck through behind it. Both markers are on this test and always were; what the row says is which of them the run was rated at.
1 owner
payments-team
Severity
blockernormal
Set up0.17 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.09 ms
No steps named here
Description
Red at a level worth waking up for.
1 owner
payments-team
Severity
criticalnormal
Set up0.20 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.08 ms
No steps named here
Description
Red, and nobody's evening. The panel sorts by the ladder rather than by the numbers, so this stays under `blocker` however much redder it gets - a table that put trivial on top because trivial had more failures would be arguing with the words in it.
Severity
trivialnormal
Set up0.19 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.09 ms
No steps named here
Description
Rated `blocker`. Nothing is nearer than anything else here, so the worse of the two is taken. Reading a blocker down to a minor because of the order two decorators happen to sit in is the one mistake here that hides work.
Severity
blockerminornormal
Set up0.14 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
One level, not two - a shift key must not split the counts in half.
Severity
criticalnormal
Set up0.11 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Empty brackets name no level. It is not a sixth level and it is not `normal`; it is a marker somebody left unfinished, so it stays a plain badge and the module's `normal` is what this test is rated at.
Severity
normal
1 marker
severity
Set up0.12 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.09 ms
No steps named here
Description
Shown and filterable rather than dropped - a filter that silently omits a test is worse than one that shows a typo - but sorted after `trivial` everywhere, because it is far more often a typo than a level somebody meant, and a typo must not outrank a blocker.
Severity
catastrophicnormal
Set up0.10 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
Rated `minor`, from the class - which its tooltip says.
Severity
minornormal
Set up0.09 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
And nearer still wins again: `blocker`, over the class's `minor`.
Severity
blockerminornormal
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.15 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.16 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.06 ms
No steps named here
1 marker
xfail
Set up0.10 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.06 ms
No steps named here
1 marker
xfail
Set up0.14 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
The happy path, as three named steps.
1 marker
smoke
1 fixture
cart
Set up25 ms
Open a session25 ms
Test body221 ms
Log in as amyuser=amy90 ms
Open the login page35 ms
Submit credentials155 ms
Add A-12 to the cartquantity=1, sku=A-1245 ms
Check out85 ms
Total the basket25 ms
Charge the card60 ms
Tear down11 ms
Close the session11 ms
Description
A parametrized case keeps its own row and its own steps.
2 markers
parametrize(quantity)regression
1 parameter
quantity = 2
1 fixture
cart
Set up25 ms
Open a session25 ms
Test body135 ms
Log in as amyuser=amy90 ms
Open the login page35 ms
Submit credentials155 ms
Add B-7 to the cartquantity=2, sku=B-745 ms
Check the basket holds 20.01 ms
Tear down10 ms
Close the session10 ms
Description
A parametrized case keeps its own row and its own steps.
2 markers
parametrize(quantity)regression
1 parameter
quantity = 5
1 fixture
cart
Set up21 ms
Open a session21 ms
Test body133 ms
Log in as amyuser=amy88 ms
Open the login page32 ms
Submit credentials155 ms
Add B-7 to the cartquantity=5, sku=B-745 ms
Check the basket holds 50.01 ms
Tear down11 ms
Close the session11 ms
Description
The point of the tab: the failure names the step, not just the test.
1 marker
regression
1 fixture
cart
Set up25 ms
Open a session24 ms
Test body224 ms
Log in as amyuser=amy90 ms
Open the login page35 ms
Submit credentials155 ms
Add DECLINE to the cartquantity=1, sku=DECLINE45 ms
Check out89 ms
Total the basket25 ms
Charge the card64 ms
AssertionError: card declined by the gateway
Tear down13 ms
Close the session13 ms
Description
Nothing is declared here, and the tab is still not empty.
1 marker
slow
1 fixture
cart
Set up23 ms
Open a session22 ms
Test body55 ms
No steps named here
Tear down11 ms
Close the session11 ms
Description
Gathered calls are siblings under the step that fanned them out. Not a chain three deep, which is what a stack shared by every coroutine on the one thread made of them - and the timings say they overlapped, which is the reason for writing it this way at all.
1 marker
regression
Set up0.13 ms
No steps named here
Test body73 ms
Fetch the catalogue72 ms
Fetch the books sectionsection=books72 ms
Parse the response21 ms
Fetch the music sectionsection=music72 ms
Parse the response21 ms
Fetch the film sectionsection=film72 ms
Parse the response21 ms
Tear down0.10 ms
No steps named here
Description
Timed across the call, so the failure lands on the step that raised. Closing the step on the *building* of the coroutine reported this one as a step that passed in no time at all, and the test failed underneath it.
1 marker
regression
Set up0.21 ms
No steps named here
Test body41 ms
Check out41 ms
Charge the card41 ms
AssertionError: gateway timed out
Tear down0.13 ms
No steps named here
Description
A card the bank declines must not complete the order.
A second owner on top of the module's. Both are shown and both are counted: two owner markers are two teams that put their name on this, and picking one would take the other off the hook.
``@pytest.mark.jira`` with nothing in its brackets names no issue. It stays a plain badge rather than becoming a link to the tracker's front page, which would look like the link had worked.
1 owner
payments-team
1 marker
jira
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
A third team, so the Owner pills have something to choose between.
xfail is the suite working as declared, not a test misbehaving. Counting it as a failure would put every xfail-marked test at the top of the "always failing" list, where nothing is wrong.
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.13 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
An archive from a newer version must not invent failures here.
Set up0.09 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.03 ms
No steps named here
1 fixture
tmp_path
Set up0.40 ms
No steps named here
Test body1.26 ms
No steps named here
Tear down0.08 ms
No steps named here
1 fixture
tmp_path
Set up0.32 ms
No steps named here
Test body0.72 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
The distinction is the whole point of the table. A test that only ever fails is a bug with an owner; sending it to the top of a flakiness list sends somebody hunting a race that is not there.
1 fixture
tmp_path
Set up0.39 ms
No steps named here
Test body0.68 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
One failure is a failure; a standing one is a different conversation.
1 fixture
tmp_path
Set up0.31 ms
No steps named here
Test body0.25 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
The least ambiguous evidence there is: same build, two answers.
1 fixture
tmp_path
Set up0.26 ms
No steps named here
Test body0.23 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
A test skipped between two passes has not changed its mind twice.
1 fixture
tmp_path
Set up0.25 ms
No steps named here
Test body0.53 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Nothing was decided, which is not the same as a rate of zero.
1 fixture
tmp_path
Set up0.25 ms
No steps named here
Test body0.40 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.31 ms
No steps named here
Test body0.50 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.45 ms
No steps named here
Test body0.41 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.36 ms
No steps named here
Test body0.51 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
The name carries the same number, but as text. "1788194287.2" sorts after "1788194287.271306" as a string and before it as a number, which would silently shuffle the history.
1 fixture
tmp_path
Set up0.27 ms
No steps named here
Test body0.42 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
One unreadable file must not cost the whole tab.
1 fixture
tmp_path
Set up0.32 ms
No steps named here
Test body0.48 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
A pipeline that reruns the suite produces same-minute builds. Three identical ticks on an x axis is a chart nobody can read.
1 fixture
tmp_path
Set up0.33 ms
No steps named here
Test body0.57 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
Builds predating the duration key must not be drawn as instant.
1 fixture
tmp_path
Set up0.29 ms
No steps named here
Test body0.34 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
An epoch stamp written where a duration belongs is not a duration. Reports written before the duration came from pytest's own phase timings billed a test that merged a shard bundle mid-run with every second since 1970, and those archives are still on disk. Summed, one of them is the whole "time in tests" tile; drawn, it is the only bar the slowest-tests chart has room for.
1 fixture
tmp_path
Set up0.27 ms
No steps named here
Test body0.25 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Minutes stop being a unit somewhere, and "608889402m 03s" is past it.
Set up0.08 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.08 ms
No steps named here
1 fixture
tmp_path
Set up0.33 ms
No steps named here
Test body0.52 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.29 ms
No steps named here
Test body0.23 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.25 ms
No steps named here
Test body0.26 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
An empty list, not seven zero bars claiming every test was instant.
1 fixture
tmp_path
Set up0.24 ms
No steps named here
Test body0.21 ms
No steps named here
Tear down0.04 ms
No steps named here
1 fixture
tmp_path
Set up0.25 ms
No steps named here
Test body0.49 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Both pass half the time; only one of them tells you anything. A test that alternates has the same pass rate as one the team already knows is broken, and is the more expensive of the two to live with.
1 fixture
tmp_path
Set up0.35 ms
No steps named here
Test body1.03 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
Which of the two a message carries is down to how the traceback was rendered, not to what went wrong.
Set up0.12 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
pytest prints these with no type name at all, and they are far too common a failure to leave in the unclassified pile.
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
A failure is stored as pytest's E-lines with the marker taken off; an error is stored as the traceback exactly as printed.
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
A chained failure prints the original traceback first and the exception that actually came out last - which is the one pytest itself reports.
Set up0.09 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
An assertion prints what it compared, and that can be anything at all - including the text of somebody else's exception.
Set up0.10 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Saying more than the message supports is how a report stops being believed: a fixture that could not be found names no exception.
Set up0.09 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
A pass carries no message, and an expected failure is an outcome the suite asked for rather than one anybody is triaging.
1 fixture
tmp_path
Set up0.38 ms
No steps named here
Test body0.37 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
It failed for a reason worth grouping, and the reason is in the same field - the tab reads ERROR with FAIL everywhere else too.
1 fixture
tmp_path
Set up0.41 ms
No steps named here
Test body0.32 ms
No steps named here
Tear down0.06 ms
No steps named here
1 fixture
tmp_path
Set up0.38 ms
No steps named here
Test body0.30 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
A list headed by "Unclassified: 3" is a list that has answered nothing.
1 fixture
tmp_path
Set up0.36 ms
No steps named here
Test body0.29 ms
No steps named here
Tear down0.06 ms
No steps named here
1 fixture
tmp_path
Set up0.26 ms
No steps named here
Test body0.35 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Which is not the same as a group that has not moved, and reads differently on the page.
1 fixture
tmp_path
Set up0.26 ms
No steps named here
Test body0.20 ms
No steps named here
Tear down0.04 ms
No steps named here
1 fixture
tmp_path
Set up0.29 ms
No steps named here
Test body0.27 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.27 ms
No steps named here
Test body0.23 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Naming the first of twelve one-offs reads as a lead. That a run failed twelve different ways is the finding.
1 fixture
tmp_path
Set up0.35 ms
No steps named here
Test body0.23 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.37 ms
No steps named here
Test body0.31 ms
No steps named here
Tear down0.06 ms
No steps named here
1 fixture
tmp_path
Set up0.29 ms
No steps named here
Test body1.02 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.29 ms
No steps named here
Test body0.69 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Drawing an all-skipped build as 0% invents a cliff that never happened.
1 fixture
tmp_path
Set up0.30 ms
No steps named here
Test body0.97 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.35 ms
No steps named here
Test body0.76 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
The slowest-tests names ride on an attribute, escaped, and are parsed back - so a parametrized case can be called anything at all without landing anywhere the browser is willing to run it.
1 fixture
tmp_path
Set up0.29 ms
No steps named here
Test body0.49 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
The page is assembled by substituting %(name)% - a test named like one would otherwise be replaced instead of shown.
1 fixture
tmp_path
Set up0.34 ms
No steps named here
Test body0.57 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
The table is sortable, but what it opens on should already be the list to work through.
1 fixture
tmp_path
Set up0.44 ms
No steps named here
Test body1.01 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
The strip is markup with no text in it, so without an explicit order every row ties and clicking the History header does nothing.
1 fixture
tmp_path
Set up0.32 ms
No steps named here
Test body0.85 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
"9 TimeoutException" says what broke; the names say whether it is one page object nine tests go through or nine unrelated waits.
1 fixture
tmp_path
Set up0.30 ms
No steps named here
Test body0.66 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Every other longitudinal panel is blank until there are two builds. This one reads the current build alone, and a first run that is red is exactly when somebody wants it.
1 fixture
tmp_path
Set up0.28 ms
No steps named here
Test body0.57 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
A card standing empty over a passing run is a card that gets ignored when it does have something to say.
1 fixture
tmp_path
Set up0.32 ms
No steps named here
Test body0.69 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
The names listed under each exception are the same arbitrary text as everywhere else - a parametrized case can be called anything at all.
1 fixture
tmp_path
Set up0.39 ms
No steps named here
Test body0.74 ms
No steps named here
Tear down0.09 ms
No steps named here
Description
The overlay shows the card's own items with the hiding taken off, so what a card lists and what opening it shows cannot drift apart - and no test name is written into the page twice to make that work.
1 fixture
tmp_path
Set up0.39 ms
No steps named here
Test body1.10 ms
No steps named here
Tear down0.06 ms
No steps named here
1 fixture
tmp_path
Set up0.34 ms
No steps named here
Test body0.82 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
The four cards under the charts truncate the same list the same way.
1 fixture
tmp_path
Set up0.31 ms
No steps named here
Test body0.97 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.35 ms
No steps named here
Test body0.62 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
The tail of that list is one-offs, and it is counted rather than dropped: a run whose failures are all different is itself the finding.
1 fixture
tmp_path
Set up0.42 ms
No steps named here
Test body1.41 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
One dialog serves every "and N more" on the tab: it copies the card's own items across, so it has to search and scroll rather than assume the list is short.
1 fixture
tmp_path
Set up0.37 ms
No steps named here
Test body1.45 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
No output.json is not a crash - it is a report that was never written.
1 fixture
tmp_path
Set up0.34 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.06 ms
No steps named here
1 fixture
tmp_path
Set up0.31 ms
No steps named here
Test body0.41 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Picking one would take a team off the hook for a test they signed.
1 fixture
tmp_path
Set up0.30 ms
No steps named here
Test body0.28 ms
No steps named here
Tear down0.08 ms
No steps named here
1 fixture
tmp_path
Set up0.31 ms
No steps named here
Test body0.27 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
It is not a team, and reading it among them invites somebody to go and find out who Unowned is.
1 fixture
tmp_path
Set up0.29 ms
No steps named here
Test body0.33 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.36 ms
No steps named here
Test body0.24 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
A test that moved teams should page the team that has it today.
1 fixture
tmp_path
Set up0.33 ms
No steps named here
Test body0.34 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Older archives simply have no key, and inventing one would be worse.
1 fixture
tmp_path
Set up0.28 ms
No steps named here
Test body0.21 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Leaving a deleted test in makes a team's numbers unfixable.
1 fixture
tmp_path
Set up0.29 ms
No steps named here
Test body0.32 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
One test with two hundred runs must not decide a team's number.
1 fixture
tmp_path
Set up0.28 ms
No steps named here
Test body0.31 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Against the whole suite every bar would be a stub.
1 fixture
tmp_path
Set up0.29 ms
No steps named here
Test body0.24 ms
No steps named here
Tear down0.04 ms
No steps named here
1 fixture
tmp_path
Set up0.35 ms
No steps named here
Test body0.27 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.43 ms
No steps named here
Test body0.66 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
A table of one row called Unowned is a table saying nothing.
1 fixture
tmp_path
Set up0.34 ms
No steps named here
Test body0.64 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.30 ms
No steps named here
Test body0.69 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
The placeholders and the ConfigVars behind them, kept in step.
Set up0.07 ms
No steps named here
Test body0.51 ms
No steps named here
Tear down0.04 ms
No steps named here
1 fixture
tmp_path
Set up0.33 ms
No steps named here
Test body0.34 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Unlike an owner. record_severity has already picked between the markers.
1 fixture
tmp_path
Set up0.34 ms
No steps named here
Test body0.26 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
A table that put trivial over blocker would argue with the words in it.
1 fixture
tmp_path
Set up0.42 ms
No steps named here
Test body0.35 ms
No steps named here
Tear down0.06 ms
No steps named here
1 fixture
tmp_path
Set up0.37 ms
No steps named here
Test body0.29 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
A typo is not a sixth level and must not outrank blocker.
1 fixture
tmp_path
Set up0.36 ms
No steps named here
Test body0.27 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
A test raised to blocker this month is a blocker, not an average.
1 fixture
tmp_path
Set up0.30 ms
No steps named here
Test body0.34 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.35 ms
No steps named here
Test body0.26 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.35 ms
No steps named here
Test body0.31 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.33 ms
No steps named here
Test body0.26 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.50 ms
No steps named here
Test body0.35 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
A table whose only row is Unrated is a table saying nothing.
1 fixture
tmp_path
Set up0.49 ms
No steps named here
Test body0.65 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.38 ms
No steps named here
Test body0.68 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.44 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.12 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.13 ms
No steps named here
Test body0.18 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body2.12 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.12 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
1 fixture
tmp_path
Set up0.49 ms
No steps named here
Test body0.22 ms
No steps named here
Tear down0.06 ms
No steps named here
1 fixture
tmp_path
Set up0.40 ms
No steps named here
Test body0.18 ms
No steps named here
Tear down0.06 ms
No steps named here
1 fixture
tmp_path
Set up0.32 ms
No steps named here
Test body0.28 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.30 ms
No steps named here
Test body0.28 ms
No steps named here
Tear down0.06 ms
No steps named here
1 fixture
tmp_path
Set up0.38 ms
No steps named here
Test body0.24 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.50 ms
No steps named here
Test body0.27 ms
No steps named here
Tear down0.06 ms
No steps named here
1 fixture
tmp_path
Set up0.38 ms
No steps named here
Test body0.28 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.31 ms
No steps named here
Test body0.35 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.31 ms
No steps named here
Test body0.23 ms
No steps named here
Tear down0.04 ms
No steps named here
1 fixture
tmp_path
Set up0.30 ms
No steps named here
Test body0.55 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.49 ms
No steps named here
Test body0.55 ms
No steps named here
Tear down0.06 ms
No steps named here
1 fixture
tmp_path
Set up0.40 ms
No steps named here
Test body0.38 ms
No steps named here
Tear down0.06 ms
No steps named here
1 fixture
tmp_path
Set up0.33 ms
No steps named here
Test body0.57 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.34 ms
No steps named here
Test body0.40 ms
No steps named here
Tear down0.11 ms
No steps named here
1 fixture
tmp_path
Set up0.36 ms
No steps named here
Test body0.50 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.31 ms
No steps named here
Test body0.31 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.31 ms
No steps named here
Test body0.30 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.31 ms
No steps named here
Test body0.13 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.49 ms
No steps named here
Test body0.10 ms
No steps named here
Tear down0.07 ms
No steps named here
Set up0.25 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.09 ms
No steps named here
Set up0.21 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.07 ms
No steps named here
Set up0.18 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.08 ms
No steps named here
Set up0.19 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.08 ms
No steps named here
Set up0.18 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.07 ms
No steps named here
Set up0.21 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.08 ms
No steps named here
Set up0.20 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.07 ms
No steps named here
Set up0.31 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.11 ms
No steps named here
Set up0.20 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.07 ms
No steps named here
Set up0.16 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.12 ms
No steps named here
Set up0.14 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.26 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.08 ms
No steps named here
Set up0.30 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.07 ms
No steps named here
Set up0.22 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.07 ms
No steps named here
Set up0.21 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.07 ms
No steps named here
Set up0.20 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.07 ms
No steps named here
Set up0.22 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.11 ms
No steps named here
Set up0.23 ms
No steps named here
Test body0.12 ms
No steps named here
Tear down0.13 ms
No steps named here
Set up0.21 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.15 ms
No steps named here
Set up0.16 ms
No steps named here
Test body2.10 ms
No steps named here
Tear down0.09 ms
No steps named here
Set up0.12 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Not every client hands over something with items().
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.18 ms
No steps named here
Set up0.37 ms
No steps named here
Test body0.19 ms
No steps named here
Tear down0.12 ms
No steps named here
Set up0.90 ms
No steps named here
Test body0.47 ms
No steps named here
Tear down0.08 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.09 ms
No steps named here
Tear down0.17 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.15 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.29 ms
No steps named here
Test body0.17 ms
No steps named here
Tear down0.07 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.73 ms
No steps named here
Test body0.14 ms
No steps named here
Tear down0.37 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.21 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.12 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Callers that never need the warnings back just pass keywords.
Set up0.08 ms
No steps named here
Test body0.17 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
A merge leg's options already carry the matrix's clock and its shards; silently letting a keyword overwrite one drops the shards on the floor.
Set up0.11 ms
No steps named here
Test body0.18 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
The phases are whole milliseconds from pytest's own report.
Set up0.08 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.17 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.03 ms
No steps named here
Description
A raw one makes the document unparseable, and a CI collector that cannot parse the xml reports the build as having run no tests at all.
Set up0.07 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.31 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
1 fixture
tmp_path
Set up0.49 ms
No steps named here
Test body416 ms
No steps named here
Tear down0.21 ms
No steps named here
1 fixture
tmp_path
Set up0.51 ms
No steps named here
Test body395 ms
No steps named here
Tear down0.19 ms
No steps named here
Description
A GET has no request body, and an empty part earns no tab of its own.
1 fixture
tmp_path
Set up0.50 ms
No steps named here
Test body402 ms
No steps named here
Tear down0.20 ms
No steps named here
1 fixture
tmp_path
Set up0.52 ms
No steps named here
Test body363 ms
No steps named here
Tear down0.19 ms
No steps named here
Description
The report is a build artifact; people paste them into tickets.
1 fixture
tmp_path
Set up0.45 ms
No steps named here
Test body369 ms
No steps named here
Tear down0.20 ms
No steps named here
1 fixture
tmp_path
Set up0.49 ms
No steps named here
Test body356 ms
No steps named here
Tear down0.18 ms
No steps named here
Description
An API response is far likelier to carry one of these than a log line.
1 fixture
tmp_path
Set up0.44 ms
No steps named here
Test body369 ms
No steps named here
Tear down0.23 ms
No steps named here
1 fixture
tmp_path
Set up0.48 ms
No steps named here
Test body383 ms
No steps named here
Tear down0.21 ms
No steps named here
1 fixture
tmp_path
Set up0.54 ms
No steps named here
Test body376 ms
No steps named here
Tear down0.20 ms
No steps named here
1 fixture
tmp_path
Set up0.45 ms
No steps named here
Test body372 ms
No steps named here
Tear down0.20 ms
No steps named here
1 fixture
tmp_path
Set up0.53 ms
No steps named here
Test body364 ms
No steps named here
Tear down0.18 ms
No steps named here
Description
The bug screenshots had: an unclaimed payload illustrating a test that never produced it.
1 fixture
tmp_path
Set up0.47 ms
No steps named here
Test body364 ms
No steps named here
Tear down0.20 ms
No steps named here
Description
The recipe most people reach for: capture on failure from an autouse fixture, which runs after the test body is done.
1 fixture
tmp_path
Set up0.53 ms
No steps named here
Test body355 ms
No steps named here
Tear down0.19 ms
No steps named here
Description
An empty tab is the only moment anyone reads setup instructions.
1 fixture
tmp_path
Set up0.49 ms
No steps named here
Test body377 ms
No steps named here
Tear down0.17 ms
No steps named here
1 marker
skipif(reason=needs pytest-rerunfailures)
1 fixture
tmp_path
Set up0.47 ms
No steps named here
Test body372 ms
No steps named here
Tear down0.23 ms
No steps named here
1 fixture
tmp_path
Set up0.48 ms
No steps named here
Test body666 ms
No steps named here
Tear down0.21 ms
No steps named here
1 marker
parametrize(name)
1 parameter
name = Authorization
Set up0.25 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.08 ms
No steps named here
1 marker
parametrize(name)
1 parameter
name = proxy-authorization
Set up0.13 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(name)
1 parameter
name = Cookie
Set up0.15 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.06 ms
No steps named here
1 marker
parametrize(name)
1 parameter
name = Set-Cookie
Set up0.12 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(name)
1 parameter
name = X-Api-Key
Set up0.10 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(name)
1 parameter
name = api_key
Set up0.12 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
1 marker
parametrize(name)
1 parameter
name = access_token
Set up0.10 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(name)
1 parameter
name = refresh_token
Set up0.15 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.07 ms
No steps named here
1 marker
parametrize(name)
1 parameter
name = password
Set up0.22 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(name)
1 parameter
name = client_secret
Set up0.12 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(name)
1 parameter
name = Content-Type
Set up0.11 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(name)
1 parameter
name = Accept
Set up0.10 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(name)
1 parameter
name = User-Agent
Set up0.10 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(name)
1 parameter
name = X-Request-Id
Set up0.11 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(name)
1 parameter
name = Content-Length
Set up0.17 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.16 ms
No steps named here
Test body0.09 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.13 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
The token in a login response is the one people attach while debugging.
Set up0.11 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
It would otherwise land in the title, the meta strip and the curl line.
Set up0.13 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
{"name": ..., "value": ...} is how a HAR - and most specs - write one.
Set up0.09 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.33 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
`123` parses as JSON, but re-serialising it says nothing at all.
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Half a response is exactly what you want to see when a call is cut off.
An unescaped quote makes the line something the shell will not run.
Set up0.09 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
1 marker
parametrize(count,expected)
2 parameters
count = 12expected = 12 B
Set up0.41 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.07 ms
No steps named here
1 marker
parametrize(count,expected)
2 parameters
count = 1536expected = 1.5 KB
Set up0.15 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.06 ms
No steps named here
1 marker
parametrize(count,expected)
2 parameters
count = 2097152expected = 2.0 MB
Set up0.18 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.07 ms
No steps named here
1 marker
parametrize(code,expected)
2 parameters
code = 200expected = 2xx
Set up0.20 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.07 ms
No steps named here
1 marker
parametrize(code,expected)
2 parameters
code = 301expected = 3xx
Set up0.14 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(code,expected)
2 parameters
code = 404expected = 4xx
Set up0.34 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(code,expected)
2 parameters
code = 503expected = 5xx
Set up0.17 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(code,expected)
2 parameters
code = Noneexpected =
Set up0.12 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(code,expected)
2 parameters
code = expected =
Set up0.12 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(code,expected)
2 parameters
code = 999expected =
Set up0.12 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(code,expected)
2 parameters
code = oopsexpected =
Set up0.12 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
The opposite of a log: a payload puts its status and error at the top.
Set up0.09 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
A parametrised test name carries brackets and slashes into the download.
Set up0.10 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.12 ms
No steps named here
Description
An empty payload would put a rail entry there with nothing behind it.
Set up0.12 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.09 ms
No steps named here
Tear down0.04 ms
No steps named here
1 fixture
tmp_path
Set up0.46 ms
No steps named here
Test body0.36 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Of everything you can attach, a saved payload is the likeliest to hold one.
1 fixture
tmp_path
Set up0.46 ms
No steps named here
Test body0.27 ms
No steps named here
Tear down0.06 ms
No steps named here
1 fixture
tmp_path
Set up0.39 ms
No steps named here
Test body0.31 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.29 ms
No steps named here
Test body0.20 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
Nothing to key a redaction off, and mangling a config would be worse.
1 fixture
tmp_path
Set up0.34 ms
No steps named here
Test body0.27 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.15 ms
No steps named here
Test body0.18 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.16 ms
No steps named here
Test body0.11 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
The host is the same for every call in most suites; the path is not.
Set up0.11 ms
No steps named here
Test body0.10 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
httpx says reason_phrase and content where requests says reason and body.
Set up0.08 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
A streamed response raises on .text; the attachment loses that one field.
Set up0.08 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
A proxy in front of the client is the reason the override exists.
Set up0.08 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Not through its own screenshot(), which hands back a path.
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
There is nowhere here to await one, and an unawaited coroutine would be reported against the test as a warning of its own.
1 fixture
recwarn
Set up0.24 ms
No steps named here
Test body0.10 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
A Mock answers every call there is, so it would be photographed on the strength of a method it does not have - and calling it records a call the test may be asserting on.
Set up0.08 ms
No steps named here
Test body2.68 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
The whole point: a suite that has never heard of this package. No conftest, no attach, no fixture of its own - just a browser the test was holding when it failed.
1 fixture
tmp_path
Set up0.37 ms
No steps named here
Test body366 ms
No steps named here
Tear down0.16 ms
No steps named here
Description
Photographing a green run is a lot of pictures of pages that were fine, and every one of them is a round trip to the browser.
1 fixture
tmp_path
Set up0.46 ms
No steps named here
Test body373 ms
No steps named here
Tear down0.19 ms
No steps named here
1 fixture
tmp_path
Set up0.49 ms
No steps named here
Test body347 ms
No steps named here
Tear down0.25 ms
No steps named here
1 fixture
tmp_path
Set up0.51 ms
No steps named here
Test body377 ms
No steps named here
Tear down0.18 ms
No steps named here
Description
--report-screenshots is about the pictures nobody asked for. One handed to attach() was asked for, and is kept whatever that option says.
1 fixture
tmp_path
Set up0.47 ms
No steps named here
Test body376 ms
No steps named here
Tear down0.19 ms
No steps named here
Description
A suite that already calls attach() from a hook of its own would otherwise get the same page twice, once from each of us.
1 fixture
tmp_path
Set up0.51 ms
No steps named here
Test body366 ms
No steps named here
Tear down0.17 ms
No steps named here
Description
unittest puts the browser on ``self`` in setUp rather than in a fixture, so there is no funcarg to find it in.
1 fixture
tmp_path
Set up0.45 ms
No steps named here
Test body390 ms
No steps named here
Tear down0.18 ms
No steps named here
Description
The test never ran, so nothing it could have written would have helped. A login fixture that fails is the ordinary shape of this, and the page the browser was left sitting on is the whole answer to why.
1 fixture
tmp_path
Set up0.47 ms
No steps named here
Test body424 ms
No steps named here
Tear down0.19 ms
No steps named here
Description
The fixture name is a hint, not the test. What makes it a browser is that it can hand over a picture.
1 fixture
tmp_path
Set up0.54 ms
No steps named here
Test body377 ms
No steps named here
Tear down0.17 ms
No steps named here
Description
Asking for the page, the context and the browser at once is the ordinary way to write a Playwright test, and all three lead to the same picture.
1 fixture
tmp_path
Set up0.49 ms
No steps named here
Test body365 ms
No steps named here
Tear down0.16 ms
No steps named here
Description
A test driving two of them is comparing them, and one picture of the pair says nothing about which one went wrong.
1 fixture
tmp_path
Set up0.47 ms
No steps named here
Test body375 ms
No steps named here
Tear down0.19 ms
No steps named here
Description
Only there. On the ordinary single-picture row the tooltip would say what the row already says.
1 fixture
tmp_path
Set up0.52 ms
No steps named here
Test body413 ms
No steps named here
Tear down0.18 ms
No steps named here
1 fixture
tmp_path
Set up0.52 ms
No steps named here
Test body369 ms
No steps named here
Tear down0.16 ms
No steps named here
Description
The report has no picture of that one. It still has everything else.
1 fixture
tmp_path
Set up0.48 ms
No steps named here
Test body361 ms
No steps named here
Tear down0.17 ms
No steps named here
Set up0.22 ms
No steps named here
Test body0.09 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
pytest-bdd renamed these across majors, and the report wants either.
Set up0.10 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
A step object is somebody else's; reading it must not fail the test.
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
'' is what the badge renders as nothing; 'None' would be printed.
Set up0.09 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
1 marker
parametrize(keyword)
1 parameter
keyword = Given
Set up0.11 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(keyword)
1 parameter
keyword = WHEN
Set up0.16 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(keyword)
1 parameter
keyword = then
Set up0.12 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(keyword)
1 parameter
keyword = And
Set up0.15 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.06 ms
No steps named here
1 marker
parametrize(keyword)
1 parameter
keyword = But
Set up0.15 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
A dialect spelling its keyword differently still gets a real badge.
Set up0.10 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.41 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Placeholders are filled in before the hook fires, so 2 not <count>.
Set up0.11 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
A target_fixture cart is plumbing; printing it beside the step is noise.
Set up0.10 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
1 marker
parametrize(value)
1 parameter
value = None
Set up0.11 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(value)
1 parameter
value = []
Set up0.11 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(value)
1 parameter
value = not a dict
Set up0.17 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.06 ms
No steps named here
1 marker
parametrize(value)
1 parameter
value = 7
Set up0.19 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.16 ms
No steps named here
Test body0.15 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.21 ms
No steps named here
Test body0.19 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
pytest-bdd never calls after_step for a failure, so this is its close.
Set up0.09 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.16 ms
No steps named here
Description
Nothing opened this one, so the tab only names it if this does both.
Set up0.12 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.12 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
An absolute path says nothing useful in a published report.
Set up0.11 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.21 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
A plain pytest test must not be reported as part of somebody's feature.
Set up0.15 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.14 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
The whole of the defence, and it fails loudly rather than subtly. Without optionalhook, pytest does not warn and does not skip the hook - it refuses to start at all, with `unknown hook 'pytest_bdd_after_step'`, for every user who has not installed pytest-bdd. Which is most of them.
Set up0.31 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.04 ms
No steps named here
1 fixture
tmp_path
Set up0.43 ms
No steps named here
Test body425 ms
No steps named here
Tear down0.20 ms
No steps named here
1 fixture
tmp_path
Set up0.57 ms
No steps named here
Test body417 ms
No steps named here
Tear down0.19 ms
No steps named here
1 fixture
tmp_path
Set up0.49 ms
No steps named here
Test body386 ms
No steps named here
Tear down0.16 ms
No steps named here
1 fixture
tmp_path
Set up0.57 ms
No steps named here
Test body396 ms
No steps named here
Tear down0.20 ms
No steps named here
1 fixture
tmp_path
Set up0.60 ms
No steps named here
Test body451 ms
No steps named here
Tear down0.24 ms
No steps named here
1 fixture
tmp_path
Set up0.57 ms
No steps named here
Test body395 ms
No steps named here
Tear down0.17 ms
No steps named here
1 fixture
tmp_path
Set up0.50 ms
No steps named here
Test body386 ms
No steps named here
Tear down0.16 ms
No steps named here
1 fixture
tmp_path
Set up0.47 ms
No steps named here
Test body439 ms
No steps named here
Tear down0.21 ms
No steps named here
1 fixture
tmp_path
Set up0.55 ms
No steps named here
Test body395 ms
No steps named here
Tear down0.18 ms
No steps named here
1 fixture
tmp_path
Set up0.48 ms
No steps named here
Test body383 ms
No steps named here
Tear down0.17 ms
No steps named here
1 fixture
tmp_path
Set up0.50 ms
No steps named here
Test body415 ms
No steps named here
Tear down0.24 ms
No steps named here
1 fixture
tmp_path
Set up0.66 ms
No steps named here
Test body413 ms
No steps named here
Tear down0.18 ms
No steps named here
1 fixture
tmp_path
Set up0.50 ms
No steps named here
Test body0.52 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
One of the twenty other files sitting in an artifact folder.
1 fixture
tmp_path
Set up0.30 ms
No steps named here
Test body0.17 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.29 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
1 fixture
tmp_path
Set up0.28 ms
No steps named here
Test body0.16 ms
No steps named here
Tear down0.04 ms
No steps named here
1 fixture
tmp_path
Set up0.58 ms
No steps named here
Test body0.23 ms
No steps named here
Tear down0.07 ms
No steps named here
1 fixture
tmp_path
Set up0.33 ms
No steps named here
Test body0.17 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Read hopefully, it is a build whose numbers are quietly incomplete.
1 fixture
tmp_path
Set up0.30 ms
No steps named here
Test body0.25 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Better than refusing: an unreadable version is not a version from the future, and the upgrades walk it forward from the start.
1 fixture
tmp_path
Set up1.73 ms
No steps named here
Test body0.85 ms
No steps named here
Tear down0.13 ms
No steps named here
1 fixture
tmp_path
Set up0.55 ms
No steps named here
Test body0.38 ms
No steps named here
Tear down0.08 ms
No steps named here
Description
An id is what every ordering and every screenshot path is keyed on, so two anonymous bundles must not collapse into one.
Set up0.14 ms
No steps named here
Test body0.11 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
A leg sanitises its own id before it writes; a bundle is a file that arrived from CI and was never asked to.
Set up0.10 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.08 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.09 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.31 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Kept whole rather than picked apart, so a newer field is not lost.
Set up0.12 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
A bundle written before a key existed must answer, not raise from inside a comprehension.
Set up0.23 ms
No steps named here
Test body0.11 ms
No steps named here
Tear down0.07 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
1 fixture
tmp_path
Set up0.45 ms
No steps named here
Test body0.51 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
A CI step that unpacks one artifact knows where it put it.
1 fixture
tmp_path
Set up0.37 ms
No steps named here
Test body0.26 ms
No steps named here
Tear down0.06 ms
No steps named here
1 fixture
tmp_path
Set up0.50 ms
No steps named here
Test body0.30 ms
No steps named here
Tear down0.06 ms
No steps named here
1 fixture
tmp_path
Set up0.41 ms
No steps named here
Test body1.45 ms
No steps named here
Tear down0.09 ms
No steps named here
1 fixture
tmp_path
Set up0.64 ms
No steps named here
Test body0.15 ms
No steps named here
Tear down0.06 ms
No steps named here
1 fixture
tmp_path
Set up0.57 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.06 ms
No steps named here
2 fixtures
tmp_pathbundles
Set up1.63 ms
No steps named here
Test body212 ms
No steps named here
Tear down0.18 ms
No steps named here
Description
output.json is read back by the next build's archive rail, which does data['status'] with no guard - a merged build that wrote none would take the following run's whole report down.
2 fixtures
tmp_pathbundles
Set up1.32 ms
No steps named here
Test body205 ms
No steps named here
Tear down0.17 ms
No steps named here
Description
--exit-code is what lets a merge stand in for the run in CI.
2 fixtures
tmp_pathbundles
Set up1.22 ms
No steps named here
Test body217 ms
No steps named here
Tear down0.28 ms
No steps named here
1 fixture
tmp_path
Set up0.60 ms
No steps named here
Test body245 ms
No steps named here
Tear down0.23 ms
No steps named here
2 fixtures
tmp_pathbundles
Set up1.43 ms
No steps named here
Test body107 ms
No steps named here
Tear down0.19 ms
No steps named here
Description
The usual cause is a CI step that unpacked the artifacts one directory deeper than the merge was told.
2 fixtures
tmp_pathcapsys
Set up0.60 ms
No steps named here
Test body1.96 ms
No steps named here
Tear down0.08 ms
No steps named here
3 fixtures
tmp_pathbundlescapsys
Set up1.46 ms
No steps named here
Test body1.55 ms
No steps named here
Tear down0.08 ms
No steps named here
Description
-q asks for a quiet terminal, not for a merge that stops noticing things: the warnings are still collected and --strict still reads them.
2 fixtures
tmp_pathcapsys
Set up0.47 ms
No steps named here
Test body2.36 ms
No steps named here
Tear down0.09 ms
No steps named here
2 fixtures
tmp_pathcapsys
Set up0.42 ms
No steps named here
Test body1.42 ms
No steps named here
Tear down0.09 ms
No steps named here
2 fixtures
tmp_pathcapsys
Set up0.53 ms
No steps named here
Test body1.42 ms
No steps named here
Tear down0.08 ms
No steps named here
Description
A pipeline that publishes into GitLab or Azure and keeps no HTML at all should not have to write a build it will throw away.
2 fixtures
tmp_pathbundles
Set up1.02 ms
No steps named here
Test body2.07 ms
No steps named here
Tear down0.07 ms
No steps named here
2 fixtures
tmp_pathcapsys
Set up0.39 ms
No steps named here
Test body0.78 ms
No steps named here
Tear down0.08 ms
No steps named here
2 fixtures
tmp_pathbundles
Set up1.27 ms
No steps named here
Test body1.17 ms
No steps named here
Tear down0.09 ms
No steps named here
3 fixtures
tmp_pathbundlescapsys
Set up1.31 ms
No steps named here
Test body1.00 ms
No steps named here
Tear down0.08 ms
No steps named here
Description
The notes go to stderr precisely so this stays a document.
3 fixtures
tmp_pathbundlescapsys
Set up1.06 ms
No steps named here
Test body0.96 ms
No steps named here
Tear down0.07 ms
No steps named here
2 fixtures
tmp_pathcapsys
Set up0.39 ms
No steps named here
Test body0.72 ms
No steps named here
Tear down0.07 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
'.html' in it without ending in it is read as a file name, and the report lands in the current directory under the folder's name.
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Otherwise it reaches os.makedirs at the end and comes out as a FileExistsError traceback over a merge that had otherwise succeeded.
1 fixture
tmp_path
Set up0.39 ms
No steps named here
Test body0.18 ms
No steps named here
Tear down0.04 ms
No steps named here
1 fixture
tmp_path
Set up0.46 ms
No steps named here
Test body0.15 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(value)
1 parameter
value = earliest
Set up0.11 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.06 ms
No steps named here
1 marker
parametrize(value)
1 parameter
value = now
Set up0.09 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(value)
1 parameter
value = EARLIEST
Set up0.09 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
1 marker
parametrize(value)
1 parameter
value = now
Set up0.09 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down1.65 ms
No steps named here
Description
--dry-run never reaches the render, so this is where a typo is caught.
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
python -m works in a pip-installed wheel, a tox environment and a checkout alike, so a pipeline never has to find out which it is in.
Set up0.11 ms
No steps named here
Test body98 ms
No steps named here
Tear down0.20 ms
No steps named here
Description
main returns it so that every caller agrees about who owns SystemExit.
Set up0.18 ms
No steps named here
Test body96 ms
No steps named here
Tear down0.17 ms
No steps named here
Set up0.18 ms
No steps named here
Test body0.81 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.60 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
The two answers naming different folders is far worse than either being surprising: the shard would clean one and write its screenshots to another.
Set up0.07 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.26 ms
No steps named here
Test body0.37 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.16 ms
No steps named here
Test body0.12 ms
No steps named here
Tear down0.07 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.11 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.13 ms
No steps named here
Test body0.12 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.19 ms
No steps named here
Test body0.10 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.15 ms
No steps named here
Test body0.11 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.20 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body1.68 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body1.00 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.44 ms
No steps named here
Test body372 ms
No steps named here
Tear down0.17 ms
No steps named here
1 fixture
tmp_path
Set up0.50 ms
No steps named here
Test body404 ms
No steps named here
Tear down0.22 ms
No steps named here
1 fixture
tmp_path
Set up0.65 ms
No steps named here
Test body414 ms
No steps named here
Tear down0.21 ms
No steps named here
1 fixture
tmp_path
Set up0.56 ms
No steps named here
Test body394 ms
No steps named here
Tear down0.19 ms
No steps named here
1 fixture
tmp_path
Set up0.62 ms
No steps named here
Test body735 ms
No steps named here
Tear down0.21 ms
No steps named here
Set up0.24 ms
No steps named here
Test body0.19 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.14 ms
No steps named here
Set up0.14 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.15 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.18 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.11 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
coverage.py does, and a file with no statements missing none of them.
Set up0.11 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
--cov-fail-under=60 means 62% is a pass, whatever the default bands say.
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.12 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.14 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.14 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
coverage.py already reports these relative; resolving them against the working directory would be inventing a path nobody asked about.
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.18 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.46 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Recomputing it would put the report at odds with the terminal beside it: with branch coverage on, percent_covered already folds branches in.
Set up0.33 ms
No steps named here
Test body0.23 ms
No steps named here
Tear down0.11 ms
No steps named here
Set up0.22 ms
No steps named here
Test body0.09 ms
No steps named here
Tear down0.24 ms
No steps named here
Description
Releases before covered_branches report the total and the partial count.
Set up0.21 ms
No steps named here
Test body0.35 ms
No steps named here
Tear down0.24 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
(lines + branches) covered over (lines + branches) valid, so a report read from the xml and one read from the live run agree.
Set up0.08 ms
No steps named here
Test body0.24 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.10 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
A cap that kept the alphabetical head would hide exactly the files the tab is opened to find.
Set up0.10 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
An empty report reads as 100%: coverage.py calls a file with no statements fully covered, so an empty one would open a tab announcing a perfect score.
Set up0.14 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.15 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
A build that ran without coverage is not a drop to zero.
Set up0.10 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
1 fixture
tmpdir
Set up0.62 ms
No steps named here
Test body0.35 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
The kind comes off the first byte, not the extension: CI names these files whatever the pipeline felt like.
1 fixture
tmpdir
Set up0.48 ms
No steps named here
Test body0.29 ms
No steps named here
Tear down0.08 ms
No steps named here
1 fixture
tmpdir
Set up0.54 ms
No steps named here
Test body4.32 ms
No steps named here
Tear down0.09 ms
No steps named here
1 fixture
tmpdir
Set up0.93 ms
No steps named here
Test body0.24 ms
No steps named here
Tear down0.08 ms
No steps named here
1 fixture
tmpdir
Set up0.73 ms
No steps named here
Test body0.34 ms
No steps named here
Tear down0.10 ms
No steps named here
1 fixture
tmpdir
Set up0.50 ms
No steps named here
Test body0.31 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
Only deliberate report artifacts are picked up without being named. A .coverage is often left over from a run days ago, and quietly reporting yesterday's number is worse than reporting none.
1 fixture
tmpdir
Set up1.26 ms
No steps named here
Test body0.22 ms
No steps named here
Tear down0.08 ms
No steps named here
1 fixture
tmpdir
Set up0.62 ms
No steps named here
Test body0.42 ms
No steps named here
Tear down4.07 ms
No steps named here
1 fixture
tmpdir
Set up2.51 ms
No steps named here
Test body1.59 ms
No steps named here
Tear down0.56 ms
No steps named here
1 fixture
tmpdir
Set up1.90 ms
No steps named here
Test body1.08 ms
No steps named here
Tear down0.25 ms
No steps named here
Description
Reported into the page rather than raised: the tests are over by the time this runs, and a UsageError would turn a green run red over a decoration. The tab is where somebody is looking for the missing numbers.
1 fixture
tmpdir
Set up0.67 ms
No steps named here
Test body0.99 ms
No steps named here
Tear down0.09 ms
No steps named here
1 fixture
tmpdir
Set up0.78 ms
No steps named here
Test body0.14 ms
No steps named here
Tear down0.08 ms
No steps named here
Set up0.13 ms
No steps named here
Test body0.10 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.17 ms
No steps named here
Test body0.38 ms
No steps named here
Tear down0.12 ms
No steps named here
Description
The one number that must never drift. A report generated beside a terminal that says 74% and showing 63% is a bug report waiting to be filed, so the figure is coverage.py's own rather than one recomputed here.
1 fixture
tmp_path
Set up0.66 ms
No steps named here
Test body432 ms
No steps named here
Tear down0.18 ms
No steps named here
1 fixture
tmp_path
Set up0.66 ms
No steps named here
Test body436 ms
No steps named here
Tear down0.21 ms
No steps named here
Description
A Branches column of dashes says nothing except that --cov-branch was not passed, so the column is dropped rather than filled with zeroes.
1 fixture
tmp_path
Set up0.60 ms
No steps named here
Test body920 ms
No steps named here
Tear down0.21 ms
No steps named here
Description
Every xdist worker measures its own slice; pytest-cov combines them on the controller long before the report is written.
1 fixture
tmp_path
Set up0.63 ms
No steps named here
Test body1.21 s
No steps named here
Tear down0.18 ms
No steps named here
1 fixture
tmp_path
Set up0.64 ms
No steps named here
Test body853 ms
No steps named here
Tear down0.19 ms
No steps named here
1 fixture
tmp_path
Set up0.57 ms
No steps named here
Test body362 ms
No steps named here
Tear down0.17 ms
No steps named here
Description
--cov=src against a project with no src is the easiest mistake to make, and the generic setup guide is the least useful answer to it: it tells somebody to install pytest-cov and pass --cov when they have just done both.
1 fixture
tmp_path
Set up0.64 ms
No steps named here
Test body434 ms
No steps named here
Tear down0.19 ms
No steps named here
Description
Nothing measured it, so there is nothing to explain - just the guide.
1 fixture
tmp_path
Set up0.56 ms
No steps named here
Test body411 ms
No steps named here
Tear down0.24 ms
No steps named here
1 fixture
tmp_path
Set up0.62 ms
No steps named here
Test body423 ms
No steps named here
Tear down0.19 ms
No steps named here
Description
Issue #203's other half: reach your own pages from the report.
1 fixture
tmp_path
Set up0.56 ms
No steps named here
Test body371 ms
No steps named here
Tear down0.19 ms
No steps named here
Description
The nav is assembled out of whatever a command line said, and the report is a build artifact that gets published and passed round.
1 fixture
tmp_path
Set up0.57 ms
No steps named here
Test body376 ms
No steps named here
Tear down0.25 ms
No steps named here
Description
--cov-fail-under is pytest-cov's flag; asking for it without pytest-cov raises, and a missing target is not a reason to fail the render.
1 marker
parametrize(error)
1 parameter
error = AttributeError('no such option')
Set up0.23 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
--cov-fail-under is pytest-cov's flag; asking for it without pytest-cov raises, and a missing target is not a reason to fail the render.
1 marker
parametrize(error)
1 parameter
error = ValueError('unregistered')
Set up0.10 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
None is 'we do not know', which must not read as a pass.
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
79.999999999 against --cov-fail-under=80 passed the build.
Set up0.65 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
50% against a target of 40 is a pass, however low it looks.
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
A hand-edited or half-written report must not take the tab down.
Set up0.07 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
A coverage.json from three days ago reads exactly like a fresh one.
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.13 ms
No steps named here
Test body0.25 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
A Branches 0/0 tile says only that --cov-branch was not passed.
Set up0.12 ms
No steps named here
Test body0.20 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.21 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.14 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.21 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
'0/0' reads as none of its branches being covered, which is not what a file with no branches at all is saying.
Set up0.07 ms
No steps named here
Test body0.12 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.16 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.14 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.12 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
+0.0 reads as a measurement; 'no change' reads as the answer.
Set up0.11 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
build_summary is what every reader hands to every renderer, so the two halves agreeing about its shape is the thing worth checking.
Set up0.08 ms
No steps named here
Test body0.34 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Read rather than raced for: pytest-cov has stopped and combined its data - every xdist worker's share included - long before this runs.
Set up0.13 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.12 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.15 ms
No steps named here
Test body0.62 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
'No data to report', a coverage too old to write json, an unreadable data file - none of it is worth failing a finished test run over.
Set up0.10 ms
No steps named here
Test body0.12 ms
No steps named here
Tear down0.05 ms
No steps named here
2 fixtures
tmp_pathmonkeypatch
Set up0.53 ms
No steps named here
Test body0.32 ms
No steps named here
Tear down0.06 ms
No steps named here
2 fixtures
tmp_pathmonkeypatch
Set up0.36 ms
No steps named here
Test body0.14 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.08 ms
No steps named here
Description
It keeps this report one file, and a relative link survives the folder being moved or published somewhere else.
1 fixture
tmp_path
Set up0.43 ms
No steps named here
Test body0.23 ms
No steps named here
Tear down0.06 ms
No steps named here
1 fixture
tmp_path
Set up0.48 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
coverage.py names the folder whether or not --cov-report=html was asked for, and annotations disagreeing with the summary beside them are worse than no link at all.
1 fixture
tmp_path
Set up0.40 ms
No steps named here
Test body0.24 ms
No steps named here
Tear down0.08 ms
No steps named here
1 fixture
tmp_path
Set up0.42 ms
No steps named here
Test body0.21 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
The .coverage sqlite file, read through coverage.py itself - the shape --coverage-data hands the merge for every shard of a matrix.
1 fixture
tmp_path
Set up0.47 ms
No steps named here
Test body83 ms
No steps named here
Tear down0.18 ms
No steps named here
1 fixture
tmp_path
Set up0.53 ms
No steps named here
Test body0.63 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
Guarded as a whole: read_coverage_path is handed whatever somebody typed after --report-coverage-file.
1 fixture
tmp_path
Set up0.38 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.36 ms
No steps named here
Test body0.21 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.35 ms
No steps named here
Test body0.27 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
The flag is an instruction; the session is only what this run happened to measure.
1 fixture
tmp_path
Set up0.42 ms
No steps named here
Test body0.22 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
Almost always --cov names something that is never imported, and telling somebody to install pytest-cov when they just did is the least useful thing this page could do.
1 fixture
tmp_path
Set up0.50 ms
No steps named here
Test body0.42 ms
No steps named here
Tear down0.08 ms
No steps named here
Description
Not every suite measures coverage, and a notice on every one of them would be noise.
1 fixture
tmp_path
Set up0.40 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.12 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.37 ms
No steps named here
Tear down0.08 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.24 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.16 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.19 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.16 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.16 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.20 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.13 ms
No steps named here
Test body0.27 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
Passing rows included: the link is the row's address, not a failure's.
Set up0.14 ms
No steps named here
Test body9.50 ms
No steps named here
Tear down0.09 ms
No steps named here
Description
The whole point: a link is opened against whatever report is at that address by now, and a positional anchor would still resolve - to whichever test has since taken that place.
Set up0.17 ms
No steps named here
Test body2.70 ms
No steps named here
Tear down0.09 ms
No steps named here
Set up0.16 ms
No steps named here
Test body1.57 ms
No steps named here
Tear down0.08 ms
No steps named here
Description
A test inside a class is listed under its own name, which two classes in one file can share; the node id is what has the class in front of it.
Set up0.14 ms
No steps named here
Test body1.45 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
A file that failed to import has no node id. Falling back to the two names is worse than a node id and better than a row nothing can link to.
Set up0.13 ms
No steps named here
Test body1.11 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
`pytest-repeat` runs one node id as many rows. Two rows carrying one id is invalid HTML and, worse, silent: the link would open the first of them whichever was meant.
Set up0.13 ms
No steps named here
Test body1.66 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
It travels as the `#` end of a URL through a chat window and a ticket. `::`, `/` and the brackets round a parameter would all have to be escaped, and a fragment full of `%5B` is not a link anyone wants to read.
Set up0.15 ms
No steps named here
Test body0.10 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
The readable half is cut back to a whole word; the digits after it are taken from the whole node id, so two ids that agree until their last few characters are still told apart.
Set up0.14 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
A link pasted from yesterday's report opens today's on the same test.
Set up0.11 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
The slug drops what the two ids differ by; the digits are taken from the node id itself, which still has it.
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body1.19 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
One id, read off the <tr> the button sits in, so the address the link points at and the id the row answers to cannot drift apart.
Set up0.09 ms
No steps named here
Test body0.55 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.12 ms
No steps named here
Test body0.74 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
#test-metrics and #test-steps begin with the four letters every row anchor begins with. The map is read first, and its two entries are what keep those two links opening tabs rather than searching for a row.
Set up0.14 ms
No steps named here
Test body0.95 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
DataTables keeps only the current page in the document, so a row on page four is not there for getElementById to find - and the table has to be paged to it before it is anywhere at all.
Set up0.15 ms
No steps named here
Test body0.44 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
A filtered-out row is on no page at all, and landing on "No matching records found" is not what the link was pasted for. The status chips over the table are a column search of their own, and "show me the failures" is the likeliest of the three to be hiding the row - so the chip state goes with the searches, or the counts stay those of a filter that is off.
Set up0.10 ms
No steps named here
Test body0.59 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
The regression: the middle test read back as 1788535128.1 seconds. Measured against a start it had just zeroed itself, its duration was the current unix time - 56 years - and the slowest-tests chart drew every other test in the suite as a sliver beside it.
1 fixture
report
Set up1.39 s
No steps named here
Test body0.17 ms
No steps named here
Tear down0.12 ms
No steps named here
Description
Every test's time added up is the suite's time, near enough. One epoch-sized duration is all it takes for the "time in tests" tile to read 608889402m, so what matters is not any single number but that the total stays inside the run that produced it.
1 fixture
report
Set up0.15 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Not the moment its last test began setting up. Analytics sorts builds on this number and labels every point of every chart with it, so a stamp taken at the end of the run puts the build on the trend minutes after it happened - and two builds of one pipeline in the wrong order.
1 fixture
report
Set up0.09 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
The Time column used to floor everything quicker than 10ms at "0.0". Two decimal places of *seconds* is what the record carried, so a unit suite printed a column of zeros - and "0.0" reads as a test that was never timed rather than one that finished before the clock moved.
1 fixture
report
Set up0.09 ms
No steps named here
Test body0.81 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
The cell carries the unit that fits - "0.44 ms" beside "1.23 s" - and those two compared as text put the slowest test of the run in the middle, so the raw number of seconds rides along on data-order.
1 fixture
report
Set up0.08 ms
No steps named here
Test body0.31 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
"Darwin 24.6.0" is a true answer to a question nobody asked. Whoever reads this row thinks in macOS versions, distributions and architectures.
Set up0.23 ms
No steps named here
Test body0.33 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
Two suites that agree about 3.11 and disagree about PyPy - or about which half of a Windows install they are on - install different wheels.
Set up0.15 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
One process is not a worker count, and a row reading "1" invites the reader to wonder which seven crashed.
Set up0.22 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.07 ms
No steps named here
Set up0.19 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
xdist runs fewer workers than -n when there are fewer tests than workers. A report claiming eight when three ran reads as five crashes.
Set up0.15 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.14 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
-n auto is resolved by xdist before this reads it, but a shim config answers whatever it was handed, and this runs while a report is written.
Set up0.15 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.13 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.04 ms
No steps named here
1 fixture
monkeypatch
Set up0.13 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
GITHUB_RUN_ID does not change when a workflow is re-run, so the bare url opens the latest attempt - which is somebody else's failure.
1 fixture
monkeypatch
Set up0.21 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.06 ms
No steps named here
1 fixture
monkeypatch
Set up0.15 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.06 ms
No steps named here
1 fixture
monkeypatch
Set up0.14 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Assembling one would only be right on gitlab.com; CI_PIPELINE_URL is right behind a reverse proxy and on a self-hosted install too.
1 fixture
monkeypatch
Set up0.14 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.06 ms
No steps named here
1 fixture
monkeypatch
Set up0.13 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
BUILD_URL is absent on a controller installed without a configured root url, which is common enough on self-hosted Jenkins to be worth handling.
1 fixture
monkeypatch
Set up0.12 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
"This ran on a build agent and not on somebody's laptop" is most of what the row is for, and it is worth saying about a system nobody has heard of.
1 fixture
monkeypatch
Set up0.19 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.08 ms
No steps named here
Description
CI=false is a real thing to write in a matrix that turns a step off, and reading it as "this is CI" gets it exactly backwards.
1 fixture
monkeypatch
Set up0.25 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.07 ms
No steps named here
1 fixture
monkeypatch
Set up0.16 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
The merge rebuilds this from a shard's json rather than re-detecting.
Set up0.13 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
A CI checkout is a detached HEAD, where git answers "HEAD" - true, and useless. The system itself knows the branch somebody clicked.
1 fixture
monkeypatch
Set up0.13 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
GITHUB_REF_NAME on a pull_request event is "42/merge", which is not a branch anybody recognises.
1 fixture
monkeypatch
Set up0.13 ms
No steps named here
Test body15 ms
No steps named here
Tear down0.20 ms
No steps named here
Description
A report from outside a repository is a report with no Branch row, not a traceback and not a row reading "fatal: not a git repository".
1 fixture
tmp_path
Set up0.65 ms
No steps named here
Test body28 ms
No steps named here
Tear down0.22 ms
No steps named here
1 fixture
monkeypatch
Set up0.25 ms
No steps named here
Test body0.09 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
monkeypatch
Set up0.17 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.14 ms
No steps named here
Test body37 ms
No steps named here
Tear down0.08 ms
No steps named here
Description
Sorted the way a person reads a list, not the way ASCII does - which puts every capitalised distribution above every lowercase one.
Set up0.18 ms
No steps named here
Test body27 ms
No steps named here
Tear down0.09 ms
No steps named here
Set up0.17 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.13 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.13 ms
No steps named here
Test body0.30 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.14 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Every value on this panel now comes from an environment variable, and anything that can set one must not be able to put a javascript: url into a report somebody opens.
Set up0.13 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
A few hundred entries, and a full dependency inventory published into a file that gets attached to tickets - a fine thing to do deliberately.
Set up0.13 ms
No steps named here
Test body28 ms
No steps named here
Tear down0.19 ms
No steps named here
Set up0.24 ms
No steps named here
Test body53 ms
No steps named here
Tear down0.16 ms
No steps named here
Set up0.24 ms
No steps named here
Test body55 ms
No steps named here
Tear down0.20 ms
No steps named here
Description
A team that publishes its own Branch row means that one, and two rows disagreeing about the branch is worse than either of them alone.
1 fixture
monkeypatch
Set up0.26 ms
No steps named here
Test body16 ms
No steps named here
Tear down0.19 ms
No steps named here
1 fixture
monkeypatch
Set up0.29 ms
No steps named here
Test body28 ms
No steps named here
Tear down0.17 ms
No steps named here
1 fixture
monkeypatch
Set up0.24 ms
No steps named here
Test body30 ms
No steps named here
Tear down0.20 ms
No steps named here
Description
The cell shows the first fifty characters; both controls get all of it.
Set up0.28 ms
No steps named here
Test body1.74 ms
No steps named here
Tear down0.08 ms
No steps named here
Set up0.19 ms
No steps named here
Test body1.21 ms
No steps named here
Tear down0.07 ms
No steps named here
Set up0.23 ms
No steps named here
Test body2.57 ms
No steps named here
Tear down0.09 ms
No steps named here
Set up0.20 ms
No steps named here
Test body2.59 ms
No steps named here
Tear down0.09 ms
No steps named here
Description
A test inside a class is listed under its own name; pytest wants the class in front of it, and the node id is the only place that has it.
Set up0.23 ms
No steps named here
Test body1.17 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
Pasted bare, `test_one[a b]` is two arguments to bash and a glob that matches nothing to zsh.
Set up0.16 ms
No steps named here
Test body1.30 ms
No steps named here
Tear down0.08 ms
No steps named here
Description
`pytest` on its own runs everything, which is not what the button says.
Set up0.13 ms
No steps named here
Test body1.32 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
Nothing to copy, and a button that copies an empty string reads as a bug.
Set up0.13 ms
No steps named here
Test body0.90 ms
No steps named here
Tear down0.07 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.82 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.11 ms
No steps named here
Test body1.54 ms
No steps named here
Tear down0.07 ms
No steps named here
Set up0.12 ms
No steps named here
Test body1.20 ms
No steps named here
Tear down0.07 ms
No steps named here
Set up0.12 ms
No steps named here
Test body1.06 ms
No steps named here
Tear down0.10 ms
No steps named here
Set up0.11 ms
No steps named here
Test body1.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.99 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Search, CSV, Excel and print are all built from cell text. The error rides in an attribute and every button is an icon, so all four get the cut message and the marker that says it was cut - not a traceback per row.
Set up0.10 ms
No steps named here
Test body0.97 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
The tick that replaces the icon is the same tick for all three, and it is 22 pixels wide at the end of a row somebody is reading the other end of. The word in the middle of the page is what says which one landed.
Set up0.12 ms
No steps named here
Test body0.71 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
Otherwise a reader is left with a button that did nothing and no idea why - which is what happens to the file:// clipboard in some browsers.
Set up0.14 ms
No steps named here
Test body0.48 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
The click is confirmed by the thing itself rather than by a word about it - the error, the command or the link, in the face it was written in.
Set up0.12 ms
No steps named here
Test body0.49 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
A traceback cut off square reads as the end of the message. The panel holds four lines and dissolves the rest - the same thing a cut error message does in the table it was copied from.
Set up0.10 ms
No steps named here
Test body0.60 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
The panel is a live region: "Error copied" is what a screen reader should announce, not the forty lines that follow it.
Set up0.09 ms
No steps named here
Test body0.19 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Nothing reached the clipboard, so showing what did not go there beside "Press Ctrl+C" would say two opposite things at once.
Set up0.19 ms
No steps named here
Test body0.41 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
Two of these stacking would cover the table, and a message left up would claim the clipboard still holds what that row put there.
Set up0.11 ms
No steps named here
Test body0.30 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
`role="status"` is what has a screen reader read the copy out; the tick on the button never said anything to anyone not looking at it.
Set up0.09 ms
No steps named here
Test body0.29 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
It sits over the middle of the table for a moment, and the row underneath has to stay live the whole time.
Set up0.09 ms
No steps named here
Test body0.45 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
A table of forty failures would otherwise carry 160 buttons for the one somebody wants.
Set up0.14 ms
No steps named here
Test body1.60 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
Hidden by width and opacity rather than by `display`, which cannot be transitioned - and a button with no box has no width to animate.
Set up0.12 ms
No steps named here
Test body0.63 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
Four buttons wider than the line has left, and the strip wraps - which is the right thing to do and a jump to watch happen. The widths settle in one step, so the move is known before it is made and can be animated.
Set up0.11 ms
No steps named here
Test body0.55 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Growing them reflowed the cell on every frame, so a strip with no room left jumped to the next line part-way through the animation, at whichever frame it stopped fitting.
Set up0.09 ms
No steps named here
Test body0.37 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Four buttons appearing together under the pointer reads as a flicker; a strip unfolding reads as one movement.
Set up0.09 ms
No steps named here
Test body0.27 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Four strips left open down a page is the clutter they were folded away to avoid. A click anywhere else, or Escape, shuts them all.
Set up0.12 ms
No steps named here
Test body0.89 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
A button that changes what is on screen has to say so to a reader who cannot see the change.
Set up0.11 ms
No steps named here
Test body0.49 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
A gradient can only be painted onto an element, so the characters that fade have to be one of their own.
Set up0.12 ms
No steps named here
Test body1.20 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
Nothing was cut, so there is nothing to say has been.
Set up0.10 ms
No steps named here
Test body0.94 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
A CSV and a print-out hold the cell's text, and a message cut short with nothing to say so is a trap in a file somebody reads a week later.
Set up0.09 ms
No steps named here
Test body1.16 ms
No steps named here
Tear down0.10 ms
No steps named here
Description
Or a message fades out and then opens the panel on the same fifty characters it was already showing.
Set up0.13 ms
No steps named here
Test body2.38 ms
No steps named here
Tear down0.13 ms
No steps named here
Description
Painting text with a background means `color: transparent`, and `currentcolor` inside the gradient then resolves to exactly that - a fade from nothing to nothing, which is a message that stops eight characters early with no sign it was ever cut.
Set up0.14 ms
No steps named here
Test body0.60 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.12 ms
No steps named here
Test body0.53 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.45 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.41 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.40 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.44 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.13 ms
No steps named here
Test body0.55 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.15 ms
No steps named here
Test body0.49 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.16 ms
No steps named here
Test body0.80 ms
No steps named here
Tear down0.07 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.14 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body107 ms
No steps named here
Tear down0.10 ms
No steps named here
1 fixture
tmp_path
Set up0.53 ms
No steps named here
Test body413 ms
No steps named here
Tear down0.26 ms
No steps named here
1 fixture
tmp_path
Set up0.65 ms
No steps named here
Test body414 ms
No steps named here
Tear down0.20 ms
No steps named here
1 fixture
tmp_path
Set up0.61 ms
No steps named here
Test body417 ms
No steps named here
Tear down0.23 ms
No steps named here
1 fixture
tmp_path
Set up0.86 ms
No steps named here
Test body424 ms
No steps named here
Tear down0.21 ms
No steps named here
1 fixture
tmp_path
Set up0.62 ms
No steps named here
Test body392 ms
No steps named here
Tear down0.22 ms
No steps named here
Set up0.25 ms
No steps named here
Test body2.62 ms
No steps named here
Tear down0.07 ms
No steps named here
Set up0.10 ms
No steps named here
Test body2.26 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.12 ms
No steps named here
Test body0.13 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.13 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.08 ms
No steps named here
Test body29 ms
No steps named here
Tear down0.11 ms
No steps named here
Set up0.13 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
All three, from the flag and from the ini key, with the flag winning. The plan's own flag table says {pass,fail} - what ships is the wider set, because a team whose policy is that an xfail marker left on a passing test is a defect wants it out of the pass column without turning the build red.
Set up0.14 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
Rather than falling back to the default, which is the one answer the team setting it was trying to avoid.
Set up0.10 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.39 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
The attribute is the assertion that failed; the body is everything. The leading blank line is not decoration: a captured longrepr very often starts with one, and a message attribute holding nothing but whitespace is what every consumer would then show in its summary column.
Set up0.09 ms
No steps named here
Test body0.12 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Which phase broke is read off ``phases``: a record that never got as far as a call never ran the test at all.
Set up0.09 ms
No steps named here
Test body0.18 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
pytest hands the location over as a (path, lineno, reason) tuple and the record stores the repr of it, which is not what anyone opening the build wants to read in an attribute already called ``message``.
Set up0.09 ms
No steps named here
Test body0.13 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Never a failure. Azure DevOps reads any failure or error as a failed run, so mapping a documented known bug onto one turns every suite that documents its known bugs red across three CI systems at once.
Set up0.09 ms
No steps named here
Test body0.22 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
pytest's own writer emits a plain passing testcase for a non-strict XPASS, and non-strict is the only kind that can reach here, so the default is parity - and the two other modes are there for the teams who disagree.
Set up0.14 ms
No steps named here
Test body0.27 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
A deliberate divergence from pytest, whose own mangling leaves a collection record's classname empty - and GitLab groups by classname in practice, so every broken module in the repository would be filed together under one nameless heading.
Set up0.10 ms
No steps named here
Test body0.12 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
A module skipped whole is not an error, and a build that reported it as one would be red for a file the run was told not to look at.
Set up0.10 ms
No steps named here
Test body0.10 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Loudly, and never absorbed. A shard written by a newer release than the one merging it is how this happens, and both other places that map a status turn an unknown one into something plausible instead of saying so.
Set up0.10 ms
No steps named here
Test body0.11 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Recomputed from what was emitted, never summed from what an input claimed: Jenkins recounts the children regardless and would then silently contradict the header of the same file.
Set up0.10 ms
No steps named here
Test body0.18 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
duration is measured from setup to the start of teardown and rounded to two decimals, so it drops the teardown and quantises anything quick to 0.0 - and Azure derives the end of the run from the sum of these numbers.
Set up0.11 ms
No steps named here
Test body0.10 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
One element per nodeid, never one per attempt: an extra element inflates the count, and a duplicate is read as failed by GitLab and passed by Jenkins for the same build. The count goes in system-out as well as in the properties, because GitLab ignores properties outright.
Set up0.10 ms
No steps named here
Test body0.12 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
A record's message is stored raw - the page escapes it only at render time - so a terminal control byte written by a failing test would otherwise go straight into the file the whole pipeline parses.
Set up0.15 ms
No steps named here
Test body0.21 ms
No steps named here
Tear down0.08 ms
No steps named here
Description
Two shards that both ran a test, or two ids that differ only in a character the mangling drops. An emitted duplicate is worse than a renamed test: the same build is then reported with two different outcomes depending on which CI server is looking at it.
Set up0.17 ms
No steps named here
Test body0.19 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
A collector resolves the path against the file it read it from, which is not where the screenshots live whenever CI is told to put the xml somewhere of its own.
1 fixture
tmp_path
Set up0.52 ms
No steps named here
Test body0.82 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
At most one of each element per testcase. pytest's own writer emits system-out twice for a skipped test, and a document with two of them is one whose second some parsers drop and others concatenate.
Set up0.12 ms
No steps named here
Test body0.13 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
And writes the same test names pytest's own --junitxml would. The two writers run over the same suite in turn and their (classname, name) sets are compared, because the mangling is a port of pytest's own and the whole reason to port it rather than invent one is that a file written here stays diffable against a file written there.
1 fixture
tmp_path
Set up0.57 ms
No steps named here
Test body767 ms
No steps named here
Tear down0.22 ms
No steps named here
Description
One document from the controller, not one per worker: four files that each hold a quarter of the suite are four runs to every CI server that globs for them.
1 fixture
tmp_path
Set up0.59 ms
No steps named here
Test body729 ms
No steps named here
Tear down0.20 ms
No steps named here
Description
A leg that is not also merging renders nothing, so an xml from it would be a quarter of the run - and a CI glob would ingest four of those plus the merged one and count every test twice.
1 fixture
tmp_path
Set up0.57 ms
No steps named here
Test body274 ms
No steps named here
Tear down0.23 ms
No steps named here
Description
A missing file reads as "the job never ran", which is the one thing a run that finished cleanly is not. The page has nothing to show and is not written; the document says tests="0" and is.
1 fixture
tmp_path
Set up0.65 ms
No steps named here
Test body242 ms
No steps named here
Tear down0.20 ms
No steps named here
Description
The xml is written before the page - deliberately, so an empty run still gets one - which is exactly why it may not raise: a bad path would otherwise take the html report, output.json and the archived build with it, over a typo, in the one hook that runs after every test has finished.
1 fixture
tmp_path
Set up0.69 ms
No steps named here
Test body400 ms
No steps named here
Tear down0.28 ms
No steps named here
Description
Nothing has failed yet, and "0%" against a blank name reads as a bug.
Set up0.30 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.17 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
The percentage is failures over every build the suite appeared in, so a suite that fails once in fifty is not shown beside one that fails always.
Set up0.09 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
importlib.metadata calls it `name`; the old pkg_resources called it `project_name`, and pytest hands over whichever it has.
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.36 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.14 ms
No steps named here
Set up0.14 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.16 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
A decorator stacked in a way inspect cannot see through still has to leave the test running.
Set up0.20 ms
No steps named here
Test body0.20 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.13 ms
No steps named here
Test body0.09 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.15 ms
No steps named here
Test body0.11 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.14 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Every line is read, because a traceback puts the raise at the bottom.
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
1 fixture
tmp_path
Set up0.53 ms
No steps named here
Test body385 ms
No steps named here
Tear down0.20 ms
No steps named here
1 fixture
tmp_path
Set up0.64 ms
No steps named here
Test body393 ms
No steps named here
Tear down0.21 ms
No steps named here
1 fixture
tmp_path
Set up0.64 ms
No steps named here
Test body400 ms
No steps named here
Tear down0.25 ms
No steps named here
1 fixture
tmp_path
Set up0.59 ms
No steps named here
Test body396 ms
No steps named here
Tear down0.20 ms
No steps named here
1 fixture
tmp_path
Set up0.63 ms
No steps named here
Test body371 ms
No steps named here
Tear down0.21 ms
No steps named here
1 fixture
tmp_path
Set up0.54 ms
No steps named here
Test body353 ms
No steps named here
Tear down0.20 ms
No steps named here
1 fixture
tmp_path
Set up0.60 ms
No steps named here
Test body358 ms
No steps named here
Tear down0.21 ms
No steps named here
1 fixture
tmp_path
Set up0.62 ms
No steps named here
Test body387 ms
No steps named here
Tear down0.25 ms
No steps named here
1 marker
skipif(reason=needs pytest-rerunfailures)
1 fixture
tmp_path
Set up0.73 ms
No steps named here
Test body395 ms
No steps named here
Tear down0.22 ms
No steps named here
Description
A column of dashes under -s reads as a broken feature, so the page says what is suppressing its own output.
1 fixture
tmp_path
Set up0.60 ms
No steps named here
Test body386 ms
No steps named here
Tear down0.29 ms
No steps named here
1 fixture
tmp_path
Set up0.57 ms
No steps named here
Test body381 ms
No steps named here
Tear down0.21 ms
No steps named here
1 fixture
tmp_path
Set up0.59 ms
No steps named here
Test body373 ms
No steps named here
Tear down0.24 ms
No steps named here
1 fixture
tmp_path
Set up0.64 ms
No steps named here
Test body368 ms
No steps named here
Tear down0.21 ms
No steps named here
1 fixture
tmp_path
Set up0.70 ms
No steps named here
Test body363 ms
No steps named here
Tear down0.25 ms
No steps named here
1 fixture
tmp_path
Set up0.64 ms
No steps named here
Test body378 ms
No steps named here
Tear down0.22 ms
No steps named here
1 fixture
tmp_path
Set up0.60 ms
No steps named here
Test body366 ms
No steps named here
Tear down0.22 ms
No steps named here
1 fixture
tmp_path
Set up0.56 ms
No steps named here
Test body367 ms
No steps named here
Tear down0.25 ms
No steps named here
1 fixture
tmp_path
Set up0.58 ms
No steps named here
Test body382 ms
No steps named here
Tear down0.24 ms
No steps named here
1 fixture
tmp_path
Set up0.61 ms
No steps named here
Test body425 ms
No steps named here
Tear down0.23 ms
No steps named here
1 fixture
tmp_path
Set up0.65 ms
No steps named here
Test body379 ms
No steps named here
Tear down0.24 ms
No steps named here
1 fixture
tmp_path
Set up0.58 ms
No steps named here
Test body407 ms
No steps named here
Tear down0.23 ms
No steps named here
1 fixture
tmp_path
Set up0.67 ms
No steps named here
Test body365 ms
No steps named here
Tear down0.20 ms
No steps named here
1 fixture
tmp_path
Set up0.73 ms
No steps named here
Test body381 ms
No steps named here
Tear down0.21 ms
No steps named here
1 fixture
tmp_path
Set up0.57 ms
No steps named here
Test body361 ms
No steps named here
Tear down0.24 ms
No steps named here
1 fixture
tmp_path
Set up0.61 ms
No steps named here
Test body382 ms
No steps named here
Tear down0.26 ms
No steps named here
1 fixture
tmp_path
Set up0.71 ms
No steps named here
Test body370 ms
No steps named here
Tear down0.23 ms
No steps named here
1 fixture
tmp_path
Set up0.61 ms
No steps named here
Test body378 ms
No steps named here
Tear down0.21 ms
No steps named here
Description
xfail(raises=ValueError) should not read as <class 'ValueError'>.
Set up0.18 ms
No steps named here
Test body0.14 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.12 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.10 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.36 ms
No steps named here
Description
It carries every row the test will ever run with - a hundred per badge.
Set up0.07 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
The condition was evaluated at import; only the reason still means anything.
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.09 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Showing it invites people to go looking for a decorator not in their file.
Set up0.11 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Round the wrong way this takes down every test in the run.
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.12 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Otherwise the row reads as a mapping nobody wrote.
Set up0.10 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
A collector that is not a test function still has to answer.
Set up0.11 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
An indented docstring keeps the indentation of the file it is in.
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
A DoctestItem has no .function, and ``None.__doc__`` is not empty - it is NoneType's own docstring, so every doctest row in a --doctest-modules run was described as "The type of the None singleton."
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Guarded as a whole: a blank description beats a test nobody can see.
Set up0.11 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
`report_path` reads any path with '.html' in it as a file name, so a folder called `my.html.d` would put the report in the current directory under that name. Refused before anything is read, because by the time somebody goes looking for the report the tests have long since run.
2 fixtures
tmp_pathcapsys
Set up0.70 ms
No steps named here
Test body1.66 ms
No steps named here
Tear down0.08 ms
No steps named here
Description
The guard is about '.html' in the middle of a path, never about the word: naming the file itself is half of what the flag is for, and a check that refused it would send everybody who does to --html-report ./somewhere.
2 fixtures
tmp_pathcapsys
Set up0.61 ms
No steps named here
Test body111 ms
No steps named here
Tear down0.13 ms
No steps named here
Description
The usual cause is a CI step that unpacked the artifacts one directory deeper than the merge was pointed at, so the path it walked is the answer.
2 fixtures
tmp_pathcapsys
Set up0.54 ms
No steps named here
Test body0.90 ms
No steps named here
Tear down0.08 ms
No steps named here
Description
The merge spells the retention flags the way a pytest run spells them, so it refuses them in the same words - and refuses them up front, because these three are read deep inside render() where a UsageError would abort a build that had already rotated the archive.
2 fixtures
tmp_pathcapsys
Set up0.42 ms
No steps named here
Test body1.26 ms
No steps named here
Tear down0.10 ms
No steps named here
Description
A retried CI leg whose artifact landed twice is common and recoverable, so the merge keeps the copy that finished later and says which two files it was choosing between rather than failing the whole matrix over it.
2 fixtures
tmp_pathcapsys
Set up0.46 ms
No steps named here
Test body107 ms
No steps named here
Tear down0.13 ms
No steps named here
Description
The question `inspect` answers is "did all four artifacts arrive, and do they overlap" - asked before a merge, and asked again when the totals were not the totals somebody expected. It must be safe to ask at any point.
2 fixtures
tmp_pathcapsys
Set up0.52 ms
No steps named here
Test body1.50 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
`--json` exists so a pipeline can check the count of arrived shards itself; it is the same merge, reported as a document.
2 fixtures
tmp_pathcapsys
Set up0.45 ms
No steps named here
Test body1.33 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
The summary of the build that would have been produced, arrived at by the same code - which is what makes it worth running before the real one.
2 fixtures
tmp_pathcapsys
Set up0.57 ms
No steps named here
Test body1.32 ms
No steps named here
Tear down0.08 ms
No steps named here
Description
render() writes no page for a run that produced no records - the `if self._records:` guard a plain pytest run obeys too - and a matrix whose shards collected nothing between them lands exactly there. Naming the path anyway would send the next step of a pipeline off to publish a file that was never written, and it would blame the copy.
2 fixtures
tmp_pathcapsys
Set up0.51 ms
No steps named here
Test body1.42 ms
No steps named here
Tear down0.09 ms
No steps named here
Description
`merge ./report --html-report ./report` is the sequential flow's own command, and the shards live under it. The merge owns `<out>/pytest_screenshots` and clears it; the sources sit in `<out>/shards/<id>/pytest_screenshots` and must still be there afterwards, or a second merge of the same folder finds the images it staged the first time gone.
2 fixtures
tmp_pathcapsys
Set up0.56 ms
No steps named here
Test body104 ms
No steps named here
Tear down0.13 ms
No steps named here
Description
A pytest run defaults to 'auto' because somebody is very often sat in front of it. A merge is run to look at four downloaded artifacts, or by a CI step, and neither wants a tab stolen.
3 fixtures
tmp_pathmonkeypatchcapsys
Set up0.65 ms
No steps named here
Test body116 ms
No steps named here
Tear down0.18 ms
No steps named here
Description
--strict is a verdict about the merge being complete, not about the build being publishable: the page that explains what was folded is exactly what somebody reading the failed step needs to open.
2 fixtures
tmp_pathcapsys
Set up0.55 ms
No steps named here
Test body106 ms
No steps named here
Tear down0.13 ms
No steps named here
Description
The two flags answer two different questions. A pipeline that wants the step to go red on a failing test says --exit-code; one that only wants to know all four artifacts arrived says --strict. A merge that read the tests' verdict without being asked would make the second impossible.
2 fixtures
tmp_pathcapsys
Set up0.52 ms
No steps named here
Test body220 ms
No steps named here
Tear down0.17 ms
No steps named here
Description
Two codes for one failure, because the two commands produced different amounts. `merge` has the report on disk and says so - something was made, and 2 is reserved for nothing at all - while the XML is the only thing `junit` exists to write.
2 fixtures
tmp_pathcapsys
Set up1.21 ms
No steps named here
Test body107 ms
No steps named here
Tear down0.12 ms
No steps named here
Description
A test that failed and was photographed on one shard and passed on the next folds into a row from the second shard holding the first shard's images. `junit` copies nothing, so it has to name those files the way a merged report holds them - under the shard that took them, never under the shard the surviving row came from.
2 fixtures
tmp_pathcapsys
Set up0.56 ms
No steps named here
Test body2.84 ms
No steps named here
Tear down0.09 ms
No steps named here
Description
A pipeline that publishes test results into GitLab or Azure and keeps no HTML should not have to write a build it will throw away.
2 fixtures
tmp_pathcapsys
Set up0.49 ms
No steps named here
Test body1.67 ms
No steps named here
Tear down0.08 ms
No steps named here
Description
The entry point, end to end: the same merge, reached the way CI reaches it. Everything above calls main() directly, so without this nothing would notice the command had stopped being installable.
1 fixture
tmp_path
Set up0.41 ms
No steps named here
Test body224 ms
No steps named here
Tear down0.27 ms
No steps named here
Description
The behaviour before legs were given one, and still the default.
Set up0.28 ms
No steps named here
Test body0.17 ms
No steps named here
Tear down0.08 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
The failure it causes is invisible: the totals are simply larger than the run was.
Set up0.09 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Printed every time, with no threshold: there is no honest way to decide from inside a merge that a bundle is stale.
Set up0.09 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.18 ms
No steps named here
Description
Printing the epoch instead would read as a real date.
Set up0.10 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
The figure is real but it is that shard's share, and a headline number that quietly means a quarter of the run is worse than no number.
Set up0.07 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Averaging them would be a made-up number; adding them counts shared lines twice. The answer is to combine the data, and the notice says how.
Set up0.08 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
An empty Coverage tab on a sharded run reads as a broken feature; the notice is what makes it read as a run that was never asked to measure.
Set up0.08 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
1 fixture
tmp_path
Set up0.55 ms
No steps named here
Test body0.32 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Pointing the flag at the wrong file is the ordinary mistake, and a silent empty tab does not help anybody find it.
1 fixture
tmp_path
Set up0.50 ms
No steps named here
Test body1.65 ms
No steps named here
Tear down0.07 ms
No steps named here
1 fixture
tmp_path
Set up0.49 ms
No steps named here
Test body0.17 ms
No steps named here
Tear down0.09 ms
No steps named here
1 fixture
tmp_path
Set up0.45 ms
No steps named here
Test body0.32 ms
No steps named here
Tear down0.06 ms
No steps named here
1 fixture
tmp_path
Set up0.51 ms
No steps named here
Test body0.34 ms
No steps named here
Tear down0.06 ms
No steps named here
1 fixture
tmp_path
Set up0.43 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.36 ms
No steps named here
Test body0.22 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Read off each shard's own config: under the shim the merge cannot tell that -s was passed, so the column would lose its explanation.
Set up0.25 ms
No steps named here
Test body0.50 ms
No steps named here
Tear down0.08 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.11 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
The same test under two spellings is two rows and two histories.
Set up0.10 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.09 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
A container that checked out at /src and a runner that used /home/runner/work ran the same test.
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
An empty node id is quarantined; the original is worth more than that.
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.14 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.07 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.11 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Two CI jobs downloading the same four artifacts have to agree.
Set up0.09 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
The later run is the one whose result the pipeline acted on.
Set up0.10 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Read by every step that has to break a tie the same way twice.
Set up0.12 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.12 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.16 ms
No steps named here
Description
It names the file the next build archives this one as, and orders the builds in Analytics - so it is asked about rather than assumed.
Set up0.12 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.12 ms
No steps named here
Test body0.09 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
The panel used to be the one part of a merged page that could not be honest - it described the merging process, which ran no tests at all.
Set up0.10 ms
No steps named here
Test body0.30 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.15 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.21 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.13 ms
No steps named here
Test body0.28 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.14 ms
No steps named here
Test body0.17 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.14 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
One summarised row would be a lie.
Set up0.09 ms
No steps named here
Test body0.17 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.15 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.14 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.16 ms
No steps named here
Test body0.12 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Two legs from two different builds is a matrix that should not have been merged, and a link to the first one would be read as the whole build's.
Set up0.09 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Smoothing this over would hide the one thing that explains why half the matrix failed.
Set up0.09 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
An older leg carries none of these keys, and reads back as a leg that had nothing to say - which is what it is.
Set up0.08 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Eight identical three-hundred entry rows is not a panel, it is a wall.
Set up0.09 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
This is the case the row is for: the leg that failed had another version of something installed.
Set up0.11 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
The fallback that keeps a half-upgraded matrix readable: one leg's bundle knows about the os key and the other's does not.
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.14 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.22 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
A leg that published "commit=..." through --build-info meant that one.
Set up0.09 ms
No steps named here
Test body0.12 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Longest row on the page by two orders of magnitude, and everything above it is what somebody opened the panel for.
Set up0.09 ms
No steps named here
Test body0.14 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.52 ms
No steps named here
Test body1.27 ms
No steps named here
Tear down0.11 ms
No steps named here
Description
This leg has already written its own bundle, so the folder is the whole run - adding to _records would count this leg's tests twice.
1 fixture
tmp_path
Set up0.48 ms
No steps named here
Test body0.79 ms
No steps named here
Tear down0.07 ms
No steps named here
1 fixture
tmp_path
Set up0.43 ms
No steps named here
Test body0.55 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
The panel on a merged report used to describe the merging machine, which ran no tests at all.
1 fixture
tmp_path
Set up0.40 ms
No steps named here
Test body4.55 ms
No steps named here
Tear down0.08 ms
No steps named here
Description
So --report-junit on this leg writes the document the merge command would have written from the same bundles.
1 fixture
tmp_path
Set up0.61 ms
No steps named here
Test body0.96 ms
No steps named here
Tear down0.08 ms
No steps named here
1 fixture
tmp_path
Set up0.48 ms
No steps named here
Test body0.09 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
Said out loud on every merge, not only when something looks wrong: this is the only place the build's provenance is ever recorded.
2 fixtures
tmp_pathcapsys
Set up0.56 ms
No steps named here
Test body0.70 ms
No steps named here
Tear down0.09 ms
No steps named here
Description
The folder is persistent, so a leg renamed or dropped since the last run leaves a bundle the next --report-shard-merge would report as part of this build - six tests when four ran.
1 fixture
tmp_path
Set up0.57 ms
No steps named here
Test body0.89 ms
No steps named here
Tear down0.07 ms
No steps named here
2 fixtures
tmp_pathcapsys
Set up0.48 ms
No steps named here
Test body1.06 ms
No steps named here
Tear down0.09 ms
No steps named here
2 fixtures
tmp_pathcapsys
Set up0.48 ms
No steps named here
Test body1.13 ms
No steps named here
Tear down0.10 ms
No steps named here
1 fixture
tmp_path
Set up0.57 ms
No steps named here
Test body0.54 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
The behaviour before tokens existed, and still the default.
1 fixture
tmp_path
Set up0.51 ms
No steps named here
Test body0.83 ms
No steps named here
Tear down0.14 ms
No steps named here
2 fixtures
tmp_pathcapsys
Set up0.49 ms
No steps named here
Test body0.92 ms
No steps named here
Tear down0.12 ms
No steps named here
1 fixture
tmp_path
Set up0.52 ms
No steps named here
Test body0.80 ms
No steps named here
Tear down0.08 ms
No steps named here
2 fixtures
tmp_pathcapsys
Set up0.57 ms
No steps named here
Test body0.35 ms
No steps named here
Tear down0.07 ms
No steps named here
Set up0.10 ms
No steps named here
Test body1.42 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.30 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.34 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
The message fits in the cell, so there is nothing to open a panel for - but it is still worth copying.
Set up0.09 ms
No steps named here
Test body0.37 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.33 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.32 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.29 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.94 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
The button is left in the row and hidden by `data-logs`, so a test that captured nothing shows a dash instead of opening an empty panel.
Set up0.07 ms
No steps named here
Test body0.51 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
A thumbnail on a Test Metrics row: the picture itself, and a link that opens it in the same overlay the gallery uses.
Set up0.11 ms
No steps named here
Test body0.29 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.36 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.35 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
CSS hides an empty pill, which is how a plain text entry avoids one.
Set up0.09 ms
No steps named here
Test body0.30 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.22 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.27 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.41 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.17 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.27 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
A dash, not a 0: this file has no branches to cover, which is not the same statement as none of its branches being covered.
Set up0.08 ms
No steps named here
Test body0.21 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.17 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.16 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.18 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
The two halves of a tab's identity have to agree. Renaming this tab's label and hash silently broke the Test Metrics button that jumps to it, because the jump found the nav link by its href. Nothing failed - the link was still there, it had just stopped being found.
Set up0.13 ms
No steps named here
Test body27 ms
No steps named here
Tear down0.08 ms
No steps named here
Description
Aliases included: #attachments still has to reach the renamed tab.
Set up0.11 ms
No steps named here
Test body27 ms
No steps named here
Tear down0.08 ms
No steps named here
Description
#attachments was the hash before the tab was renamed; links were shared.
Set up0.12 ms
No steps named here
Test body29 ms
No steps named here
Tear down0.10 ms
No steps named here
Description
The dashboard's inline strip and the page-level footer, both of them. A report outlives the run that wrote it, and the release it was drawn by is stated nowhere else on the page - so a page rendered without the version is a page nobody can date to a release later on.
Set up0.16 ms
No steps named here
Test body26 ms
No steps named here
Tear down0.09 ms
No steps named here
Set up0.14 ms
No steps named here
Test body39 ms
No steps named here
Tear down0.08 ms
No steps named here
Description
A component that is handed all its fields must leave none behind. The substitution is a plain string replace, so a field renamed in the template and not in the caller leaves a raw `%(name)%` sitting on the page. Nothing fails; the report just shows the placeholder to whoever opens it.
Set up0.11 ms
No steps named here
Test body2.14 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
The two halves of a component have to exist together. A template with no class is dead markup; a class with no template raises only when something first tries to render it, which is at the end of a run somebody has already waited for.
Set up0.08 ms
No steps named here
Test body0.12 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body109 ms
No steps named here
Tear down0.09 ms
No steps named here
Set up0.11 ms
No steps named here
Test body3.97 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body102 ms
No steps named here
Tear down0.08 ms
No steps named here
Set up0.15 ms
No steps named here
Test body100 ms
No steps named here
Tear down0.09 ms
No steps named here
Set up0.13 ms
No steps named here
Test body100 ms
No steps named here
Tear down0.08 ms
No steps named here
Set up0.14 ms
No steps named here
Test body0.11 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
The marker said it would fail and it did not, which is worth its own colour - a green row would hide that the marker is now wrong.
Set up0.10 ms
No steps named here
Test body0.10 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Only the call phase says how a test went; setup passing says nothing.
Set up0.09 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
The source and the frames are noise on a row; the raised line is not.
Set up0.09 ms
No steps named here
Test body0.09 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
wasxfail on a failed call is pytest reporting the strict-xfail case.
Set up0.08 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
A fixture that blew up did not run the test at all, and a row that says FAIL sends people reading a test that never executed.
Set up0.08 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.10 ms
No steps named here
Description
A fixture failure is usually explained by the frames above it.
Set up0.16 ms
No steps named here
Test body0.14 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.19 ms
No steps named here
Test body0.14 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.14 ms
No steps named here
Test body0.11 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.15 ms
No steps named here
Test body0.09 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
The page shows the assertion; a JUnit collector shows the reason.
Set up0.09 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
A test with no steps of its own still says where its time went.
1 marker
parametrize(phase)
1 parameter
phase = setup
Set up0.11 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
A test with no steps of its own still says where its time went.
1 marker
parametrize(phase)
1 parameter
phase = call
Set up0.14 ms
No steps named here
Test body0.10 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
A test with no steps of its own still says where its time went.
1 marker
parametrize(phase)
1 parameter
phase = teardown
Set up0.12 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Setup, call and teardown each report their own, so no single report has the whole picture.
Set up0.12 ms
No steps named here
Test body0.10 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.12 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Every one of these options is new, so the run that sets none of them has to behave exactly as it did before they existed.
Set up0.15 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.14 ms
No steps named here
Test body0.16 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
A store_true flag has no "off" to pass, so the only way it can lose to an ini key is by not being asked about at all.
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.12 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.24 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.12 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.11 ms
No steps named here
Description
There is nothing sensible to do with the ask: the leg that merges is the last leg of the run, and a process that is not a shard has no records of its own to add to what it would be merging.
Set up0.11 ms
No steps named here
Test body0.50 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
A value of "///" is not empty, but nothing of it survives being made fit for a directory name, and a shard whose id came out empty would write over the report base itself.
Set up0.11 ms
No steps named here
Test body0.12 ms
No steps named here
Tear down0.09 ms
No steps named here
Description
The flag itself is answered by argparse's choices, so this is the ini half - where nothing gates the value - and the run stops rather than quietly handing back the default the team set the key to get away from.
Set up0.12 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Nobody types --report-shard-run on a matrix, so the everyday answer has to come from the CI system itself; the attempt travels with the id because a re-run of a matrix must not answer the token the first attempt's bundles are already stamped with.
1 fixture
monkeypatch
Set up0.13 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
monkeypatch
Set up0.12 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
monkeypatch
Set up0.12 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
An empty token is not a failure: a merging leg without one merges every bundle beside it, which is what --report-shard-reset is for.
Set up0.12 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
A key whose dest nothing parses into is a setting that does nothing. It would be written onto config.option and read by no one, and the only symptom is a profile that appears to be ignored.
Set up0.12 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
The drift this guards against is silent and goes both ways. "Did the command line say this?" is answered by comparing the option against the default it would hold if nobody had. A default that drifts from the flag's own makes a profile either stop overriding anything - the option never looks unset - or start overriding a value somebody typed.
Set up0.10 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Same drift, one layer down. argparse refuses a bad choice on the command line and cannot see one in a profile, so the profile checks the value itself - against a list that has to stay the flag's list, or a profile starts refusing something the flag accepts.
Set up0.16 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
`logs` and `report_logs` are one setting under two spellings. A profile is very often a block of ini keys moved under a name, so the ini spelling has to work; the short one is what the documented examples use.
Set up0.15 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.13 ms
No steps named here
Description
The whole mechanism, end to end and in one test. Every helper below reads its option exactly the way it did before profiles existed. That they answer the ci profile is the proof that nothing else in the tree needs to know profiles are there.
1 fixture
tmp_path
Set up0.54 ms
No steps named here
Test body1.07 ms
No steps named here
Tear down0.09 ms
No steps named here
Description
pytest.ini gets profiles too, as a section per name. A repository with a pytest.ini and no pyproject.toml should not have to grow one to say "ci" in a single word.
1 fixture
tmp_path
Set up0.54 ms
No steps named here
Test body0.54 ms
No steps named here
Tear down0.08 ms
No steps named here
Description
The documented html_report value has a % in every segment of it. ConfigParser's default interpolation raises on those, so reading these sections with it would fail every project that writes a dated report path - and fail it at configure time, before a single test ran.
1 fixture
tmp_path
Set up0.46 ms
No steps named here
Test body0.49 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
And wins it whole, rather than key by key. The ini file is the one pytest itself picked as this run's configuration. Half a profile from each file would be a shape nobody composed, and which half came from where would depend on which file pytest happened to pick.
1 fixture
tmp_path
Set up0.47 ms
No steps named here
Test body0.39 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
The hyphen is the real name; the underscore is what people type.
1 fixture
tmp_path
Set up0.40 ms
No steps named here
Test body0.16 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
The run everybody has: no profile asked for, nothing to read.
1 fixture
tmp_path
Set up0.39 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.30 ms
No steps named here
Description
Three ways to name a profile, in the order they are documented in.
1 fixture
tmp_path
Set up0.50 ms
No steps named here
Test body0.25 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
So that `pytest` on its own is already the shape this repo agreed on.
1 fixture
tmp_path
Set up0.52 ms
No steps named here
Test body0.29 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
The way out of a pinned default, for the one run that wants none of it. Spelled 'none' rather than as an empty --report-profile= because argparse cannot tell an empty value from an option nobody passed.
1 fixture
tmp_path
Set up0.47 ms
No steps named here
Test body0.23 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
Because that word already means "no profile" on the command line.
1 fixture
tmp_path
Set up0.41 ms
No steps named here
Test body0.18 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
--report-profile=CI should not fail over a capital. And what the report then shows is the name in the file, not the one typed: the file is where somebody goes to read what the run was shaped by.
1 fixture
tmp_path
Set up0.40 ms
No steps named here
Test body0.21 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
For the same reason the resolved path and shard id are. An xdist worker is handed a copy of these options rather than a chance to work the answer out again, and the Environment panel reads the name from here to say which shape produced the report.
1 fixture
tmp_path
Set up0.47 ms
No steps named here
Test body0.22 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
Whoever typed the flag meant this run, and this run is the smaller scope. addopts counts as typed, deliberately: it is a line somebody wrote for this repository about every run of it, and a profile quietly overriding it would make two config files disagree with nothing on the page saying so.
1 fixture
tmp_path
Set up0.67 ms
No steps named here
Test body0.30 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
What an override is for: one job, one machine, one debugging session. Above the profile because the profile is what the repository committed and the variable is somebody saying otherwise without editing a file everybody else reads.
1 fixture
tmp_path
Set up0.45 ms
No steps named here
Test body0.24 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
Because it is the more specific answer, and the selected one.
1 fixture
tmp_path
Set up0.42 ms
No steps named here
Test body0.21 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
A profile is a set of overrides, not a replacement configuration. Everything the profile is silent about goes on being answered the way it was before profiles existed, which is what makes adopting one cheap.
1 fixture
tmp_path
Set up0.41 ms
No steps named here
Test body0.18 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
The reuse half of reusable profiles. What is true of every shape of this run - the title, the coverage tab - is written once, and a profile says only what it changes.
1 fixture
tmp_path
Set up0.44 ms
No steps named here
Test body0.21 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
Otherwise it would be a layer that only sometimes exists. A base that switched itself off when nobody passed --report-profile would make a bare `pytest` and `pytest --report-profile=ci` differ in ways neither file mentions.
1 fixture
tmp_path
Set up0.40 ms
No steps named here
Test body0.16 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Same layer, same name, in the file that project happens to keep.
1 fixture
tmp_path
Set up0.40 ms
No steps named here
Test body0.26 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
build_info, report_link and report_link_pattern all work this way. They are documented as adding to the ini file's rather than replacing it, and a profile is one more place the same list is written from. Replacing would mean a profile that wants to add one row has to restate every row the repository already had.
1 fixture
tmp_path
Set up0.39 ms
No steps named here
Test body0.29 ms
No steps named here
Tear down0.20 ms
No steps named here
Description
`jira = "..."` under a table, rather than a list of "jira=..." strings. It is what anybody writing TOML reaches for first, and the ini file's indented block is what anybody writing ini reaches for; both mean the same MARKER=URL pairs.
1 fixture
tmp_path
Set up0.46 ms
No steps named here
Test body0.21 ms
No steps named here
Tear down0.06 ms
No steps named here
1 fixture
tmp_path
Set up0.56 ms
No steps named here
Test body0.31 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
One commit sha per job is the case this exists for.
1 fixture
tmp_path
Set up0.46 ms
No steps named here
Test body0.23 ms
No steps named here
Tear down0.06 ms
No steps named here
1 fixture
tmp_path
Set up0.39 ms
No steps named here
Test body0.16 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
An ini profile has no booleans, only the text somebody typed.
1 fixture
tmp_path
Set up0.40 ms
No steps named here
Test body0.23 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.56 ms
No steps named here
Test body0.22 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
The only discovery path that exists at the moment somebody needs it.
1 fixture
tmp_path
Set up0.46 ms
No steps named here
Test body0.78 ms
No steps named here
Tear down0.25 ms
No steps named here
1 fixture
tmp_path
Set up0.59 ms
No steps named here
Test body0.33 ms
No steps named here
Tear down0.07 ms
No steps named here
1 fixture
tmp_path
Set up0.53 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
A typo'd key is a setting that silently does nothing, which is worse. Naming the file matters because the run that fails may be a CI job and the file may be one of two that could have held it.
1 fixture
tmp_path
Set up0.44 ms
No steps named here
Test body0.28 ms
No steps named here
Tear down0.06 ms
No steps named here
1 fixture
tmp_path
Set up0.41 ms
No steps named here
Test body0.22 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
The failure this whole feature exists to prevent, in one test. argparse refuses `--report-logs=fail` outright. A profile bypasses argparse entirely, so without this check the value would land on the option, fail the helper's own `in LOG_MODES` test, and fall through to 'all' - a run that kept every log because somebody typed six letters instead of seven.
1 fixture
tmp_path
Set up0.41 ms
No steps named here
Test body0.17 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(value, expected)
2 parameters
expected = takes a numbervalue = "ten"
1 fixture
tmp_path
Set up0.46 ms
No steps named here
Test body0.17 ms
No steps named here
Tear down0.06 ms
No steps named here
1 marker
parametrize(value, expected)
2 parameters
expected = cannot be negativevalue = -1
1 fixture
tmp_path
Set up0.66 ms
No steps named here
Test body0.28 ms
No steps named here
Tear down0.07 ms
No steps named here
1 fixture
tmp_path
Set up0.54 ms
No steps named here
Test body0.24 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
Because "no such profile" about a file that is right there is a bad hour.
1 fixture
tmp_path
Set up0.42 ms
No steps named here
Test body0.21 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
A project configured by pytest.ini may keep one pytest never opens. Refusing to start over a file this plugin has no stake in would be breaking runs that were working - so it is remembered rather than raised, and only surfaces if a profile then cannot be found.
1 fixture
tmp_path
Set up0.41 ms
No steps named here
Test body0.22 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
pytest reads these files more forgivingly than configparser does. A file this cannot parse is not by itself an error - the run started off it - but a --report-profile that then cannot be found has to say where to look, or the profile sitting in that very file reads as missing.
1 fixture
tmp_path
Set up0.43 ms
No steps named here
Test body0.24 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
`PYTEST_HTML_REPORTER_JUNIT=` in a matrix leg means "I am not saying". Reading it as "write no junit" would let an unset variable override a file that does say something, which is the opposite of an override.
Set up0.10 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
It is a namespace anybody can write into, and a guess is a poor reason to fail somebody's CI run.
Set up0.10 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
No file, no --report-profile: the variables are still an override.
1 fixture
tmp_path
Set up0.42 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
Which is what every error message above is built out of.
1 fixture
tmp_path
Set up0.55 ms
No steps named here
Test body0.29 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
One pytest, one --report-profile, and the report shows all of it. The unit tests above drive apply_profile directly, which proves the resolution. This proves the wiring: that it happens early enough in pytest_configure for the settings to be in force by the time anything reads them, and that the report says which shape it was built in.
1 fixture
tmp_path
Set up0.54 ms
No steps named here
Test body379 ms
No steps named here
Tear down0.20 ms
No steps named here
1 fixture
tmp_path
Set up0.66 ms
No steps named here
Test body368 ms
No steps named here
Tear down0.19 ms
No steps named here
Description
At configure time, with one line, the way a bad --archive-count does.
1 fixture
tmp_path
Set up0.66 ms
No steps named here
Test body212 ms
No steps named here
Tear down0.30 ms
No steps named here
Description
Without the addini, `report_profile = ci` in a pytest.ini reads as unset. getini raises, _ini turns that into None, and the profile everybody thinks is pinned silently is not.
Set up0.28 ms
No steps named here
Test body0.11 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
Because an xdist worker is configured from a copy of the controller's. How much of that copy survives the trip is xdist's business rather than something to depend on: a worker that started with the controller's already-extended list and then extended it again would put every build-info row in the report twice.
1 fixture
tmp_path
Set up0.58 ms
No steps named here
Test body0.37 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
The controller renders the report, but the workers run the tests. Which is where the settings that change what is collected in the first place - screenshots, steps, logs - are actually in force, so a profile that reached the controller alone would be a report shaped by one set of answers over records gathered under another.
1 fixture
tmp_path
Set up0.44 ms
No steps named here
Test body680 ms
No steps named here
Tear down0.25 ms
No steps named here
Description
`[tool:pytest]` is how a setup.cfg names pytest's own section. Somebody keeping their profiles there will reach for `[tool:...]` first, and a section that reads exactly right and does nothing is a bad hour.
1 fixture
tmp_path
Set up0.71 ms
No steps named here
Test body0.57 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
The shape somebody writes first, and the panel it used to produce. A profile is very often a block of ini keys moved under a name, and the obvious way to write one is to copy the keys and leave the originals where they are. build_info adds up rather than replaces, so the profile's rows and the ini file's rows both reached the panel and every one of them was rendered twice - two Branch rows sayingโฆ
1 fixture
tmp_path
Set up0.49 ms
No steps named here
Test body0.31 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
Which is the same rule, and the reason it is first-wins rather than a set. The lists are built highest layer first, so collapsing to the first answer per label is what makes a profile outrank the ini key it overlaps - the precedence every other setting already has.
1 fixture
tmp_path
Set up0.49 ms
No steps named here
Test body0.60 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
The same collision, on the dict the patterns are read into. Written straight through, the last entry won - and the last entry is the ini file's, so a profile that repointed `jira` at another tracker was silently overruled by the key it was meant to override.
1 fixture
tmp_path
Set up0.66 ms
No steps named here
Test body0.37 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
The collision a profile is written to cause, and used to lose. A profile that names a row the shared table already names is the ordinary way to write one - the table holds what every shape agrees on and the profile changes the one row that differs.
setting = <tests.unit.test_profiles._ListSetting object at 0x1063d0ad0>
1 fixture
tmp_path
Set up0.62 ms
No steps named here
Test body0.54 ms
No steps named here
Tear down0.10 ms
No steps named here
Description
The collision a profile is written to cause, and used to lose. A profile that names a row the shared table already names is the ordinary way to write one - the table holds what every shape agrees on and the profile changes the one row that differs.
setting = <tests.unit.test_profiles._ListSetting object at 0x1071851d0>
1 fixture
tmp_path
Set up0.58 ms
No steps named here
Test body0.28 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
The collision a profile is written to cause, and used to lose. A profile that names a row the shared table already names is the ordinary way to write one - the table holds what every shape agrees on and the profile changes the one row that differs.
setting = <tests.unit.test_profiles._ListSetting object at 0x107185450>
1 fixture
tmp_path
Set up0.73 ms
No steps named here
Test body0.27 ms
No steps named here
Tear down0.12 ms
No steps named here
Description
The variable is the layer that exists to override the committed files. One job, one machine or one debugging session saying otherwise without editing a file everybody else reads - which it cannot do if the file it is overriding wins.
setting = <tests.unit.test_profiles._ListSetting object at 0x1063d0ad0>
1 fixture
tmp_path
Set up0.56 ms
No steps named here
Test body0.27 ms
No steps named here
Tear down0.08 ms
No steps named here
Description
The variable is the layer that exists to override the committed files. One job, one machine or one debugging session saying otherwise without editing a file everybody else reads - which it cannot do if the file it is overriding wins.
setting = <tests.unit.test_profiles._ListSetting object at 0x1071851d0>
1 fixture
tmp_path
Set up0.54 ms
No steps named here
Test body0.22 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
The variable is the layer that exists to override the committed files. One job, one machine or one debugging session saying otherwise without editing a file everybody else reads - which it cannot do if the file it is overriding wins.
setting = <tests.unit.test_profiles._ListSetting object at 0x107185450>
1 fixture
tmp_path
Set up0.53 ms
No steps named here
Test body0.23 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
Overriding a label is not replacing the list. The rows a lower layer contributed and nobody argued with have to survive, or every profile would have to restate the whole table to change one line of it - which is the thing lists adding up exists to avoid.
setting = <tests.unit.test_profiles._ListSetting object at 0x1063d0ad0>
1 fixture
tmp_path
Set up0.60 ms
No steps named here
Test body0.27 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
Overriding a label is not replacing the list. The rows a lower layer contributed and nobody argued with have to survive, or every profile would have to restate the whole table to change one line of it - which is the thing lists adding up exists to avoid.
setting = <tests.unit.test_profiles._ListSetting object at 0x1071851d0>
1 fixture
tmp_path
Set up0.50 ms
No steps named here
Test body0.21 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
Overriding a label is not replacing the list. The rows a lower layer contributed and nobody argued with have to survive, or every profile would have to restate the whole table to change one line of it - which is the thing lists adding up exists to avoid.
setting = <tests.unit.test_profiles._ListSetting object at 0x107185450>
1 fixture
tmp_path
Set up0.47 ms
No steps named here
Test body0.18 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
A higher layer changes a value; it does not reorder the panel. The shared table is where the rows are written in the order somebody wants to read them down the Environment panel. A profile answering one of them differently should not move that row to the bottom - or, worse, to the top, which is what appending the winning layer's entries would do.
setting = <tests.unit.test_profiles._ListSetting object at 0x1063d0ad0>
1 fixture
tmp_path
Set up0.53 ms
No steps named here
Test body0.23 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
A higher layer changes a value; it does not reorder the panel. The shared table is where the rows are written in the order somebody wants to read them down the Environment panel. A profile answering one of them differently should not move that row to the bottom - or, worse, to the top, which is what appending the winning layer's entries would do.
setting = <tests.unit.test_profiles._ListSetting object at 0x1071851d0>
1 fixture
tmp_path
Set up0.64 ms
No steps named here
Test body0.24 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
A higher layer changes a value; it does not reorder the panel. The shared table is where the rows are written in the order somebody wants to read them down the Environment panel. A profile answering one of them differently should not move that row to the bottom - or, worse, to the top, which is what appending the winning layer's entries would do.
setting = <tests.unit.test_profiles._ListSetting object at 0x107185450>
1 fixture
tmp_path
Set up0.56 ms
No steps named here
Test body0.21 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
Which is what a file saying the same thing twice means. Only worth pinning because the fold that makes a higher layer win is the same fold, and "the last one wins" inside a layer would have made a profile's own entries argue with each other in file order.
setting = <tests.unit.test_profiles._ListSetting object at 0x1063d0ad0>
1 fixture
tmp_path
Set up0.50 ms
No steps named here
Test body0.21 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
Which is what a file saying the same thing twice means. Only worth pinning because the fold that makes a higher layer win is the same fold, and "the last one wins" inside a layer would have made a profile's own entries argue with each other in file order.
setting = <tests.unit.test_profiles._ListSetting object at 0x1071851d0>
1 fixture
tmp_path
Set up0.52 ms
No steps named here
Test body0.19 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
Which is what a file saying the same thing twice means. Only worth pinning because the fold that makes a higher layer win is the same fold, and "the last one wins" inside a layer would have made a profile's own entries argue with each other in file order.
setting = <tests.unit.test_profiles._ListSetting object at 0x107185450>
1 fixture
tmp_path
Set up0.50 ms
No steps named here
Test body0.17 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Case-insensitively, because that is what first_answer_per_key does. A profile writing `Branch` over a shared table's `branch` is one row being overridden, not two rows in a panel disagreeing - and the two ends have to agree about that or the layering settles arguments the reader never sees.
1 fixture
tmp_path
Set up0.54 ms
No steps named here
Test body0.23 ms
No steps named here
Tear down0.06 ms
No steps named here
1 fixture
tmp_path
Set up0.55 ms
No steps named here
Test body0.43 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
The half that answers "why is my setting not the one in the report". A profile listed as overridden is a profile that was found and read and then outranked, which is a different problem - and a different fix - from one that was never read at all.
1 fixture
tmp_path
Set up0.48 ms
No steps named here
Test body0.25 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
And each row its own source, because the layers differ per label.
1 fixture
tmp_path
Set up0.49 ms
No steps named here
Test body0.27 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
Which is the reason explain runs before apply_profile and not after. A profile reaches the run by being written onto config.option - that is the whole mechanism - so once it has been there is nothing left to tell a value somebody typed from a value a profile chose, and an explanation built afterwards would report every one of them as typed.
1 fixture
tmp_path
Set up0.55 ms
No steps named here
Test body0.29 ms
No steps named here
Tear down0.07 ms
No steps named here
1 fixture
tmp_path
Set up0.65 ms
No steps named here
Test body0.12 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
Thirty rows of defaults would bury the two lines somebody came for.
1 fixture
tmp_path
Set up0.49 ms
No steps named here
Test body0.37 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
"Which configuration file was read" is half of every question asked.
1 fixture
tmp_path
Set up0.45 ms
No steps named here
Test body0.23 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
The reader is standing in the project; the absolute path is the noise. Kept absolute on the Explanation itself, because an error about a file somebody has to open should say exactly which one.
1 fixture
tmp_path
Set up0.51 ms
No steps named here
Test body0.28 ms
No steps named here
Tear down0.06 ms
No steps named here
1 fixture
tmp_path
Set up0.53 ms
No steps named here
Test body0.28 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
The point of asking before the run: no forty minutes of CI to find out. StandaloneConfig has to find the same file pytest would and read the same [pytest] section out of it, or the bottom layer of the answer is missing - and a build_info written in [pytest] years ago is exactly the layer somebody has forgotten is there.
1 fixture
tmp_path
Set up0.60 ms
No steps named here
Test body0.78 ms
No steps named here
Tear down0.08 ms
No steps named here
Description
pytest picks pyproject.toml as the config file only when it carries [tool.pytest.ini_options], and this has to agree with it about that.
1 fixture
tmp_path
Set up0.58 ms
No steps named here
Test body0.39 ms
No steps named here
Tear down0.07 ms
No steps named here
1 fixture
tmp_path
Set up0.57 ms
No steps named here
Test body0.22 ms
No steps named here
Tear down0.06 ms
No steps named here
2 fixtures
tmp_pathcapsys
Set up0.62 ms
No steps named here
Test body1.33 ms
No steps named here
Tear down0.09 ms
No steps named here
2 fixtures
tmp_pathcapsys
Set up0.60 ms
No steps named here
Test body1.32 ms
No steps named here
Tear down0.07 ms
No steps named here
2 fixtures
tmp_pathcapsys
Set up0.64 ms
No steps named here
Test body1.43 ms
No steps named here
Tear down0.08 ms
No steps named here
Description
End to end, because the flag's whole job is to be readable in output somebody pastes into an issue when a setting did not take.
1 fixture
tmp_path
Set up0.52 ms
No steps named here
Test body375 ms
No steps named here
Tear down0.24 ms
No steps named here
Set up0.18 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
1 marker
parametrize(name)
1 parameter
name = attach_text
Set up0.15 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.08 ms
No steps named here
1 marker
parametrize(name)
1 parameter
name = attach_json
Set up0.10 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
1 marker
parametrize(name)
1 parameter
name = attach_api
Set up0.10 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
1 marker
parametrize(name)
1 parameter
name = attach_file
Set up0.14 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.06 ms
No steps named here
1 marker
parametrize(name)
1 parameter
name = attach_text
Set up0.14 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.06 ms
No steps named here
1 marker
parametrize(name)
1 parameter
name = attach_json
Set up0.11 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(name)
1 parameter
name = attach_api
Set up0.10 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(name)
1 parameter
name = attach_file
Set up0.10 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Pillow's own error - "cannot identify image file" - says nothing about the mistake, which is now an easy one to make.
Set up0.08 ms
No steps named here
Test body0.16 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
An unset option must not shadow the ini key that was set.
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
`report_open = off` is somebody asking for no browser. Do not open one.
Set up0.09 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.28 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
CI=false is a real thing to write, and means the opposite of CI=true.
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.03 ms
No steps named here
Description
pythonw and friends have no stdout to ask. sys.__stdout__ is replaced rather than None being passed as the argument: None is is_interactive's *default*, so `is_interactive(None)` reads the real stdout of whoever is running the suite - which meant this passed when the run was piped into a file and failed the moment somebody ran it from a terminal, where the real stdout is a tty.
1 fixture
monkeypatch
Set up0.10 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.03 ms
No steps named here
1 fixture
monkeypatch
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
1 fixture
monkeypatch
Set up0.10 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
pytest replaced sys.stdout with a capture long before the report is written.
1 fixture
monkeypatch
Set up0.10 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.10 ms
No steps named here
Description
The one case only the tty check catches: a desktop OS with nobody on it.
Set up0.11 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.03 ms
No steps named here
Description
The escape hatch for a setup the guesswork reads wrongly.
Set up0.07 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.03 ms
No steps named here
1 fixture
tmpdir
Set up0.67 ms
No steps named here
Test body6.26 ms
No steps named here
Tear down0.08 ms
No steps named here
Description
A raw path with a space in it loses everything after the space.
1 fixture
tmpdir
Set up0.54 ms
No steps named here
Test body0.24 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
A run in a loop should not bury the desktop under one window per run.
1 fixture
tmpdir
Set up0.45 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
1 fixture
tmpdir
Set up0.43 ms
No steps named here
Test body0.09 ms
No steps named here
Tear down0.04 ms
No steps named here
1 fixture
tmpdir
Set up0.44 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmpdir
Set up0.49 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
The tests are over and the report is written; a missing browser is not a failure.
1 fixture
tmpdir
Set up0.50 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
webbrowser.open returns False when it found nothing to run.
1 fixture
tmpdir
Set up0.51 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
Still no browser and still no exception, with the run already over.
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Milliseconds alone collided - two tests can finish inside one.
1 fixture
tmp_path
Set up0.55 ms
No steps named here
Test body0.11 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.57 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.06 ms
No steps named here
1 fixture
tmp_path
Set up0.47 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.06 ms
No steps named here
1 fixture
tmp_path
Set up0.45 ms
No steps named here
Test body0.30 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.48 ms
No steps named here
Test body0.24 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
Every automatic one is taken from teardown, with nothing open.
1 fixture
tmp_path
Set up0.56 ms
No steps named here
Test body0.39 ms
No steps named here
Tear down0.09 ms
No steps named here
1 fixture
tmp_path
Set up0.59 ms
No steps named here
Test body0.26 ms
No steps named here
Tear down0.07 ms
No steps named here
1 fixture
tmp_path
Set up0.46 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.45 ms
No steps named here
Test body0.25 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Shards sharing one folder would overwrite each other's pictures.
1 fixture
tmp_path
Set up0.44 ms
No steps named here
Test body0.28 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
Naming the fixture too would be a tooltip repeating the row.
1 fixture
tmp_path
Set up0.55 ms
No steps named here
Test body0.31 ms
No steps named here
Tear down0.09 ms
No steps named here
1 fixture
tmp_path
Set up1.07 ms
No steps named here
Test body0.14 ms
No steps named here
Tear down0.11 ms
No steps named here
1 fixture
tmp_path
Set up0.95 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
A response body in a cell would be swept into the search index and into every CSV, Excel and print export.
1 fixture
tmp_path
Set up0.56 ms
No steps named here
Test body0.45 ms
No steps named here
Tear down0.07 ms
No steps named here
1 fixture
tmp_path
Set up0.83 ms
No steps named here
Test body0.59 ms
No steps named here
Tear down0.15 ms
No steps named here
1 fixture
tmp_path
Set up1.39 ms
No steps named here
Test body0.40 ms
No steps named here
Tear down0.10 ms
No steps named here
1 fixture
tmp_path
Set up1.09 ms
No steps named here
Test body0.14 ms
No steps named here
Tear down0.10 ms
No steps named here
1 fixture
tmp_path
Set up0.93 ms
No steps named here
Test body0.14 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
They are thousands of characters each; repeating them would double the size of the file to no end.
1 fixture
tmp_path
Set up0.62 ms
No steps named here
Test body0.11 ms
No steps named here
Tear down0.07 ms
No steps named here
1 fixture
tmp_path
Set up0.64 ms
No steps named here
Test body0.12 ms
No steps named here
Tear down0.06 ms
No steps named here
1 fixture
tmp_path
Set up0.79 ms
No steps named here
Test body0.10 ms
No steps named here
Tear down0.06 ms
No steps named here
1 fixture
tmp_path
Set up0.50 ms
No steps named here
Test body0.11 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
pytest-bdd never calls after_step for a failure, so this is its close.
1 fixture
tmp_path
Set up0.50 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Nothing opened it, so the tab only names that line if this does both.
1 fixture
tmp_path
Set up0.59 ms
No steps named here
Test body0.11 ms
No steps named here
Tear down0.07 ms
No steps named here
1 fixture
tmp_path
Set up0.63 ms
No steps named here
Test body0.10 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
An attachment left behind is reported as the next test's own - the bug screenshots had before every record started claiming the pending image.
1 fixture
tmp_path
Set up0.57 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.06 ms
No steps named here
1 fixture
tmp_path
Set up0.49 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.06 ms
No steps named here
1 marker
parametrize(status)
1 parameter
status = FAIL
1 fixture
tmp_path
Set up0.53 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.08 ms
No steps named here
1 marker
parametrize(status)
1 parameter
status = ERROR
1 fixture
tmp_path
Set up0.65 ms
No steps named here
Test body0.13 ms
No steps named here
Tear down0.07 ms
No steps named here
1 fixture
tmp_path
Set up0.40 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.38 ms
No steps named here
Test body0.44 ms
No steps named here
Tear down0.06 ms
No steps named here
1 fixture
tmp_path
Set up0.41 ms
No steps named here
Test body0.55 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
A raise here would cost the run its html report, its output.json and its archived build as well as its xml - over a mistyped path.
2 fixtures
tmp_pathcapsys
Set up0.43 ms
No steps named here
Test body0.38 ms
No steps named here
Tear down0.06 ms
No steps named here
1 fixture
tmp_path
Set up0.41 ms
No steps named here
Test body0.39 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
The case the whole feature exists for. A test that fails and then passes is reported as passed, and the record left standing carries no message - so without the trail there is nothing anywhere in the report that says what it failed with.
Set up0.11 ms
No steps named here
Test body0.15 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.19 ms
No steps named here
Test body0.15 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
The count and the trail are two spellings of one fact, so they agree. A row showing three reruns and a panel showing two attempts is worse than either alone: it says the report lost one.
Set up0.11 ms
No steps named here
Test body0.22 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
An xdist worker sends back a record that was folded on the worker. Both sides can already stand for several attempts, and the record being replaced belongs between the two lists: it ran after its own attempts and before the one replacing it.
Set up0.08 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
A shard bundle is the record list as it stands in memory. Keeping whole attempts would multiply every bundle a matrix uploads by the number of times its flakiest tests were retried, and put a copy of each discarded attempt's logs and screenshots in it.
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Which is nearly every row, and is what keeps the key from costing anything.
Set up0.09 ms
No steps named here
Test body0.09 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Two records for one node id are two tests - pytest-repeat, not a retry.
Set up0.08 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Tolerant for the reason normalise_record is. The attempts inside a bundle are checked as a list and not each one as a record, so an unreadable duration is a decoration on a panel and no reason to lose the report the tests have already paid for.
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
An older shard artifact is missing the key rather than carrying an empty one.
Set up0.08 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
The trail is the superseded attempts *and then the record itself*. Stopping at the last attempt that was thrown away would leave a panel whose final line disagrees with the row that opened it.
Set up0.08 ms
No steps named here
Test body1.23 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body9.59 ms
No steps named here
Tear down0.09 ms
No steps named here
Set up0.14 ms
No steps named here
Test body0.62 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
The cell's text is what the search index and every export take. Two spellings of the count - a button beside a hidden span - would put the number in the CSV, the Excel sheet and the print-out twice.
Set up0.09 ms
No steps named here
Test body0.69 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.60 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
This panel is where a superseded attempt's full text is read. There is nowhere else left to read it: the row shows the message of the attempt that stuck, which on a test that ends up passing is empty.
Set up0.19 ms
No steps named here
Test body0.83 ms
No steps named here
Tear down0.06 ms
No steps named here
1 fixture
tmp_path
Set up0.49 ms
No steps named here
Test body425 ms
No steps named here
Tear down0.24 ms
No steps named here
1 fixture
tmp_path
Set up0.75 ms
No steps named here
Test body409 ms
No steps named here
Tear down0.21 ms
No steps named here
Set up0.18 ms
No steps named here
Test body0.17 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.10 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.09 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.11 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.15 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.12 ms
No steps named here
Tear down0.06 ms
No steps named here
1 fixture
tmp_path
Set up0.53 ms
No steps named here
Test body460 ms
No steps named here
Tear down0.19 ms
No steps named here
1 fixture
tmp_path
Set up0.75 ms
No steps named here
Test body474 ms
No steps named here
Tear down0.24 ms
No steps named here
1 fixture
tmp_path
Set up0.75 ms
No steps named here
Test body407 ms
No steps named here
Tear down0.23 ms
No steps named here
1 fixture
tmp_path
Set up0.73 ms
No steps named here
Test body390 ms
No steps named here
Tear down0.21 ms
No steps named here
Set up0.24 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
status_list holds its counts as strings; tfail mixes both.
Set up0.10 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Nothing to compare against is not the same as no change.
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
And drops the suffix: "no change" already implies the comparison, and the extra words wrap the line onto a second row in this column.
Set up0.08 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
The opposite way round to coverage, where up is the good direction.
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Caption included - a lone "SINCE LAST BUILD" over nothing reads as a bug.
Set up0.52 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Any number here takes the unit beside it, and "0 failures" - or "ยฑ0 failures" - says the opposite of what it means: no failures at all, rather than no change in them.
Set up0.10 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
The figure is already a whole phrase, and the noun is the half that misleads.
Set up0.09 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
'+3' reads differently against 3 than against 300.
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
1 fixture
monkeypatch
Set up0.11 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
is-empty hides the tile, and nothing is left over from a previous run.
1 fixture
monkeypatch
Set up0.24 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
The build at index 0 is this build, whatever rendered before it. The lists live on ConfigVars and update_trends appends to them, so a merge driven in-process - or anything else that renders twice - used to leave its own build sitting where this one's belongs. It was drawn as the newest point on the Trends chart, and the delta beside it was measured against it.
1 fixture
tmp_path
Set up0.58 ms
No steps named here
Test body0.78 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
The bug as it was seen: "+1 failure since last build" over a green run. The 1 was the earlier render's failure, and the build it was subtracted from was this one. A build with no archive behind it has nothing to compare against and the tile stays hidden.
1 fixture
tmp_path
Set up0.45 ms
No steps named here
Test body0.50 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
The Test Steps tab shows a picture under the step it was taken in.
Set up0.09 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
-1 is every automatic capture: teardown has nothing open by then.
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
An image nobody claims must not be picked up by the next test.
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Already a png - decoding it only to encode it again is time wasted.
1 fixture
tmp_path
Set up0.51 ms
No steps named here
Test body0.21 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.42 ms
No steps named here
Test body0.90 ms
No steps named here
Tear down0.08 ms
No steps named here
Set up0.17 ms
No steps named here
Test body0.13 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
Pillow's 'cannot identify image file' says nothing about the mistake.
Set up0.11 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
A crashed browser is not worth failing a test that already ran over.
Set up0.08 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Unawaited, it prints a warning against the test; the report says nothing.
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Closing it is a courtesy; failing to is not worth raising over.
Set up0.08 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
splinter's browser.screenshot() writes a file and returns its name.
Set up0.08 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.23 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.07 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
splinter's own screenshot() returns a path; its driver returns bytes.
Set up0.09 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.21 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
It answers every call, so it would be shot on a method it lacks - and the call it records may be the very thing the test asserts on.
Set up0.13 ms
No steps named here
Test body0.15 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.14 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
A suite driving a context rather than a page never asks for one.
Set up0.10 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.17 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
A pytest-bdd test takes no fixtures at all - the steps ask as they run.
Set up0.10 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
getfixturevalue would start a browser at teardown just to shoot it.
Set up0.10 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
page before the rest: it is the one worth photographing.
Set up0.16 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
A suite whose browser fixture is called something else is covered.
Set up0.15 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.07 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
setUp puts it on the instance rather than in a fixture.
Set up0.10 ms
No steps named here
Test body0.11 ms
No steps named here
Tear down0.07 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Asking for page, context and browser is the ordinary Playwright test.
Set up0.08 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
A row of thumbnails stops being readable long before it stops growing.
Set up0.15 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
The recipe everyone writes first: attach from an autouse fixture. The reporter builds its record from its own teardown hook, so this only works because that hook now waits for the fixture finalizers to run.
1 fixture
tmp_path
Set up0.61 ms
No steps named here
Test body410 ms
No steps named here
Tear down0.19 ms
No steps named here
Description
The other documented recipe has to keep working.
1 fixture
tmp_path
Set up0.74 ms
No steps named here
Test body410 ms
No steps named here
Tear down0.21 ms
No steps named here
Description
An attached image is kept whatever the test did, not only on failure.
1 fixture
tmp_path
Set up0.78 ms
No steps named here
Test body412 ms
No steps named here
Tear down0.21 ms
No steps named here
Description
How the test ended is the card's badge, and the badge is what the filter pills are keyed on - so a passing screenshot has to carry it even though it has no error to print underneath.
1 fixture
tmp_path
Set up0.76 ms
No steps named here
Test body391 ms
No steps named here
Tear down0.19 ms
No steps named here
1 fixture
tmp_path
Set up0.74 ms
No steps named here
Test body455 ms
No steps named here
Tear down0.20 ms
No steps named here
Description
Each image belongs to the test that attached it, and to no other. The failing test here attaches nothing, so it must show nothing - rather than being illustrated with the passing test's picture.
1 fixture
tmp_path
Set up0.72 ms
No steps named here
Test body469 ms
No steps named here
Tear down0.21 ms
No steps named here
Description
Two failures, two images - neither test borrowing the other's.
1 fixture
tmp_path
Set up0.91 ms
No steps named here
Test body408 ms
No steps named here
Tear down0.24 ms
No steps named here
Description
Only the tests that actually attached appear in the gallery.
1 fixture
tmp_path
Set up0.70 ms
No steps named here
Test body402 ms
No steps named here
Tear down0.22 ms
No steps named here
Description
It used to be cut to nineteen characters to fit a caption strip, which left test_login_page_renders as "test_login_page_r.." on the one line that was supposed to name it. The card gives it a line and an ellipsis of its own, so the report keeps the whole name and lets CSS end it.
1 fixture
tmp_path
Set up0.72 ms
No steps named here
Test body405 ms
No steps named here
Tear down0.22 ms
No steps named here
Description
Crossing to another tab to find out what the failure looked like is the step the column removes: the thumbnail is on the row that failed.
1 fixture
tmp_path
Set up0.91 ms
No steps named here
Test body481 ms
No steps named here
Tear down0.23 ms
No steps named here
1 fixture
tmp_path
Set up0.72 ms
No steps named here
Test body417 ms
No steps named here
Tear down0.23 ms
No steps named here
Description
Not the gallery's. Arrowing off a row's screenshot and into one from a tab the reader is not looking at is not what the arrow keys are for.
1 fixture
tmp_path
Set up0.82 ms
No steps named here
Test body434 ms
No steps named here
Tear down0.21 ms
No steps named here
1 fixture
tmp_path
Set up0.74 ms
No steps named here
Test body421 ms
No steps named here
Tear down0.21 ms
No steps named here
1 fixture
tmp_path
Set up0.83 ms
No steps named here
Test body31 ms
No steps named here
Tear down0.24 ms
No steps named here
Description
Not ConfigVars._start_execution_time - that is reset by every pytest_runtest_setup, so by report time it holds the last test's setup.
1 fixture
tmp_path
Set up0.72 ms
No steps named here
Test body29 ms
No steps named here
Tear down0.21 ms
No steps named here
1 fixture
tmp_path
Set up0.75 ms
No steps named here
Test body59 ms
No steps named here
Tear down0.22 ms
No steps named here
Description
_collected is filled by pytest_collection_modifyitems, which never fires on an xdist controller - the workers collect - so an `-n 4` leg would write 0 beside a bundle holding four hundred records.
1 fixture
tmp_path
Set up0.74 ms
No steps named here
Test body29 ms
No steps named here
Tear down0.21 ms
No steps named here
1 fixture
tmp_path
Set up0.70 ms
No steps named here
Test body29 ms
No steps named here
Tear down0.22 ms
No steps named here
1 fixture
tmp_path
Set up0.71 ms
No steps named here
Test body30 ms
No steps named here
Tear down0.21 ms
No steps named here
Description
The merge very often runs in a job of its own: a merging step asking its own environment which CI run this was would answer with the merge job, and one running on a laptop would answer with nothing at all.
2 fixtures
tmp_pathmonkeypatch
Set up0.74 ms
No steps named here
Test body29 ms
No steps named here
Tear down0.21 ms
No steps named here
2 fixtures
tmp_pathmonkeypatch
Set up0.75 ms
No steps named here
Test body0.31 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
A merge reading an old bundle beside a new one has to put the two legs' rows side by side without either going blank, so the key every older bundle carries is still written.
1 fixture
tmp_path
Set up0.61 ms
No steps named here
Test body30 ms
No steps named here
Tear down0.25 ms
No steps named here
1 fixture
tmp_path
Set up0.76 ms
No steps named here
Test body32 ms
No steps named here
Tear down0.20 ms
No steps named here
Description
Collected on the shard for the same reason as everything else here - the merging machine's site-packages is not the one the tests imported from.
1 fixture
tmp_path
Set up0.78 ms
No steps named here
Test body62 ms
No steps named here
Tear down0.18 ms
No steps named here
Description
A merge reading them back as lists and comparing against tuples would find no build info anywhere.
1 fixture
tmp_path
Set up0.77 ms
No steps named here
Test body32 ms
No steps named here
Tear down0.26 ms
No steps named here
Description
Under the merge shim getoption('capture') is None, so the '-s suppressed stdout' explanation would silently vanish from a merged report.
1 fixture
tmp_path
Set up0.73 ms
No steps named here
Test body30 ms
No steps named here
Tear down0.21 ms
No steps named here
1 fixture
tmp_path
Set up0.72 ms
No steps named here
Test body29 ms
No steps named here
Tear down0.19 ms
No steps named here
1 fixture
tmp_path
Set up0.69 ms
No steps named here
Test body29 ms
No steps named here
Tear down0.18 ms
No steps named here
1 fixture
tmp_path
Set up0.70 ms
No steps named here
Test body28 ms
No steps named here
Tear down0.19 ms
No steps named here
1 fixture
tmp_path
Set up0.70 ms
No steps named here
Test body27 ms
No steps named here
Tear down0.19 ms
No steps named here
1 fixture
tmp_path
Set up1.07 ms
No steps named here
Test body30 ms
No steps named here
Tear down0.25 ms
No steps named here
Description
Whatever this version puts in a record is what the next one reads out.
1 fixture
tmp_path
Set up0.77 ms
No steps named here
Test body30 ms
No steps named here
Tear down0.21 ms
No steps named here
1 fixture
tmp_path
Set up0.76 ms
No steps named here
Test body27 ms
No steps named here
Tear down0.24 ms
No steps named here
Description
It is relative to this shard's report base, and the merged report is written somewhere else entirely - the link would go nowhere.
1 fixture
tmp_path
Set up0.73 ms
No steps named here
Test body29 ms
No steps named here
Tear down0.18 ms
No steps named here
Description
Losing the tile is a decoration; failing the leg loses the records.
1 fixture
tmp_path
Set up0.72 ms
No steps named here
Test body29 ms
No steps named here
Tear down0.21 ms
No steps named here
1 fixture
tmp_path
Set up0.72 ms
No steps named here
Test body29 ms
No steps named here
Tear down0.19 ms
No steps named here
1 fixture
tmp_path
Set up0.72 ms
No steps named here
Test body30 ms
No steps named here
Tear down0.20 ms
No steps named here
Description
The round trip is the whole contract between a leg and the merge.
1 fixture
tmp_path
Set up0.75 ms
No steps named here
Test body28 ms
No steps named here
Tear down0.22 ms
No steps named here
Description
Three legs point at one --html-report, and the next CI run points the same three at it again.
2 fixtures
tmp_pathcapsys
Set up0.78 ms
No steps named here
Test body28 ms
No steps named here
Tear down0.19 ms
No steps named here
Description
"1/4" and "1-4" both sanitise to "1-4"; without this the second leg replaces the first's records and the merge reports a matrix a quarter smaller with nothing saying a leg was lost.
2 fixtures
tmp_pathcapsys
Set up0.71 ms
No steps named here
Test body56 ms
No steps named here
Tear down0.24 ms
No steps named here
2 fixtures
tmp_pathcapsys
Set up0.80 ms
No steps named here
Test body29 ms
No steps named here
Tear down0.21 ms
No steps named here
Set up0.16 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
A bundle written by a version that had not grown a key yet must answer the default rather than raise from inside a comprehension.
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
The whole point of a leg: records for the merge, and no build. Four legs that each wrote a build would leave four trend points, four archive files and four Analytics entries behind for one run, with nothing on disk afterwards saying which four belonged together.
1 fixture
tmp_path
Set up0.63 ms
No steps named here
Test body278 ms
No steps named here
Tear down0.23 ms
No steps named here
Description
Lossless, because the merge indexes these keys without asking. build_report sorts on a hard record['index'] and the row builders index eight more, so a bundle that dropped a key would not degrade the merged report - it would end the merge in a traceback after every test in the matrix had already run.
1 fixture
tmp_path
Set up0.82 ms
No steps named here
Test body277 ms
No steps named here
Tear down0.24 ms
No steps named here
Description
An `-n 2` leg is one leg of the matrix, however many processes it ran in. The bundle is written after the xdist worker guard, so the controller - which by then holds both workers' records, already merged - writes it once rather than each worker writing an eighth of the tests over the top of the others.
1 fixture
tmp_path
Set up0.79 ms
No steps named here
Test body646 ms
No steps named here
Tear down0.24 ms
No steps named here
Description
`collected` is the first number somebody opening a bundle looks at. It used to be len(reporter._collected), which is empty on an xdist controller because pytest_collection_modifyitems never fires there - the workers do the collecting - so an `-n 4` leg wrote 0 beside a bundle holding four hundred records.
1 fixture
tmp_path
Set up0.85 ms
No steps named here
Test body723 ms
No steps named here
Tear down0.22 ms
No steps named here
Description
The images travel inside the bundle, because the record only names them. collect_screenshots stores a bare name with no path and no extension, so a picture that stayed in the report folder would be a picture the merge could not find from the artifact it was handed.
1 fixture
tmp_path
Set up0.81 ms
No steps named here
Test body303 ms
No steps named here
Tear down0.22 ms
No steps named here
Description
Two legs, one --html-report, and neither may sweep the other's images. The legs of a sequential run are pointed at one report folder on purpose, so a leg that cleaned the folder rather than its own directory would leave the merge holding records that name pictures no longer on disk.
1 fixture
tmp_path
Set up0.85 ms
No steps named here
Test body577 ms
No steps named here
Tear down0.31 ms
No steps named here
Description
An id made entirely of separators is a usage error, not a fallback. Sanitising "//" leaves nothing, and a shard whose id came out empty would file its bundle in the report base and empty the report's own screenshot folder on the way.
1 fixture
tmp_path
Set up0.83 ms
No steps named here
Test body227 ms
No steps named here
Tear down0.23 ms
No steps named here
Description
A leg named v1.html gets the same fresh directory as every other leg. The branch used to clean only this directory's pytest_screenshots folder, which leaves everything else in it behind - a half-written temp file from a killed run, a records.json from a run that collected more tests than this one does. A leg owns its whole directory, and everything in it belongs to the run starting now, which is โฆ
1 fixture
tmp_path
Set up0.89 ms
No steps named here
Test body280 ms
No steps named here
Tear down0.22 ms
No steps named here
Description
"1/4" and "1-4" name one directory, and the second leg buries the first. Overwriting is the everyday case and is right - the same leg re-runs, and the next CI run points the same legs at the same folder. Two *different* legs landing on one directory is not, and the report would otherwise show a matrix a quarter smaller with nothing anywhere saying a leg had been lost.
2 fixtures
tmp_pathcapsys
Set up0.91 ms
No steps named here
Test body0.98 ms
No steps named here
Tear down0.09 ms
No steps named here
Description
A killed job's half file must not cost the matrix the other three legs.
1 fixture
tmp_path
Set up0.54 ms
No steps named here
Test body0.58 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
Named and refused, never read hopefully. Silently dropping a quarter of a matrix produces a report that is wrong in a way nobody looking at it can see.
1 fixture
tmp_path
Set up0.54 ms
No steps named here
Test body0.40 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
A stale artifact from a newer release is a message, not an INTERNALERROR. merge_into runs from pytest_terminal_summary, after every test has already finished, so a raise there costs the run its report, its output.json and its verdict - over a file somebody left in a folder.
1 fixture
tmp_path
Set up0.60 ms
No steps named here
Test body286 ms
No steps named here
Tear down0.21 ms
No steps named here
Description
A bundle from another release of the plugin still renders. build_report sorts on record['index'] and record['worker'] with no get(), so a record carrying the index as a string and no worker at all is exactly the shape that raises a TypeError in the middle of a merge.
1 fixture
tmp_path
Set up0.74 ms
No steps named here
Test body133 ms
No steps named here
Tear down0.11 ms
No steps named here
Description
Dropped, counted and named, rather than taking the merge down with it. A record with no node id cannot be grouped against a duplicate, ordered or linked to - and this is found after the whole matrix has finished, which is the worst moment there is to raise.
1 fixture
tmp_path
Set up0.65 ms
No steps named here
Test body105 ms
No steps named here
Tear down0.11 ms
No steps named here
Description
A field a newer release added is carried, not dropped. The bundle is the interchange format between two installations of this plugin, and a merge that discarded what it did not recognise would make every upgrade lossy in one direction.
1 fixture
tmp_path
Set up0.66 ms
No steps named here
Test body0.58 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
The same four artifacts make the same report whatever order they arrive in. Two CI jobs downloading the same bundles in different orders have to produce the same rows, or the diff between two builds is unreadable - and "2-4" has to sort before "10-16", which plain string order does not do.
1 fixture
tmp_path
Set up0.53 ms
No steps named here
Test body330 ms
No steps named here
Tear down0.11 ms
No steps named here
Description
One file split over three machines is one row in Suite Metrics. A suite counted once per shard would report three suites where the run had one, and Suite Highlights and Analytics both count off that.
1 fixture
tmp_path
Set up0.68 ms
No steps named here
Test body109 ms
No steps named here
Tear down0.11 ms
No steps named here
Description
A file that would not import is read before the tests that did run. A collection failure is given a negative index by the plugin for exactly this reason, and four shards renumbering from zero would otherwise bury it in the middle of the table.
1 fixture
tmp_path
Set up0.66 ms
No steps named here
Test body108 ms
No steps named here
Tear down0.11 ms
No steps named here
Description
Every leg collects the whole suite, so a broken import is in every bundle. Four copies of one collection failure is not a matrix that broke four times, and counting it four times would put three tests into the totals that never existed.
1 fixture
tmp_path
Set up0.74 ms
No steps named here
Test body112 ms
No steps named here
Tear down0.10 ms
No steps named here
Description
A file that failed to import on one machine failed to import, full stop. Another leg skipping it - a platform marker, a missing optional dependency - does not make the import work, so the error is the record worth keeping and the reason worth showing.
1 fixture
tmp_path
Set up0.71 ms
No steps named here
Test body117 ms
No steps named here
Tear down0.12 ms
No steps named here
Description
One test checked out at two paths is one test, once --strip-path-prefix says so. A container that built at /src and a runner that used /home/runner/work report the same test under two spellings, which is two rows, two Analytics histories and two entries in the JUnit file.
1 fixture
tmp_path
Set up0.65 ms
No steps named here
Test body0.79 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
One test that ran in two shards is one row that says it ran twice. An overlapping matrix is a mistake worth seeing, and the honest shape of it is the row the report already has for a retried test.
1 fixture
tmp_path
Set up0.55 ms
No steps named here
Test body104 ms
No steps named here
Tear down0.10 ms
No steps named here
Description
The fold counts the attempts, and the trail says what they did. A test that failed on one shard and passed on another leaves a row saying PASS with no message of its own, so the failure is only anywhere in the report if the dropped attempt was kept.
1 fixture
tmp_path
Set up0.65 ms
No steps named here
Test body0.63 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
Two folds stack: the retry inside a shard, then the shards themselves. The count has always summed both. The trail has to as well, or a report can say four attempts and show two.
1 fixture
tmp_path
Set up0.55 ms
No steps named here
Test body0.55 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
first, last and worst are 'this shard's answer', not a fold. Manufacturing a trail under them would report attempts the chosen record never made.
1 fixture
tmp_path
Set up0.72 ms
No steps named here
Test body1.15 ms
No steps named here
Tear down0.08 ms
No steps named here
Description
The answer is the same whatever happens to be installed on this machine. store_test_record folds attempts only when pytest-rerunfailures is loaded, which is a decision about the *merging* process and has nothing to do with the matrix, so the merge never routes duplicates through it.
2 fixtures
tmp_pathmonkeypatch
Set up0.68 ms
No steps named here
Test body217 ms
No steps named here
Tear down0.11 ms
No steps named here
Description
The three policies that keep one attempt keep the one they are named for.
1 fixture
tmp_path
Set up0.68 ms
No steps named here
Test body1.17 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
A pipeline that says an overlap is a bug is told which test and which legs. "the matrix overlaps" is not an answer anybody can act on; the node id and the two shard ids are.
1 fixture
tmp_path
Set up0.55 ms
No steps named here
Test body0.55 ms
No steps named here
Tear down0.22 ms
No steps named here
Description
The survivor keeps its own step list, not two shards' lists end to end. A step is addressed by its position in its own record's list, and both attachments and screenshots point at a step by that number - so splicing two records' steps together re-points every one of them at the wrong step.
1 fixture
tmp_path
Set up0.57 ms
No steps named here
Test body0.83 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
Two legs' first screenshots are both "<ms>-1", and both have to survive. screenshot_name() is milliseconds plus a counter that restarts at 1 in every process, so a shared folder would leave one machine's picture on the other machine's row.
1 fixture
tmp_path
Set up0.62 ms
No steps named here
Test body108 ms
No steps named here
Tear down0.13 ms
No steps named here
Description
The picture of the only failure in the matrix is not thrown away. Under the default policy the survivor is the last shard's record and its empty `screenshots` is back-filled from a loser - so a test photographed when it failed on s1 and then passed on s2 becomes an s2 row holding an s1 image. Looking that up under the *row's* shard finds nothing and drops it as missing while it is sitting on diskโฆ
1 fixture
tmp_path
Set up0.69 ms
No steps named here
Test body121 ms
No steps named here
Tear down0.11 ms
No steps named here
Description
A card with a broken image reads as a bug in the report. The note says which file the bundle promised and did not carry, which is the only thing that can be acted on.
1 fixture
tmp_path
Set up0.77 ms
No steps named here
Test body106 ms
No steps named here
Tear down0.11 ms
No steps named here
Description
One field feeds the row strip, the Screenshots card and the step tree. All three build "pytest_screenshots/" + the record's name + ".png" by plain concatenation, so a name that was rewritten for one of them and not the others would leave two of the three pointing nowhere.
1 fixture
tmp_path
Set up0.68 ms
No steps named here
Test body105 ms
No steps named here
Tear down0.11 ms
No steps named here
Description
The dashboard, the totals and the table have to agree with each other.
1 fixture
tmp_path
Set up0.68 ms
No steps named here
Test body114 ms
No steps named here
Tear down0.11 ms
No steps named here
Description
The second build in one process is a build, not the first one doubled. generate_json_data accumulates its counters with += and the template renders that family rather than the assigned one, so a merge that did not reset them would double every number on the dashboard and raise nothing.
1 fixture
tmp_path
Set up0.73 ms
No steps named here
Test body222 ms
No steps named here
Tear down0.11 ms
No steps named here
Description
Three legs merged twice leave two builds in the history, not six. The archive, the trend and Analytics are all counted per build, so a matrix that rotated once per shard would show four builds a day where the project ran one.
1 fixture
tmp_path
Set up0.71 ms
No steps named here
Test body213 ms
No steps named here
Tear down0.12 ms
No steps named here
Description
The build is dated when the tests ran, not when somebody merged them. That timestamp names the file the next build archives this one as, labels the point on the trend chart and orders the builds in Analytics, so a matrix that started before midnight and was merged after it must not be filed on the wrong day.
1 fixture
tmp_path
Set up0.68 ms
No steps named here
Test body107 ms
No steps named here
Tear down0.11 ms
No steps named here
Description
The one panel on a merged report that could otherwise not be honest. Filled from the merging machine it would describe a process that ran none of the tests, on a platform the suite may never have touched.
1 fixture
tmp_path
Set up0.65 ms
No steps named here
Test body107 ms
No steps named here
Tear down0.10 ms
No steps named here
Description
The explanation for an empty Logs column comes from the leg that ran with -s. The merge asks nothing of its own configuration here: under the shim getoption("capture") is None, so this sentence would silently vanish and leave a column of dashes with nothing to explain it.
1 fixture
tmp_path
Set up0.69 ms
No steps named here
Test body105 ms
No steps named here
Tear down0.12 ms
No steps named here
Description
No key at all, so the build reads as "not measured" rather than as zero. A zero would be archived, and the trend chart would carry it for as long as the archive is kept. Four shards' percentages are never averaged either, which is why the notice names `coverage combine`.
1 fixture
tmp_path
Set up0.72 ms
No steps named here
Test body107 ms
No steps named here
Tear down0.11 ms
No steps named here
Description
One already-combined artifact, named by the operator, is the whole answer. discover_coverage is never called by a merge: it searches the working directory and stops at the first hit, so a stale coverage.json on the merging machine would become this build's number.
1 fixture
tmp_path
Set up0.68 ms
No steps named here
Test body106 ms
No steps named here
Tear down0.10 ms
No steps named here
Description
Two legs' data files combined measure more than either one did. Each file below covers half of the module, and the same half is not counted twice: the merged figure is the union, which is the number an average of the two percentages could never arrive at.
1 fixture
tmp_path
Set up0.66 ms
No steps named here
Test body114 ms
No steps named here
Tear down0.11 ms
No steps named here
Description
The usual cli-beats-ini pair, for the value that decides what gets merged.
1 fixture
tmp_path
Set up0.69 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Nobody types --report-shard-run on a matrix, so the CI job's own id answers.
1 fixture
monkeypatch
Set up0.10 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
And a leg with no token merges everything, as it always has. An empty token says nothing about which run a bundle came from, so refusing to merge on the strength of it would break every matrix that does not run on a CI system this plugin recognises. That case is --report-shard-reset's.
1 fixture
monkeypatch
Set up0.10 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
GITHUB_RUN_ID deliberately survives a re-run, and the attempt number does not. Without the attempt the second try would merge the first try's bundles, and the tests it re-ran would appear twice.
1 fixture
tmp_path
Set up0.59 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
Jenkins build 41 and Drone build 41 are not the same run.
1 fixture
tmp_path
Set up0.54 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Yesterday's leg is put aside and named, not counted into today's totals. The legs of a sequential run are pointed at one persistent --html-report, so <base>/shards accumulates; a leg that was renamed or dropped since the last run leaves its bundle sitting there, and the failure it causes is invisible - the totals are simply larger than the run was.
1 fixture
tmp_path
Set up0.64 ms
No steps named here
Test body395 ms
No steps named here
Tear down0.23 ms
No steps named here
Description
The deterministic answer for a run that has no token to filter on. The first leg of the run says so, and whatever the last run left behind is gone before this one writes a byte. Never implied by --report-shard or by --report-shard-merge, because it deletes the other legs' work.
1 fixture
tmp_path
Set up0.83 ms
No steps named here
Test body397 ms
No steps named here
Tear down0.24 ms
No steps named here
Description
Every build says out loud which bundles it was made of, and when they ran. There is no honest way to decide from inside a merge that a bundle is stale - every bundle beside a merging leg was written before it - so nothing is guessed and the times are simply shown. That is what turns a leftover leg from two silent extra tests into a line in the CI log.
1 fixture
tmp_path
Set up0.81 ms
No steps named here
Test body396 ms
No steps named here
Tear down0.22 ms
No steps named here
Description
Two machines' artifacts in a folder, one command, one report. The shape a CI matrix actually has: each leg uploads its bundle, a final job downloads them side by side and merges what it was given.
2 fixtures
tmp_pathcapsys
Set up0.92 ms
No steps named here
Test body698 ms
No steps named here
Tear down0.21 ms
No steps named here
Description
Two halves merged say exactly what the whole suite said in one process. The sharded sibling of the xdist test that compares a parallel run against a serial one: splitting the work must change how long it takes and nothing else.
1 fixture
tmp_path
Set up0.74 ms
No steps named here
Test body1.07 s
No steps named here
Tear down0.20 ms
No steps named here
Description
Three commands on one machine, one build - not three, and not four. The last leg merges the two beside it and renders, so the sequential flow needs no fourth command; the two before it still wrote no build of their own, which is what keeps the archive honest.
1 fixture
tmp_path
Set up0.52 ms
No steps named here
Test body1.08 s
No steps named here
Tear down0.23 ms
No steps named here
Description
The one test that cannot be confused with a bundle holding no records.
Set up0.17 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Otherwise it falls through and picks up somebody else's variable.
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.12 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
1 marker
parametrize(value)
1 parameter
value = true
Set up0.12 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.06 ms
No steps named here
1 marker
parametrize(value)
1 parameter
value = True
Set up0.11 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
1 marker
parametrize(value)
1 parameter
value = 1
Set up0.10 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
1 marker
parametrize(value)
1 parameter
value = yes
Set up0.09 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
1 marker
parametrize(value)
1 parameter
value = on
Set up0.10 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
1 marker
parametrize(value)
1 parameter
value = false
Set up0.11 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(value)
1 parameter
value = 0
Set up0.17 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.06 ms
No steps named here
1 marker
parametrize(value)
1 parameter
value = no
Set up0.12 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(value)
1 parameter
value =
Set up0.09 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(value)
1 parameter
value = maybe
Set up0.17 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
It deletes the other legs' work, so it has to be the flag the user typed.
Set up0.11 ms
No steps named here
Test body0.09 ms
No steps named here
Tear down0.08 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.12 ms
No steps named here
Test body0.10 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
This id becomes a directory component under the report base.
Set up0.12 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
A merge written back into the same folder must not delete its sources.
Set up0.08 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.32 ms
No steps named here
1 fixture
tmp_path
Set up0.54 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
1 fixture
tmp_path
Set up0.42 ms
No steps named here
Test body0.52 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Plain string order puts '10-16' before '2-4'.
Set up0.11 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Each part says which kind it is, so this cannot raise on any Python.
Set up0.10 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
It would land the copy outside the report directory entirely.
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
1 marker
parametrize(value)
1 parameter
value = ..
Set up0.10 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(value)
1 parameter
value = .
Set up0.11 ms
No steps named here
Test body0.12 ms
No steps named here
Tear down0.07 ms
No steps named here
1 marker
parametrize(value)
1 parameter
value =
Set up0.11 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(value)
1 parameter
value = None
Set up0.09 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(value)
1 parameter
value = /
Set up0.12 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(value)
1 parameter
value = ../
Set up0.10 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
It is a decoration on a row, and no reason to lose the row.
Set up0.08 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
1 marker
parametrize(value)
1 parameter
value = None
Set up0.09 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(value)
1 parameter
value =
Set up0.11 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.06 ms
No steps named here
1 marker
parametrize(value)
1 parameter
value = soon
Set up0.10 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(value)
1 parameter
value = []
Set up0.13 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.18 ms
No steps named here
Test body0.13 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
argparse fills every flag it did not see with None.
Set up0.07 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.12 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
0 and '' are answers; only None means nobody said.
Set up0.11 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
util._ini catches this and returns None, which is what makes each option helper use its own default. Returning '' would work for the string options and quietly break the ones that treat '' as deliberate.
Set up0.11 ms
No steps named here
Test body0.14 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
plugin.py expands --html-report once and writes it back; a namespace holding its own copy would let the two views drift, and the drift shows up as a report written to one path and screenshots to another.
Set up0.08 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Every reader in the tree guards with getattr, which only catches one.
Set up0.07 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
util._plugin_versions calls this unguarded, and a shim that crashes on the function it replaced is not a shim.
Set up0.07 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.04 ms
No steps named here
1 fixture
tmp_path
Set up0.53 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
[] is 'nothing was passed', which is not the same as 'nobody said'.
Set up0.07 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
The point of the whole class: a helper written for a pytest Config answers correctly when handed this instead.
1 fixture
tmp_path
Set up0.46 ms
No steps named here
Test body0.10 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Reached only because getini raised and getoption said None.
Set up0.08 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
00:xx is 12:xx am on a twelve-hour clock, not 0:xx.
Set up0.07 ms
No steps named here
Test body0.11 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.12 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
The argument names are a nicety; the step itself is not.
Set up0.08 ms
No steps named here
Test body0.15 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
'selenium...TimeoutException' and 'TimeoutException' are one group.
Set up0.10 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Far too common a failure to leave sitting in the unclassified pile.
Set up0.10 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
1 marker
parametrize(value,expected)
2 parameters
expected = 12value = 12
Set up0.12 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(value,expected)
2 parameters
expected = 12value = 12.7
Set up0.11 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(value,expected)
2 parameters
expected = 12value = 12.7
Set up0.13 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(value,expected)
2 parameters
expected = 0value = 0
Set up0.11 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(value)
1 parameter
value = None
Set up0.09 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(value)
1 parameter
value =
Set up0.09 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
1 marker
parametrize(value)
1 parameter
value = later
Set up0.21 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
1 marker
parametrize(value)
1 parameter
value = []
Set up0.10 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(value)
1 parameter
value = {}
Set up0.09 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(value,expected)
2 parameters
expected = 1.5value = 1.5
Set up0.11 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(value,expected)
2 parameters
expected = 2.0value = 2
Set up0.11 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(value)
1 parameter
value = None
Set up0.10 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
1 marker
parametrize(value)
1 parameter
value =
Set up0.10 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
1 marker
parametrize(value)
1 parameter
value = later
Set up0.11 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(value)
1 parameter
value = []
Set up0.10 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
A typo in an ini file must not stop the run.
Set up0.08 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
1 fixture
tmp_path
Set up0.48 ms
No steps named here
Test body0.34 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.39 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
-s means stdout never reaches the column, and a blank column with no explanation reads as a test that printed nothing.
Set up0.09 ms
No steps named here
Test body0.13 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.13 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
The mtime is the moment of the CI clone, so an age limit read from mtimes would keep every build for ever.
1 fixture
tmp_path
Set up0.49 ms
No steps named here
Test body0.18 ms
No steps named here
Tear down0.06 ms
No steps named here
1 fixture
tmp_path
Set up0.41 ms
No steps named here
Test body0.17 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.48 ms
No steps named here
Test body0.19 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.10 ms
No steps named here
Test body9.59 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Rerun is not a status a row carries, so there is no filter to send anyone to - and a control that goes nowhere is worse than a number.
Set up0.12 ms
No steps named here
Test body9.70 ms
No steps named here
Tear down0.08 ms
No steps named here
Description
The markup around the figure changed; the figure did not.
Set up0.10 ms
No steps named here
Test body9.37 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
It would open the table on "No matching records found", which is not what the figure it was read off said. Disabled rather than styled flat, so it is not a tab stop either.
Set up0.15 ms
No steps named here
Test body0.84 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.42 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Clicking FAIL, going back to the dashboard and clicking FAIL again sets the hash to what it already is, and no hashchange fires for that.
Set up0.08 ms
No steps named here
Test body0.31 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.49 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
A draw re-renders the whole table, and the hashchange the click causes asks for the same filter a second time.
Set up0.10 ms
No steps named here
Test body0.70 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.60 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
The nav link's href is a bare #test-metrics, so following it would throw away the ?status= the hash was opened with.
Set up0.07 ms
No steps named here
Test body0.33 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
?status=NOPE shows the whole table; filtering it down to no rows at all is the one reading nobody wanted.
Set up0.07 ms
No steps named here
Test body0.76 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Half the ways into the filter are the chips over the table, and a URL still saying status=FAIL over a table showing the passes is worse than one saying nothing at all.
Set up0.10 ms
No steps named here
Test body1.43 ms
No steps named here
Tear down2.31 ms
No steps named here
Description
replaceState, not a new entry per click: back should go back to the dashboard, not walk out through every chip that was tried.
Set up0.10 ms
No steps named here
Test body0.61 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.51 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Clicking the chip that is holding clears it too, but that is only discoverable once you know it - and a link opened on ?status=FAIL was never clicked into a chip at all.
Set up0.07 ms
No steps named here
Test body0.30 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.29 ms
No steps named here
Tear down0.04 ms
No steps named here
1 fixture
tmp_path
Set up0.47 ms
No steps named here
Test body438 ms
No steps named here
Tear down0.24 ms
No steps named here
1 fixture
tmp_path
Set up0.65 ms
No steps named here
Test body437 ms
No steps named here
Tear down0.20 ms
No steps named here
1 fixture
tmp_path
Set up0.63 ms
No steps named here
Test body423 ms
No steps named here
Tear down0.22 ms
No steps named here
1 fixture
tmp_path
Set up0.76 ms
No steps named here
Test body422 ms
No steps named here
Tear down0.26 ms
No steps named here
1 fixture
tmp_path
Set up0.64 ms
No steps named here
Test body411 ms
No steps named here
Tear down0.21 ms
No steps named here
1 fixture
tmp_path
Set up0.63 ms
No steps named here
Test body409 ms
No steps named here
Tear down0.20 ms
No steps named here
1 fixture
tmp_path
Set up0.59 ms
No steps named here
Test body433 ms
No steps named here
Tear down0.23 ms
No steps named here
1 fixture
tmp_path
Set up0.71 ms
No steps named here
Test body468 ms
No steps named here
Tear down0.20 ms
No steps named here
1 fixture
tmp_path
Set up0.76 ms
No steps named here
Test body491 ms
No steps named here
Tear down0.20 ms
No steps named here
1 fixture
tmp_path
Set up0.61 ms
No steps named here
Test body401 ms
No steps named here
Tear down0.22 ms
No steps named here
1 fixture
tmp_path
Set up0.69 ms
No steps named here
Test body396 ms
No steps named here
Tear down0.24 ms
No steps named here
1 fixture
tmp_path
Set up0.61 ms
No steps named here
Test body396 ms
No steps named here
Tear down0.21 ms
No steps named here
1 fixture
tmp_path
Set up0.73 ms
No steps named here
Test body424 ms
No steps named here
Tear down0.20 ms
No steps named here
1 fixture
tmp_path
Set up0.67 ms
No steps named here
Test body409 ms
No steps named here
Tear down0.22 ms
No steps named here
1 fixture
tmp_path
Set up0.69 ms
No steps named here
Test body383 ms
No steps named here
Tear down0.21 ms
No steps named here
1 fixture
tmp_path
Set up0.66 ms
No steps named here
Test body374 ms
No steps named here
Tear down0.24 ms
No steps named here
1 fixture
tmp_path
Set up0.74 ms
No steps named here
Test body410 ms
No steps named here
Tear down0.20 ms
No steps named here
1 fixture
tmp_path
Set up0.78 ms
No steps named here
Test body715 ms
No steps named here
Tear down0.25 ms
No steps named here
1 fixture
tmp_path
Set up0.69 ms
No steps named here
Test body389 ms
No steps named here
Tear down0.22 ms
No steps named here
1 fixture
tmp_path
Set up0.66 ms
No steps named here
Test body399 ms
No steps named here
Tear down0.22 ms
No steps named here
1 fixture
tmp_path
Set up0.62 ms
No steps named here
Test body401 ms
No steps named here
Tear down0.22 ms
No steps named here
1 fixture
tmp_path
Set up0.74 ms
No steps named here
Test body399 ms
No steps named here
Tear down0.22 ms
No steps named here
1 fixture
tmp_path
Set up0.64 ms
No steps named here
Test body403 ms
No steps named here
Tear down0.24 ms
No steps named here
1 fixture
tmp_path
Set up0.66 ms
No steps named here
Test body405 ms
No steps named here
Tear down0.21 ms
No steps named here
1 fixture
tmp_path
Set up0.66 ms
No steps named here
Test body398 ms
No steps named here
Tear down0.24 ms
No steps named here
1 fixture
tmp_path
Set up0.71 ms
No steps named here
Test body400 ms
No steps named here
Tear down0.24 ms
No steps named here
1 fixture
tmp_path
Set up0.66 ms
No steps named here
Test body410 ms
No steps named here
Tear down0.25 ms
No steps named here
1 fixture
tmp_path
Set up0.66 ms
No steps named here
Test body405 ms
No steps named here
Tear down0.21 ms
No steps named here
1 fixture
tmp_path
Set up0.72 ms
No steps named here
Test body407 ms
No steps named here
Tear down0.27 ms
No steps named here
Description
A step too quick for a whole millisecond keeps its decimals. Rounded to an integer it printed "0 ms", which reads as a step that never ran rather than one that was fast - and a unit test's three phases are routinely a fraction of a millisecond each.
Set up0.21 ms
No steps named here
Test body0.09 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
An archive written before steps were timed, not a measurement.
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Past ten there is nothing left for the decimals to say.
Set up0.17 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Three or four digits, so the column stays readable.
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.12 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
An archive whose per-test duration was floored still has its phases. Builds written before durations kept microseconds carry 0.0 for every quick test; the phase timings beside them are the same test measured, so the rail reads off those instead of drawing the test as instant.
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
The phases are no longer whole milliseconds, and neither is their sum.
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
The directories repeat down the whole rail and crowd out what differs.
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Scaled to the run's slowest, every test but one draws a flat line.
Set up0.08 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.15 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.06 ms
No steps named here
1 marker
parametrize(kind)
1 parameter
kind = given
Set up0.16 ms
No steps named here
Test body0.11 ms
No steps named here
Tear down0.06 ms
No steps named here
1 marker
parametrize(kind)
1 parameter
kind = when
Set up0.11 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.04 ms
No steps named here
1 marker
parametrize(kind)
1 parameter
kind = then
Set up0.09 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
1 marker
parametrize(kind)
1 parameter
kind = and
Set up0.10 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.05 ms
No steps named here
1 marker
parametrize(kind)
1 parameter
kind = but
Set up0.11 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Every ordinary pytest step would otherwise wear one that says nothing.
Set up0.08 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
The exception walks out through every step it was inside, and each is marked failed - but only the innermost one is given the text.
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
It runs from teardown with nothing open, and a photograph of the browser at the end of a failing test is a photograph of what that step left.
Set up0.07 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
None is the phase - there is no failing step to hang it on.
Set up0.07 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.10 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
This is what the tab keys off to show a scenario as a scenario.
Set up0.07 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body2.80 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
The automatic capture runs from the teardown hook, with no step open at all - so the tree files it against the step carrying the error.
1 fixture
tmp_path
Set up0.69 ms
No steps named here
Test body391 ms
No steps named here
Tear down0.20 ms
No steps named here
Description
They are two halves of one answer: what went wrong, and what it looked like when it did.
1 fixture
tmp_path
Set up0.80 ms
No steps named here
Test body387 ms
No steps named here
Tear down0.23 ms
No steps named here
Description
Which is most tests. The body is where the test ran, and a picture with nowhere better to go is still worth having on the page.
1 fixture
tmp_path
Set up0.79 ms
No steps named here
Test body399 ms
No steps named here
Tear down0.23 ms
No steps named here
Description
`attach` mid-test knows which step is open, so this one needs no guess - and it works for a test that passed, where nothing threw.
1 fixture
tmp_path
Set up0.81 ms
No steps named here
Test body392 ms
No steps named here
Tear down0.19 ms
No steps named here
Description
The tab shows one test at a time, so arrowing across its pictures should stay inside the test being read rather than travel the whole run.
1 fixture
tmp_path
Set up0.79 ms
No steps named here
Test body400 ms
No steps named here
Tear down0.19 ms
No steps named here
Description
A pytest-bdd scenario takes no fixtures of its own: every step asks for what it needs while it runs, so the page never reaches `funcargs` and the capture had nothing to look at. Nothing said so - the run simply produced no picture. This is that shape without the dependency.
1 fixture
tmp_path
Set up0.81 ms
No steps named here
Test body406 ms
No steps named here
Tear down0.30 ms
No steps named here
Description
It arrives twice now - named by the test and cached by the request - and two of one picture on a row is not two pictures.
1 fixture
tmp_path
Set up0.81 ms
No steps named here
Test body412 ms
No steps named here
Tear down0.19 ms
No steps named here
Set up0.25 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.14 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body33 ms
No steps named here
Tear down0.14 ms
No steps named here
Set up0.13 ms
No steps named here
Test body0.11 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.12 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.09 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.13 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.14 ms
No steps named here
Test body0.11 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.21 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.09 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.14 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body13 ms
No steps named here
Tear down0.09 ms
No steps named here
Set up0.12 ms
No steps named here
Test body0.09 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.18 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.09 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.13 ms
No steps named here
Test body0.11 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.09 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.09 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
tmp_path
Set up0.56 ms
No steps named here
Test body404 ms
No steps named here
Tear down0.21 ms
No steps named here
1 fixture
tmp_path
Set up0.65 ms
No steps named here
Test body388 ms
No steps named here
Tear down0.23 ms
No steps named here
1 fixture
tmp_path
Set up0.74 ms
No steps named here
Test body402 ms
No steps named here
Tear down0.24 ms
No steps named here
1 fixture
tmp_path
Set up0.70 ms
No steps named here
Test body407 ms
No steps named here
Tear down0.27 ms
No steps named here
1 fixture
tmp_path
Set up0.77 ms
No steps named here
Test body423 ms
No steps named here
Tear down0.25 ms
No steps named here
Set up0.17 ms
No steps named here
Test body28 ms
No steps named here
Tear down0.09 ms
No steps named here
Set up0.12 ms
No steps named here
Test body30 ms
No steps named here
Tear down0.09 ms
No steps named here
Set up0.14 ms
No steps named here
Test body28 ms
No steps named here
Tear down0.08 ms
No steps named here
Set up0.14 ms
No steps named here
Test body29 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.14 ms
No steps named here
Test body29 ms
No steps named here
Tear down0.09 ms
No steps named here
Set up0.12 ms
No steps named here
Test body28 ms
No steps named here
Tear down0.09 ms
No steps named here
Set up0.12 ms
No steps named here
Test body31 ms
No steps named here
Tear down0.09 ms
No steps named here
Set up0.15 ms
No steps named here
Test body28 ms
No steps named here
Tear down0.09 ms
No steps named here
Set up0.11 ms
No steps named here
Test body29 ms
No steps named here
Tear down0.07 ms
No steps named here
Set up0.10 ms
No steps named here
Test body28 ms
No steps named here
Tear down0.08 ms
No steps named here
Set up0.11 ms
No steps named here
Test body28 ms
No steps named here
Tear down0.08 ms
No steps named here
Set up0.11 ms
No steps named here
Test body28 ms
No steps named here
Tear down0.08 ms
No steps named here
Set up0.13 ms
No steps named here
Test body0.11 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
An ini linelist is indented under its key, and a person types spaces.
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
The one rule here that is not a convenience. A badge is built per test out of whatever a marker said and lands in a file that gets published and mailed on. javascript: in an href is a way to run something in whoever opens the report.
Set up0.11 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Ids are pasted out of trackers and arrive with whatever those allow.
Set up0.08 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Substituted, never formatted. str.format reads every brace in the string, and a url is a place people put them - so formatting somebody's templated query throws KeyError instead of rendering their link.
Set up0.17 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Half a url opens the tracker's front page and looks like it worked.
Set up0.10 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
`docs = https://wiki/testing` is a page, not a forgotten placeholder.
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
`text` is one string and cannot be taken back apart to find the id.
Set up0.08 ms
No steps named here
Test body0.10 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
A bare PROJ-123 says nothing about which system it is an id in.
1 fixture
patterns
Set up0.15 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.06 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
1 fixture
patterns
Set up0.12 ms
No steps named here
Test body0.23 ms
No steps named here
Tear down0.06 ms
No steps named here
1 fixture
patterns
Set up0.10 ms
No steps named here
Test body0.14 ms
No steps named here
Tear down0.05 ms
No steps named here
1 fixture
patterns
Set up0.11 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.04 ms
No steps named here
1 fixture
isolate_patterns
Set up0.09 ms
No steps named here
Test body0.12 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
The tools that ingest issue keys never open an html report.
Set up0.08 ms
No steps named here
Test body0.13 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Where pytest's own writer puts them, and so where consumers look.
Set up0.08 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
An empty property is a row every consumer shows and none can use.
Set up0.08 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Owner and severity need no configuration; everything else is invented.
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
The filter reads them off the button; one attribute has to hold both.
Set up0.07 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.03 ms
No steps named here
Description
`pytestmark` on the module and the same marker on the test is two records of one fact, and two identical pills would be a filter that splits a team in half.
Set up0.07 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.10 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
It is not a sixth level and it is not `normal` - it is unfinished.
Set up0.07 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
A typo is not a sixth level, and must not outrank blocker.
Set up0.07 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.03 ms
No steps named here
Description
Two spellings of one word must not split a suite's counts in half.
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.03 ms
No steps named here
Description
A class rated above its module is somebody correcting the outer word.
Set up0.07 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Nothing is nearer, and reading a blocker down to minor hides work.
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.03 ms
No steps named here
Description
Allure defaults; this does not - six hundred unrated tests are not normal.
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
The filter reads it off the button, already resolved to one word.
Set up0.08 ms
No steps named here
Test body0.88 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
The tab promises every marker, and says which one decided the colour.
Set up0.09 ms
No steps named here
Test body0.12 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
It answers `how bad`, which is not what a row of `slow` and `smoke` says.
1 fixture
patterns
Set up0.10 ms
No steps named here
Test body0.16 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
A testcase carrying both `normal` and `critical` is one nobody can rank.
Set up0.09 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.06 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.10 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.07 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.03 ms
No steps named here
Description
report.sections is cumulative, so a retried test hands back the earlier attempt's output alongside the current one.
Set up0.08 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.15 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
What is closest to the end of the test is what gets kept, so a chatty setup never crowds out the call it was preparing for.
Set up0.12 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
A log line that looks like a placeholder must be shown, not filled in when the page is assembled.
Set up0.11 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.03 ms
No steps named here
Description
Nothing is missing that the run asked for, so there is nothing to say.
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
Unset, pytest keeps whatever the root logger is already emitting.
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.09 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.07 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
A report is a build artifact that gets published and passed round; a nav entry must not become a way to run code in whoever opens the page.
Set up0.08 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
A one-character scheme is a drive letter everywhere it actually occurs.
Set up0.09 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.04 ms
No steps named here
Description
One render leaves its marks all over ConfigVars, and a process that renders twice - the merge, which builds one report out of several runs' records - starts from whatever the first one left. generate_json_data accumulates its dashboard counters with +=, so a second render doubles every number on the page without raising anything.
Set up0.13 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
The suite isolates ConfigVars by saving the objects it finds and putting those same objects back, so reset_config_vars has to rebind each attribute rather than empty it where it stands: a fixture holding the live list would otherwise be handed an emptied one to restore, and every _isolate_config_vars in the suite would quietly stop isolating anything.
Set up0.09 ms
No steps named here
Test body0.08 ms
No steps named here
Tear down0.05 ms
No steps named here
Description
Two rows with one label is a panel that cannot say which was used. The entries arrive highest layer first - the command line, then a profile, then the ini key - so the first answer is the one that outranks the rest.
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.03 ms
No steps named here
Tear down0.03 ms
No steps named here
Set up0.08 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.12 ms
No steps named here
Test body0.05 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.09 ms
No steps named here
Test body0.04 ms
No steps named here
Tear down0.04 ms
No steps named here
Set up0.09 ms
No steps named here
Test body1.48 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.10 ms
No steps named here
Test body0.90 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.10 ms
No steps named here
Test body1.47 ms
No steps named here
Tear down0.05 ms
No steps named here
Set up0.11 ms
No steps named here
Test body0.60 ms
No steps named here
Tear down0.06 ms
No steps named here
Description
The suite total says the run retried three times; only the row says which test they were spent on.
Set up0.11 ms
No steps named here
Test body2.16 ms
No steps named here
Tear down0.07 ms
No steps named here
Description
Every row holds its own error on itself, so two failures merged from different workers cannot end up showing each other's.
AssertionError: assert 'Example Domain' == 'Not the heading'
- Not the heading
+ Example Domain
No screenshot matches what you asked for.
No screenshots in this run
A test that fails holding a Selenium driver or a Playwright page is
photographed for you - no hook, no fixture, nothing to import. This run
was holding neither, and attached nothing of its own.
1Nothing to writeautomatic
The browser is already in the test’s own fixtures and the reporter is
already standing in its teardown, so the picture is taken there - the last
moment before the driver is quit. What makes something a browser is that it
can hand over a PNG, so Selenium, Playwright, appium, splinter and a wrapper
of your own all work, whatever the fixture is called.
def test_checkout(page): # or driver, or whatever you called it
page.goto("/cart")
assert page.locator("h1").inner_text() == "Cart"# fails, and is photographed
Failures only, by default. --report-screenshots=all photographs every
test, none turns the automatic capture off - and attach below
keeps working either way.
2Or take the picture yourself
attach takes the image rather than the browser, so anything that can
produce a PNG reaches the report - a page mid-test, a chart, a rendered PDF,
an image diff. A test that attaches its own picture is not photographed
again on the way out.
from pytest_html_reporter importattachattach(data=driver.get_screenshot_as_png()) # Seleniumattach(data=page.screenshot()) # Playwrightattach(data=await page.screenshot()) # Playwright, async API
3Async tests, and unittest
The automatic capture is synchronous, so an async Playwright page has
nowhere to await - attach from the body instead. A unittest suite that
quits its driver in tearDown has already closed the browser by the time
the capture would run, so it attaches from there, before the quit.
Every image is kept whatever the test did - a screenshot of a pass is a
baseline worth having. Each one also lands on the Screens column of the
Test Metrics row it belongs to, next to the error it explains.
The request and response behind each test, with the curl line that repeats the call
Showing what attached
Nothing matches that search.
No API logs in this run
Hand this tab the request and the response, and they are kept against the test
that produced them - both bodies, both sets of headers, and the curl line that
repeats the call.
1Attach a call
attach_api reads the response object, so requests and httpx
both work as they are - nothing else to install.
Attaching every call buries the one that matters and grows the report for
no reason. The payload worth keeping is the one behind a failure, so
attach from a fixture's teardown and let the outcome decide. The reporter
builds a test's record after the finalizers have run, which is what makes
this work.
# conftest.pyimport pytest
from pytest_html_reporter importattach_api@pytest.fixturedef api(request):
client = ApiClient()
yield client
if request.node.rep_call.failed:
attach_api(client.last_response)
# lets the fixture above see how the test ended@pytest.hookimpl(tryfirst=True, hookwrapper=True)
def pytest_runtest_makereport(item, call):
outcome = yield
setattr(item, "rep_" + outcome.get_result().when, outcome.get_result())
Put the hook in conftest.py. pytest picks one up from a test module
too, but only for that module's own tests - a conftest covers every test
under it.
3Not only API calls
Anything you would otherwise dig out of a terminal can go here beside the
call it belongs to.
Credentials are blanked out before anything is written - in headers, in a
?api_key= query string, in the curl line and in the fields of a JSON
body. A report is a build artifact, and it gets published.
Measure it and this tab fills itself in - the percentage, the split by file,
and the lines nothing touched - beside the tests that did the measuring.
1Run with coverage
Nothing else to configure. Whatever pytest-cov measured is read
straight out of the finished run, so the number here is the number your
terminal just printed.
my_package is yours to fill in. --cov takes the import name
or the path of the code under test - not the tests, and not a folder that is
not there: point it at one and the run measures nothing at all.
2Or read a report you already havefor CI
When coverage was produced by an earlier step rather than by this run.
A coverage.json, a Cobertura coverage.xml or a .coverage
data file all work - and the first two are found without being named if
they sit beside the report.
pytest --report-coverage-file=coverage.xml
3Keep the annotated source a click away
Line-by-line source is the one thing a summary cannot replace. Generate it
and it is linked from the card above; --report-link does the same for
any page of your own.
pytest --cov=my_package--cov-report=html
# anything else worth reaching from the side nav
pytest --report-link"Coverage=htmlcov/index.html"
Read, never re-run, and never framed in. Embedding htmlcov would empty
this tab the moment the report was mailed on its own - so the figures are
rendered here and the annotated source is linked.
FAIL
![Screenshot attached by test_heading_mismatch[chromium]](pytest_screenshots/1788697330423-gw0-8.png)
FAIL
FAIL
FAIL