Jesper Dramsch , , , read in 3 minutes

It's Advent of Code! 🎄

Every year I add to my utils.py and try to make the coding experience a bit nicer.

This year, I discovered that once again I didn't update the Readme after the last year. I also didn't finish last year, must've been a stressful time...

On this website and places like the profile Readme on Github, I figured out that you can use HTML comments to insert text at certain locations, basically like a placeholder.

So I could do this for the Advent of Code readme, to have a place for the stars to go once my pytest test cases pass, right?

Pytest Test Outcomes

Turns out getting the status of a test passing isn't super straight forward.

The test itself doesn't know if it passed or not, so it needs some external hooks.

My first try was the teardown fixtures, but fixtures are technically still part of the test, so they don't report the test outcome either.

@pytest.fixture(autouse=True)
def teardown(request):
    yield
    # Do thing after test?
    # i.e. teardown etc

After some sleuthing on Stackoverflow, I found Issue 230 on the Pytest Github with the solution!

(Don't read too much in that thread, some people are a bit entitled and disproportionally unfriendly putting demands on a volunteer project...)

Learning about Conftest

Apparently we can define common parts in a conftest.py in the tests/ folder.

Perfect!

No way, I'd define this hook on every single test case, I have to revamp my daily tests for the advent of code already anyways.

So, let's get hooked.

@pytest.hookimpl(tryfirst=True, hookwrapper=True)
def pytest_runtest_makereport(item, call):
    # execute all other hooks to obtain the report object
    outcome = yield
    rep = outcome.get_result()
    _test_reports = getattr(item.module, "_test_reports", {})
    _test_reports[(item.nodeid, rep.when)] = rep
    item.module._test_reports = _test_reports

Just like in the fixture, yield gives us the test run and with the hook, we can get the test results.

I can't say this code is particularly pretty, but it is something to work with.

Passing those tests!

I set up my Advent of Code to have four test cases per default. That's an example and the full test case for part 1 and part 2 for each day.

But I want to add a star ⭐ to my Readme, when I finish a part. So basically, I need to check that it's on the full test case, that it passed and that it's the test call specifically.

Time for some regex and post-processing.

@pytest.hookimpl(tryfirst=True, hookwrapper=True)
def pytest_runtest_makereport(item, call):
    # execute all other hooks to obtain the report object
    outcome = yield
    rep = outcome.get_result()
    _test_reports = getattr(item.module, "_test_reports", {})
    _test_reports[(item.nodeid, rep.when)] = rep
    item.module._test_reports = _test_reports

    year = Path(__file__).resolve().parent.parent.name

    m = re.match(r"tests/test_day([0-9]{2})\.py::test_part([0-9])", item.nodeid)
    if m:
        day, part = m.groups()
        if rep.when == "call" and rep.outcome == "passed":
            with open(
                Path(Path(__file__).resolve().parent.parent.parent, "README.md"), "r+"
            ) as f:
                readme = f.read()

                f.seek(0)
                f.write(readme.replace(f"<!--{year}.{day}.{part}-->", "⭐"))

It's part of my advent-of-code repo here.

That'll for sure be some code I look back on, wondering whenever I was capable of even writing this.

But as long as it works, I'm happy!

Conclusion

Overall, I learned a lot about pytest internals!

So today I learned about hooks in pytest and conftest.py.

I also learned something about pytests handling of test outcomes and when the test actually counts as passed.

I also learned that pytest actually reports if the setup and teardown were succesfull in addition to the call.

What a useful library!