Skip to content

← Archive

Reqnroll at Insignis · Part 2 of 2

How We Write Reqnroll Tests

The conventions behind my team's Reqnroll suite, written precisely enough for Claude to follow, from thin steps and shared drivers to proving that nothing else happened, shown in a sample you can run.


19 November 202611 min read

The second of two essays. Who Decided This Rule? is about why my team writes Reqnroll tests and what changes when Claude writes them. This one is my account of how we write them, with the code.

When I wrote down how my team at Insignis builds Reqnroll tests, I wrote it for Claude. It's a skill in an internal Claude plugin, kept in a team-wide repository in git, and the team uses it for their own Reqnroll work. A skill is written for a reader that follows it literally, so a convention can't stay vague in one. "Steps are thin" had to become "a step makes one or two driver calls and nothing else".

This is my account of the conventions we settled on. I've published a generic version of the skill in library-holds, the sample from the first essay, which Claude built by following it. Copy the skill into your own repo to get started. The examples below come from the same sample. Its scenarios README writes the conventions down the same literal way, and I've quoted it beside the main ones, so you can see what a convention looks like once it's precise enough for a model to follow. The scenarios run against a real Postgres in a container, so you'll need Docker to run them.

What the Unit Tests Can't See§

The sample's unit tests are the kind most .NET teams write, xUnit with the repository mocked by Moq. The repo has a branch, planted-bug, where the query that finds waiting holds for a returned copy forgets about format. It's one line, and it's an easy line to get wrong.

.Where(h => h.Status == HoldStatus.Waiting && h.Title == title)

On that branch all 22 unit tests pass, because the one that covers matching only checks that a mocked repository was asked for the right title and format, and never runs the query. Three of the 24 scenarios fail, the ones where a paperback, a large print or an audiobook copy is returned while Ana is waiting for a hardback. The assertion library's phrasing is clumsy, but the because in each failure is the scenario's own sentence, give or take a "should".

Expected property actual.CopyId to be <null> because Ana's hold should still be waiting, but found {2724177f-de93-4ce3-b3c6-270e26b9b4be}.
Expected property actual.CollectBy to be <null> because Ana's hold should still be waiting, but found "2027-03-17T20:00:00+00:00".

What caught the bug is the real database, and a plain xUnit integration test against the same Postgres would have caught it too. We keep the unit tests for logic that lives in one class, because they run in milliseconds and a failure points at the line. The scenarios are for journeys, and for rules someone outside the code needs to read.

Agreeing the Rules§

When I started, the rules were agreed in Gherkin. I wrote a scenario while I was building the feature, and when a case was unclear I sent product a screenshot of it, which is how they caught the state transition in the first essay.

The skill came later, and it now asks for the Rule: lines from a person, or from a ticket a person points to, before any Gherkin is written, then stops for a person to review the feature file before any bindings. The README puts the part that matters most like this.

The rules come from people. The Rule: lines say what was decided. Edge cases are proposed, then accepted or rejected one by one. Current behaviour never becomes a rule without someone confirming it's intended, because the code may be the bug.

Who Decided This Rule? covers what happened when nobody was there to answer.

Writing the Feature§

Reqnroll is the open-source fork of SpecFlow for .NET,11 Tricentis ended SpecFlow on 31 December 2024. Reqnroll was forked from it in early 2024, is BSD-licensed, and is still actively developed. and it fits in a paragraph. You write a line of plain English in a .feature file, and an attribute on a C# method binds it to code. At build time Reqnroll turns each feature into an ordinary test class, so dotnet test, Test Explorer and CI don't change at all.

The sample's rule about formats looks like this.

Rule: A copy only fills a hold for the same title in the same format

    Scenario Outline: A returned copy whose <detail> is <value> doesn't fill a hold for The Hobbit in hardback
        Given Ana placed a hold on "The Hobbit" in hardback at 2027-03-01T09:00:00Z
        When a copy is returned at 2027-03-10T14:00:00Z that matches Ana's hold except for its <detail>, which is <value>
        Then Ana's hold is still waiting
        And the copy is back on the shelf
        And nobody is told anything

        Examples:
            | detail | value            |
            | title  | The Silmarillion |
            | format | paperback        |
            | format | large print      |
            | format | audiobook        |

Each row runs as its own test, and each row says why it exists. The last two Then lines are there for what shouldn't change. A copy that fills no hold should go back on the shelf without anyone being told, and a scenario that only checked Ana's hold would miss a bug that sent her a notification anyway.

Without Reqnroll I'd have written it as an xUnit theory calling the same drivers. This one isn't in the repo.

[Theory]
[InlineData("The Silmarillion", "Hardback")]
[InlineData("The Hobbit", "Paperback")]
[InlineData("The Hobbit", "LargePrint")]
[InlineData("The Hobbit", "Audiobook")]
public async Task A_returned_copy_that_differs_from_the_hold_does_not_fill_it(string title, string format)
{
    var ana = await _holds.PlaceAsync("Ana", "The Hobbit", "Hardback", Wire.ParseInstant("2027-03-01T09:00:00Z"));
    var copy = ReturnsDriver.NewCopy(title, format, Wire.ParseInstant("2027-03-10T14:00:00Z"));

    await _returns.ReturnAsync(copy);

    await _holds.AssertWaitingAsync(ana);
    await _returns.AssertBackOnTheShelfAsync(copy);
    await _notifications.AssertNobodyToldAnythingAsync();
}

It runs the same journey against the same database. What it doesn't say is why each row is there. You only see that the first row changes the title and the others change the format by comparing them with the line that places Ana's hold. A comment beside each row would help a developer, but product would never read it. In the feature file the reason is in the table, next to the sentence it's an example of.

Strip the feature down to its Rule: lines and what's left is the spec for the journey, the four rules listed in the first essay.

Randomised Twins§

Two of those rules also have a randomised twin, sitting under the same Rule: line as the hardcoded examples. This is the one for collect-by times.

@Randomised
Scenario Outline: A copy that reaches the hold shelf <when> can be collected until closing time on the 7th day
    Given Ana placed a hold on a random title in a random format
    When a copy matching Ana's hold is returned <when>
    Then Ana's hold is ready for collection
    And Ana is told the copy is ready

    Examples:
        | when                              | run |
        | at a random time                  | 1   |
        | at a random time                  | 2   |
        | in the week before a clock change | 3   |
        | in the week before a clock change | 4   |
        | around midnight UK time           | 5   |
        | around midnight UK time           | 6   |

The run column only makes each row run again with a new seed. Each run logs its seed, and setting SCENARIO_SEED replays it exactly. The Examples table says, in words, where the randomness is aimed, at the times the rule finds hardest. A property test in xUnit would keep that in a generator, in C#, where only the people who wrote it would see what it varies. Here a reader sees the fixed examples and the generalisation of them under the same sentence, and can ask whether the generalisation is the right one.

The twin has to work out the expected collect-by time for itself, so the test code carries a second implementation of the rule, checked only by the hardcoded examples beside it. The README is specific about where it comes from.

Expected values are worked out independently. The expected collect-by time comes from NodaTime, not from the API's own calculation.

So far the twins haven't caught anything the rest of the suite missed. When I broke the collect-by calculation, the hardcoded row and a 52 millisecond unit test caught it every time.

Steps, Drivers and Context§

Underneath the feature file there are three layers. A step calls a driver, the class that knows how to drive one part of the system, and does nothing else. "Steps are thin" is the vague version. The README's is the one a model can follow.

Steps make one or two driver calls and nothing else. Drivers own every detail of the API.

So every step reads like the sentence above it.

[Then("{word}'s hold is still waiting")]
public Task ThenMembersHoldIsStillWaitingAsync(string member) => holds.AssertWaitingAsync(placed.For(member));

The README spells out how a set-up failure stays separate from a rule failure.

Actions only act. A Given that has to have taken effect uses a Require* method, which fails with "Precondition failed", so a broken set-up is never mistaken for a broken rule. A Then uses an Assert* method named after the sentence it backs.

The state one step leaves for the next lives in a context class whose properties fail by naming the step that's missing. Use "Ana's hold is still waiting" in a scenario where Ana never placed a hold, and the failure says This step needs Ana to have placed a hold, but no earlier step in the scenario did that, instead of a NullReferenceException three frames down.

Steps and drivers are organised by what the Gherkin talks about, and every feature shares them. In the sample there's a driver each for holds, returns and notifications. My first pass gave each feature file its own step class and driver, and I soon found a step, and the driver method behind it, defined twice across two related features with slightly different wording and identical code. A staff engineer I work with wasn't surprised. Steps drifting apart was the usual problem he'd seen with BDD, and he put it down to the steps being stringly typed. He suggested organising them by subject, which is what the sample does, and it makes reuse the easy path.

Reqnroll's context injection keeps that cheap. Every step class in a scenario that asks for a context or a driver in its constructor gets the same instance, so in the sample HoldSteps and ReturnSteps share one HoldsContext without either knowing the other exists. Without it I'd have been pushed back towards one step class per feature, which is where the duplicate came from.

Keeping the Bindings Honest§

Reqnroll checks that a line of English matches a method, and nothing checks that the method does what the English says. Then Ben's hold is still waiting can be bound to code that checks something else, or nothing, and it will pass forever. Thin steps and assertions named after their sentence are what keep that visible, because a reviewer can check a one-line binding against the sentence above it in seconds.

Reqnroll will also fail if one line of Gherkin matches two bindings, but it says nothing when two different lines mean the same thing, which is how my duplicate step happened. Nothing in the code reaches that, so it's part of PR review.

One phrase per meaning. Steps are matched by text, so the compiler can't tell when two phrasings mean the same thing. Before adding a step, search Steps/ for one that already says it. In review, a new step string is only accepted when its meaning differs from every existing one; otherwise the scenario reuses the existing wording.

Whoever reviews has to know the vocabulary well enough to spot a near-duplicate. Every PR is reviewed by a person, and Claude reviews alongside them, which helps with exactly that.

Before trusting a Then, break it on purpose. Change an example so that it should flip the outcome, run it, and read the failure. Until I've seen a test fail, I don't know it checks anything.22 Every deliberate break mentioned in this essay was made in the sample, run, and reverted. The one kept on purpose is the planted-bug branch.

Checking What Nobody Mentioned§

Three more conventions matter as much as anything above, and none of them is Reqnroll's. An xUnit suite driving the same app would need them all.

Check the fields no step mentions. An assertion on a hold's status alone would miss a bug that gave the hold a copy it shouldn't have. An Assert method builds the whole object it expects and compares it with what the API returned, which is why the planted bug's failure above names the copy and the collect-by time.

Prove that nothing happened. The notification goes out in the background, so checking straight away proves nothing and waiting a fixed time is a guess.

No sleeps. Every Require* and Assert* waits for the app to settle first (the outbox has nothing left to send), so "nobody is told anything" doesn't depend on a guessed delay.

In our production system, settled means message consumers that have caught up too. That part is there because of the four-eyes approval bug, which only showed once the edits going through Kafka had landed.

Keep unfixed bugs in view. When a bug can't be fixed straight away, the scenario is written for the correct behaviour and tagged @ignore, with the ticket in a comment beside it. It shows as skipped in every run instead of living in someone's memory, and when the fix lands, the tag comes off and the same scenario has to pass.

What Else It Costs§

The first essay covers what Reqnroll costs and when it isn't worth it. Two costs only show up once you're building the suite.

The first run is slower than the nine seconds I quote there, about 18 in the sample, because it also pulls the Postgres image.

Stand-ins encode your assumptions. WireMock behaves the way I believe the notification provider behaves, so if the real provider changes its API, every scenario stays green. A contract check against the provider, or an occasional run against its sandbox, is what covers it.

What They Have in Common§

Most of these conventions close off a way for a scenario to pass or fail for the wrong reason: a set-up that broke quietly, a binding that checks less than its sentence says, a wait that guessed, two phrasings of one rule drifting apart. The sample's scenarios README has the full list. None of that is specific to Claude, but writing the conventions down for a reader that follows instructions literally is what made me state each one precisely enough to quote.

Notes

  1. Tricentis ended SpecFlow on 31 December 2024. Reqnroll was forked from it in early 2024, is BSD-licensed, and is still actively developed. ↩︎

  2. Every deliberate break mentioned in this essay was made in the sample, run, and reverted. The one kept on purpose is the planted-bug branch. ↩︎