-
Notifications
You must be signed in to change notification settings - Fork 54
Description
Per Catch2 documentation:
GENERATE can be seen as an implicit SECTION, that goes from the place GENERATE is used, to the end of the scope.
[...]
The fact that GENERATE introduces a virtual SECTION can also be used to make a generator replay only some SECTIONs, without having to explicitly add a SECTION
I looked into the Catch2 integration and it seems like the Catch2 API does not include generator information as part of the sector:
// Approvals/integrations/catch/Cathc2Approvals.h
struct Catch2TestCommitRevert : Catch::TestEventListenerBase
{
...
virtual void sectionStarting(Catch::SectionInfo const& sectionInfo) override
{
currentTest.sections.push_back(sectionInfo.name); // Name here does not include the generator value
}
...
};In fact, if you look at the Catch::SectionInfo type it only includes name and source location. I'm no expert on the Catch2 internals but at first glance, I would say Catch2 generators are handled more like part of the body of a section rather than variants of the parent section (Like generated tests cases in other frameworks).
The result of this behavior is that if you run an approval inside a section with a generator, the approval is uniquely registered for the section rather than each generated variant of the section, resulting in approval failures following the execution of the different variants of the section:
enum class Keyword
{
Unknown, Foo, Bar, Quux
};
Keyword parseKeyword(const std::string_view word);
SECTION("Verify our parsing algorithm works with the most relevant words in the vocabulary of a programmer")
{
const std::string_view word = GENERATE("foo", "bar", "quux");
Approvals::verify(parseKeyword(word)); // If we approve "foo", it fails in the next iteration of the generator
}