Find minimal set of flaky tests with rspec automatically
I had same weird failing tests in a project that would only fail when running the full test suit, but not when running on their own.
I knew some other test is probably doing something funky with some side effects which makes them fail. But which other test is the culprit? How do I find this other test?
The solution is simple: bundle exec rspec --bisect
I just left it running for a bit until it provided me with a minimal reproducible set of tests that failed in combination.
With only 3 tests remaining, it was very easy to nail down exactly where the issue was.
(I knew about the concept of bisect in general, e.g. from git-bisect, but I had no idea how easy and useful it was to use that idea with a test framework)