Pip's field guide · vol. 1

How to pick awatermelon

Sweet, juicy and refreshing. Here's how to pick a ripe one every time, whether it's a fruit or a test suite. Mostly a test suite.

The key

  • Look for assertions, not size.
  • Use your senses. Then use a mutation tester.
  • Find the field spot: where the test actually touches the code.
  • Never trust one that's green all the way through.

6 signs of a ripe test suite

1

Look for the field spot

Melon: The creamy yellow patch where it sat on the ground.

Test suite: The place a test actually touches the code: an assertion. No field spot means it never sat anywhere.

Assertions = ripe
2

Check the weight

Melon: A ripe one feels heavy for its size.

Test suite: Forty lines of setup and one assertNotNull is a hollow melon in a very nice box.

Heavy checks = juicy
3

Look at the shape

Melon: Round and even beats long and lopsided.

Test suite: Even on both sides of every boundary: 99, 100 and 101, not just 50 and 250.

Both sides = round
4

Examine the skin

Melon: Dull means ripe. Shiny means not yet.

Test suite: Nothing is shinier than a 100% coverage badge. Be suspicious of anything that pleased with itself.

Shiny = suspicious
5

Look for webbing

Melon: Brown, web-like scars mean bees visited. They are the sweetest ones.

Test suite: Regression tests are scars from real bugs. Every one of them was earned the hard way.

Scars = sweeter
6

Thump it

Melon: A deep, hollow sound means it is ripe.

Test suite: A hollow suite means missed mutants. This is exactly where the analogy breaks, and why we built a tool.

Hollow = not ripe (sorry)
Pip, waiting to be thumped

Try the thump test

Give it a gentle knock. In a watermelon, a deep hollow sound means it's ripe. In a test suite, it means something else entirely.

Not thumped yet

Extra tips

Watermelons are in season in summer. Test suites are in season on every diff. Buy local: mutate the lines you changed, not the whole orchard.

Store at room temperature, chill after cutting. Run mutastic while you work. Let the threshold cool you down before you merge.

One melon is a snack, forty is a problem. If a run hands you forty missed mutants, start with the one on the line you just wrote.

Avoid these

  • Tests with no assertions at all
  • assertTrue(true), however confident it sounds
  • Mocks all the way down
  • A snapshot of the entire universe
  • Tests that only check the happy path at 50
  • A score your agent reported but nobody measured

Did you know?

Watermelon is about 92% water. Some test suites are about 92% setup.

Watermelon is both a fruit and a vegetable. A missed mutant is both a bug and a test you haven’t written yet.

There are more than 1,000 kinds of watermelon. mutastic’s first release ships five kinds of mutant. Give us time.

Japan has grown square watermelons. We have never seen a square test suite. We keep looking.

Bees help watermelon plants grow fruit. Mutants help test suites grow assertions.

Thumping is not a strategy.

Run mutastic run --since=main and know for sure.

Meet mutastic