Skip to content

Package

PonyCheck is a library for property based testing with tight integration into PonyTest.

Property Based Testing

In traditional unit testing, the developer specifies one or more input examples manually for the class or system under test and asserts on certain output conditions. The difficulty here is to find enough examples to cover all branches and cases of the class or system under test.

In property based testing, the developer defines a property, a kind of predicate for the class or system under test that should hold for all kinds or just a subset of possible input values. The property based testing engine then generates a big number of random input values and checks if the property holds for all of them. The developer only needs to specify the possible set of input values using a Generator.

This testing technique is great for finding edge cases that would easily go unnoticed with manually constructed test samples. In general it can lead to much higher coverage than traditional unit testing, with much less code to write.

How PonyCheck implements Property Based Testing

A property based test in PonyCheck consists of the following:

  • A name (mostly for integration into PonyTest)
  • One or more generators, depending on how your property is laid out. There are tons of them defined in the primitive Generators.
  • A property method that asserts a certain property for each sample generated by the Generator(s) with the help of PropertyHelper, which tries to expose a similar API as TestHelper.
  • Optionally, the method params() can be used to configure how PonyCheck executes the property by specifying a custom PropertyParams object.

The classical list-reverse example:

use "collections"
use "pony_check"

class ListReverseProperty is Property1[List[USize]]
  fun name(): String => "list/reverse"

  fun gen(): Generator[List[USize]] =>
    Generators.list_of[USize](Generators.usize())

  fun property(arg1: List[USize], ph: PropertyHelper) =>
    ph.array_eq[USize](arg1, arg1.reverse().reverse())

Integration into PonyTest

There are two ways of integrating a Property into PonyTest:

  1. In order to pass your Property to the PonyTest engine, you need to wrap it inside a Property1UnitTest.
  actor Main is TestList
    new create(env: Env) => PonyTest(env, this)

    fun tag tests(test: PonyTest) =>
      test(Property1UnitTest[String](MyStringProperty))
  1. Run as many Properties as you wish inside one PonyTest UnitTest using the convenience function PonyCheck.for_all providing a Generator, the TestHelper and the actual property function. (Note that the property function is supplied in a second application of the result to for_all.)
  class ListReversePropertyWithinAUnitTest is UnitTest
    fun name(): String => "list/reverse/forall"

    fun apply(h: TestHelper) =>
      let gen = recover val Generators.list_of[USize](Generators.usize()) end
      PonyCheck.for_all[List[USize]](gen, h)(
        {(sample, ph) =>
          ph.array_eq[Usize](arg1, arg1.reverse().reverse())
        })
      // ... possibly more properties, using `PonyCheck.for_all`

Independently of how you integrate with PonyTest, the PonyCheck machinery will instantiate the provided Generator, and will execute it for a configurable number of samples.

If the property fails using an assertion method of PropertyHelper, the failed example will be shrunken by the generator to obtain a smaller and more informative, still failing, sample for reporting.

Public Types