Progressive testing trees

The current common way that testing libraries organize tests is “mono-tests”, which each have individual setup and teardown. However, many tests likely use the same setup, hence callbacks like beforeEach, afterEach, beforeAll, and afterAll. In a testing framework like Jest, it might look like this:

describe('Widget', () => {
  beforeEach(() => {
    doSetup();
  });
  
  afterEach(() => {
    doTeardown();
  });
  
  it('should wobble', () => {
    const testValue = getTestValue();
    const result = doThing();
    expect(result).toEqual(testValue);
  });
  
  it ('should weeble', () => {
    const testValue = getTestValue();
    const result = doThing();
    expect(result).toEqual(testValue);
  });
});

The beforeEach() function consolidates the setup logic of these tests, but that logic still runs prior to each test, so the actual runtime of the setup procedure is not consolidated. The state of the test suite is destroyed by doTeardown(), so that state is wasted and extra time must be taken to reconstruct that state for the next test.

In order to consolidate the runtime procedure, we can use beforeAll and afterAll instead of beforeEach and afterEach. This will ensure that all of the state necessary to run tests is persisted as long as it’s needed, and not torn down prematurely.

Another way to consolidate the runtime is to create a test inside the test, which only runs if the preceding test succeeds. In Jest, it might look like this:

describe('Widget', () => {
  beforeEach(() => {
    doSetup();
  });
  
  it('should wobble', () => {
    const testValue = getTestValue();
    const result = doThing();
    expect(result).toEqual(testValue);
    
    it ('should weeble', () => {
      const testValue = getTestValue();
      const result = doThing();
      expect(result).toEqual(testValue);
    });
  });
});

This presents an interesting idea: progressive trees of tests.^[1]

Designers often design applications using “stories”, which provide narratives of usage scenarios and interaction paths. Functional tests can employ this same concept. An initial (“root”) test starts by testing the entrypoint/initial interaction of an application. If that test succeeds, the testing runtime continues to the first branch/child test of that initial test, which may test a particular usage scenario. If that test passes, the runtime continues to another branch or returns to the “root” to find the next branch to follow. Etc. In this way, tests can be conceptualized as nodes on a tree.

Further, the state of the tests will be passed from parent nodes to child nodes, making the tests “progressive”. This also allows for setup to be as global or local as needed depending on where that setup happens along the current branch.

Trees are a natural structure for conceptualizing functional tests.