Scott Madin

I have a bad habit of getting an idea for a software project, starting to write code without much planning ahead, very quickly discovering something I hadn’t considered or some tool I’m lacking, deciding to pivot to writing a separate utility or library to address that problem so that I can come back to the original thing later, and starting to write code for the new project without much planning ahead.

You can see where this is going, and it’s not Finishedprojectsville.

Over the years, though, I have managed to accumulate some tools to help counteract these tendencies. When I’m working on hobby projects, I usually like to work in plain C, with as little scaffolding as I can get away with — Vim will do nicely, thank you — but even so it turns out there’s some boilerplate that it’s tedious to retype every time I start a new project. So I wrote a Bash script that defines a function cproj():

/home/smadin $ cproj foo
/home/smadin $ ls
foo
/home/smadin $ cd foo
/home/smadin/foo $ ls -F
include/ Makefile src/ test/
/home/smadin/foo $ ls include
foo.h
/home/smadin/foo $ ls src
foo.c main.c
/home/smadin/foo $ ls test
foo_test.c Makefile test.h
/home/smadin/foo $ make
mkdir -p obj bin
/usr/bin/gcc -Wall -Werror -Iinclude -c -o obj/main.o src/main.c
/usr/bin/gcc -Wall -Werror -Iinclude -c -o obj/foo.o src/foo.c
make -C test
make[1]: Entering directory '/home/smadi_000/dev/foo/test'
/usr/bin/gcc -g -Wall -Werror -I. -I../include -o foo_test.exe foo_test.c ../obj/foo.o
./foo_test.exe > foo_test.log
make[1]: Leaving directory '/home/smadi_000/dev/foo/test'
/usr/bin/gcc  -o bin/foo.exe obj/main.o obj/foo.o
/home/smadin/foo $ cat test/foo_test.log
foo_test
initializing test data...
...done
test 0 of 1...
test_foo_dummy()...
foo_dummy() returned 42, expected 42
foo_test: 1 tests passed out of 1
/home/smadin/foo $

As you can see, cproj() creates a very basic skeleton for a C project, including a very primitive, hand-rolled unit-testing facility. test.h just defines an array of function pointers and manually populates it with the declared test functions:

#ifndef FOO_TEST_H
#define FOO_TEST_H

void init_test_data();

int test_foo_dummy();

int (*test_array[])() = {
    test_foo_dummy,
};

#define FOO_NUM_TESTS sizeof(test_array)/sizeof(test_array[0])

#endif /* FOO_TEST_H */

And foo_test.h:main() simply iterates over the array, breaking as soon as a test returns false. This has the advantage of being very simple and completely self-contained — the cproj() function contains the complete here-documents into which the project name is interpolated, so the only dependencies are the script itself and GCC — but also the serious disadvantage of being…very simple. There are no assertions, so every test case has to do checks and any logging manually, and return a true or false value accordingly. A single failure aborts the entire test harness, preventing accurate logging of pass/fail rate. There’s no facility for defining separate suites of test cases for related functionality, all managed by the same test harness, for more helpful reporting. And worst of all, it’s completely static: the user has to (that is, I have to) manually enter each test case function name 1) in the source file where I define the test case, 2) in the header where I declare the test case function, and 3) in the header where I define the function pointer array.

At this point you are no doubt thinking, “why on earth aren’t you just using one of the many existing unit-testing frameworks, which would solve these problems for you?” Excellent question! Now, moving on.

What I want, and I suspect what most programmers want, out of a unit test framework is for it to cause as little friction as possible. I want to have to think about the framework as little as possible in order to write tests and have them run and provide me useful output, so that in writing tests I’m spending almost all my mental energy on what I’m testing, how it’s supposed to work, and how it might fail. To remedy the defects of the current cproj approach, what I want is:

• assertions (true/false, null/not null, equal/not equal, equal/not equal for strings) with some kind of inherent expected/actual logging, so I don’t have to write those sprintf()s in each test function
• a boilerplate test harness I don’t have to write each time, which can run all the test cases without aborting on the first failure, and report a total pass rate
• test cases split into suites (with appropriate reporting in the harness) so I don’t have to put everything in one big source file
• dynamic discovery of tests so I don’t have to copy and paste each function name to three different places
• minimal extra headers or other files so that integrating testing into a project is as simple as possible

And I think I can get there, or most of the way there, with one header and one shell script: the header to define assertion macros and so forth, and the script to scan source files (test suites) in the test subdirectory, parse out the suite and test case names, and generate the per-suite headers and the harness source file. I’ve been experimenting with this idea, and I’m partway there already.

More on this soon.