Next I’ll start a real project. Or maybe I’ll just discover another bit of tooling that needs revision.
A couple of weeks ago, when I got started on what became Scuttle, I mentioned that the reason I wanted a simple but adequately featureful plain-C unit testing framework was that I was frustrated with the limitations of the more ad-hoc solution I’d built into my old cproj() shell script. It creates an even simpler test facility, but it was too simple.
Now that Scuttle v1.0.0 is done and published, I want to revisit Cproj. I haven’t touched this script in years, and I think there’s a lot of room to clean it up and make it more useful — and update it to use Scuttle as the testing framework it builds into the project skeletons it generates!
Much like Scuttle, the animating philosophy of Cproj is that it should be as self-contained as possible: the script contains its templates for the files it generates as here-documents, and runs pattern substitutions on them to produce the right output. The first time it’s run, the old Cproj actually writes out those templates to /etc/skel/proj/ (if run as root) or to $HOME/.skel/proj/, and on subsequent uses reads them from disk instead. I’m no longer sure there’s much value in doing that, so I’ll probably remove that functionality, and just use the here-docs directly every time, like Scuttle does.
After I finish fixing up Cproj, I should be ready to use my revised tooling to start a, you know, real project. Or maybe I’ll just discover another bit of tooling that needs revision.
Harvey Dent’s warning from The Dark Knight, “You either die a hero, or live long enough to see yourself become the villain,” is isomorphic to Zawinski’s Law, “Every program attempts to expand until it can read mail. Those programs which cannot so expand are replaced by ones which can.”
All right, it took me two weeks, but Scuttle v1.0.0 is now live. I think it’s pretty easy to use, and it’s very lightweight. The full README is over at github, of course, but it basically works as I outlined in the project introduction. No one really needed a new unit test framework, but I wrote one anyway, and now I’m going to use it in subsequent projects as often as I can.
Three years ago today, December 15th 2017, the best Star Wars movie to date — by the director who, to date, showed the best understanding of what Star Wars is about and what it’s for — hit theaters. So this is just a quick appreciation post for Rian Johnson and The Last Jedi, which was so much better than it needed to be, and whose most important themes were, as far as I can tell, immediately and thoughtlessly cast aside as soon as J.J. Abrams came back to the franchise.
Scuttle is now working as intended for the standard use case. As described in the introductory post, for a C project with a standard layout, adding unit testing with Scuttle requires the following steps:
Install scuttle.h either locally to the project or in the system include path
Install scuttle.sh either locally to the project or somewhere on your $PATH
Add test suite source files under test/, each corresponding to a module of your project, named test_<module>.c, using Scuttle’s simple macros
Add a test target to your main Makefile as follows:
bash scuttle.sh test
$(MAKE) -C test
Scuttle will generate test/Makefile, test/test_<module>.h and test/test_<module>_gen.h for each test/test_<module>.c suite, and the test/test_<project>.c harness; and the generated Makefile’s test target will build and run the test harness and pipe the output to test/log/test_<project>.log.
There’s a lot of room to make this more flexible and add some convenience features, but in terms of functionality it’s at MVP level now.
Tomorrow I’ll do a quick cleanup pass, make a few small tweaks I’ve already thought of, write some basic documentation, and actually push the code up to a public repo so if anyone’s interested in trying it out, it’ll be available.
Over at Ars, Richard Jensen has a great article on how the C programming language came to be. I love dives like this into the various particulars and contingencies involved in the development of things we now think of as more-or-less having “always been there”. It’s easier to do with computer technology than many other things, too, because a lot of the people who were directly involved are still alive, and can talk about why they made one decision or another.
It’s a good reminder, too, that very little of how we generally think “the world is” was inevitable—or is immutable. On the other hand, of course, the longer particular arrangements are accepted as the default without examination of their roots, the more inertia they have, and the more effort it takes to make change.
Just think: we might have ended up with 10-, 12-, even 18-bit “bytes” as the basis for our computing technology; or even trinary logic circuitry, with a three-state “trit” as the smallest unit (after all, at the circuit level, a “0” just means there’s currently 0 volts on the line, and a “1” means there’s 5 volts, but there’s no rule that says other signal levels couldn’t exist).
A basic principle of engineering, software or otherwise, is “don’t reinvent the wheel.” I’m gonna anyhow.
Last week I wrote about wanting a very lightweight C unit testing solution, and, in my bullheaded way, wanting to write it myself rather than learn an already-existing system. It’s a commonplace in software engineering that the impulse to start over from scratch is one of the worst habits of programmers; one way or another, reinventing the wheel means doing lots of work you didn’t have to, usually to get a less satisfactory result. Either you end up hand-rolling something you should have just used a library for; or you look at a mass of legacy code full of cryptic comments and weird edge-case handling, think “whew, what a mess, how hard to maintain! better to just scrap it all and start fresh, so it will be cleaner and more elegant,” only to discover along the way that all those weird edge cases really exist, and end up with just as patchwork a code base as the one you meant to improve.
All that said, I’m doing it anyway. It’s not like I’m busy right now, and this is what’s engaging my brain.
So, suppose you have a project projname, and you want to add unit testing. Your project layout looks like this:
~/projname$ ls include
~/projname$ ls src
You want to add unit tests for the foo module, which is sensible because it’s pretty complex and you want to be sure all of it works exactly right. The foo module looks like this:
#endif /* _FOO_H */
To add unit testing with Scuttle (let’s say it stands for “simple C unit testing tool, limited edition”), you just need to put scuttle.h in your include path and scuttle.sh in your executable path, create a test/ directory, and add test_projname_foo.c:
printf("foo suite init\n");
printf("foo test setup\n");
printf("foo test teardown\n");
int i = foo();
int i = foo();
SREFUTE(i == 69)
You’ll have noticed that no test_projname_foo.h header exists: scuttle.sh will generate that, as well as a test/test_projname_foo_gen.c source file with some data structures, and test/Makefile, for you.
~/projname$ ls test/
This is Scuttle, v1.0.0.
* found suite test/test_projname_foo.c
* generated suite header test/test_projname_foo.h
* generated suite data test/test_projname_foo_gen.c
* generated harness test/test_projname.c
* generated makefile test/Makefile
Type 'make -C test/ test' to build and run your test harness.
~/projname$ ls test/
~/projname$ make -C test/ test[$(CC) output]
test/test_projname > test/log/test_projname.log
~/projname$ cat test/log/test_projname.log
This is Scuttle, v1.0.0.
Running test harness for: projname
Test suite projname_foo:
*** Suite passed: 2 / 2 tests passed.
*** 1 / 1 suites passed
Aside from the simple SASSERT(x) and SREFUTE(x), Scuttle provides convenience macros SASSERT_NULL(x), SASSERT_EQ(x,y), and SASSERT_STREQ(x,y), as well as SREFUTE versions of the same.
As of this writing, scuttle.h is at or near v1.0 completeness, targets for scuttle.sh‘s output are determined, and I’m beginning work on scuttle.sh itself. More on those soon.
At this point you are no doubt thinking, “why on earth aren’t you just using one of the many existing frameworks, which would solve these problems for you?” Excellent question! Now, moving on.
I have a bad habit of getting an idea for a software project, starting to write code without much planning ahead, very quickly discovering something I hadn’t considered or some tool I’m lacking, deciding to pivot to writing a separate utility or library to address that problem so that I can come back to the original thing later, and starting to write code for the new project without much planning ahead.
You can see where this is going, and it’s not Finishedprojectsville.
Over the years, though, I have managed to accumulate some tools to help counteract these tendencies. When I’m working on hobby projects, I usually like to work in plain C, with as little scaffolding as I can get away with — Vim will do nicely, thank you — but even so it turns out there’s some boilerplate that it’s tedious to retype every time I start a new project. So I wrote a Bash script that defines a function cproj():
/home/smadin $ cproj foo
/home/smadin $ ls
/home/smadin $ cd foo
/home/smadin/foo $ ls -F
include/ Makefile src/ test/
/home/smadin/foo $ ls include
/home/smadin/foo $ ls src
/home/smadin/foo $ ls test
foo_test.c Makefile test.h
/home/smadin/foo $ make
mkdir -p obj bin
/usr/bin/gcc -Wall -Werror -Iinclude -c -o obj/main.o src/main.c
/usr/bin/gcc -Wall -Werror -Iinclude -c -o obj/foo.o src/foo.c
make -C test
make: Entering directory '/home/smadi_000/dev/foo/test'
/usr/bin/gcc -g -Wall -Werror -I. -I../include -o foo_test.exe foo_test.c ../obj/foo.o
./foo_test.exe > foo_test.log
make: Leaving directory '/home/smadi_000/dev/foo/test'
/usr/bin/gcc -o bin/foo.exe obj/main.o obj/foo.o
/home/smadin/foo $ cat test/foo_test.log
initializing test data...
test 0 of 1...
foo_dummy() returned 42, expected 42
foo_test: 1 tests passed out of 1
As you can see, cproj() creates a very basic skeleton for a C project, including a very primitive, hand-rolled unit-testing facility. test.h just defines an array of function pointers and manually populates it with the declared test functions:
And foo_test.h:main() simply iterates over the array, breaking as soon as a test returns false. This has the advantage of being very simple and completely self-contained — the cproj() function contains the complete here-documents into which the project name is interpolated, so the only dependencies are the script itself and GCC — but also the serious disadvantage of being…very simple. There are no assertions, so every test case has to do checks and any logging manually, and return a true or false value accordingly. A single failure aborts the entire test harness, preventing accurate logging of pass/fail rate. There’s no facility for defining separate suites of test cases for related functionality, all managed by the same test harness, for more helpful reporting. And worst of all, it’s completely static: the user has to (that is, I have to) manually enter each test case function name 1) in the source file where I define the test case, 2) in the header where I declare the test case function, and 3) in the header where I define the function pointer array.
At this point you are no doubt thinking, “why on earth aren’t you just using one of the many existing unit-testing frameworks, which would solve these problems for you?” Excellent question! Now, moving on.
What I want, and I suspect what most programmers want, out of a unit test framework is for it to cause as little friction as possible. I want to have to think about the framework as little as possible in order to write tests and have them run and provide me useful output, so that in writing tests I’m spending almost all my mental energy on what I’m testing, how it’s supposed to work, and how it might fail. To remedy the defects of the current cproj approach, what I want is:
assertions (true/false, null/not null, equal/not equal, equal/not equal for strings) with some kind of inherent expected/actual logging, so I don’t have to write those sprintf()s in each test function
a boilerplate test harness I don’t have to write each time, which can run all the test cases without aborting on the first failure, and report a total pass rate
test cases split into suites (with appropriate reporting in the harness) so I don’t have to put everything in one big source file
dynamic discovery of tests so I don’t have to copy and paste each function name to three different places
minimal extra headers or other files so that integrating testing into a project is as simple as possible
And I think I can get there, or most of the way there, with one header and one shell script: the header to define assertion macros and so forth, and the script to scan source files (test suites) in the test subdirectory, parse out the suite and test case names, and generate the per-suite headers and the harness source file. I’ve been experimenting with this idea, and I’m partway there already.
Blogs were these things we used to have, and the internet was better then. Correlation isn’t causation, but still, one has to wonder.
Blogs were these things we used to have, and the internet was better then. Correlation isn’t causation, but still, one has to wonder. I had a blog once, and sometimes some of my posts were very mildly popular. I helped moderate a much bigger blog, and I participated at some other blog communities. All of that kind of withered, or maybe I just fell away from it, as social media became the dominant mode of internet interaction. But what if it didn’t have to be? What if: blogs, again?
Anyway, I guess it’s worth a shot. I need something more productive to do with my time than constantly being mad because I saw a bad take on Twitter — Twitter is nothing if not an endless source of all the bad takes you could ever get mad about, and then some — and it’s hard to look at the IndieWeb movement and not think, “you know, they might be onto something there.” Individuals controlling their own space and experience on the web, using open protocols and prioritizing accessibility and interoperability over the interests of a for-profit corporation that controls a massive platform? Sounds all right.
I’ve been saying for years that Twitter’s bad for me, and I don’t think I’ll be there too much longer, now. Let’s try blogs again.