Shtory update; also, audio

Sometimes things take longer, but also sometimes I get distracted.

Well, I said “by the end of the month,” but technically I didn’t say which month.

Anyway I’m continuing to work on Shtory. I’ve had my usual struggle to balance “drilling down into tiny details instead of looking at the whole picture” with “drawing high-level block diagrams and blithely assuming the implementation will be trivial” and also with “just plunging ahead writing code without much plan, in the hopes that it will somehow come together.”

I’ve realized I need to add join and quit commands, and that I need to try to get all this PlantUML/Graphviz business working, and sort out the VSCode plugin situation, so that I can write myself out some sequence diagrams for how things are supposed to communicate over the sockets.

Also, because who can just keep their attention on one project at a time, I’ve gotten interested in DIY audio electronics — specifically guitar amps, which I dabbled in about a million years ago, and associated stuff. I used to hang out on the Solid State Guitar hobbyist forums, and I got an email recently that the guy who runs them had launched a Kickstarter for a 9V-battery amp kit of his own design, based, like my previous efforts, on the venerable and ubiquitous LM386 amplifier IC. The “Honey” amp kit has beaten its funding goal, so I’m looking forward to receiving my kit once those get shipped out; and in the meantime it’s stirred up a bunch of ideas from the dusty corners of my brain.

The first one is that I should repair the old Ruby circuit I housed in that Balvenie packing tube, and the second one is that I should actually do something with the components I still have lying around — including a pretty nice 10″ Jensen speaker, a big transformer, and an LM3886 chip —that I once meant to build a ~40-watt amp from.

The third one is that I could build a cabinet to house that speaker — quarter sheet of 1/2″ plywood with a nice hardwood veneer, splined miter corner joints with internal corner braces, 14″x14″ face plus 2″ high instrument panel, 8″ deep, yes I’ve already sketched out the cut list — but wire the speaker to a jack, and use the combo cab as a modular platform for trying out different amplifier circuits with different power supplies, features, etc. Even the little ~1/2-watt Ruby can drive a proper full-size speaker, so as long as all the amp circuits are designed for an 8Ω speaker (and don’t put out more than 50W) that should pose no problem. I’d also like to try building another Ruby or similar design, but add in some extras like an effects loop or two, the recommended bass-boost circuitry (perhaps with a bypass switch), a headphone jack, etc. Ideally I could design the cabinet so that swapping in my original Ruby board, the Honey, such a modified Ruby, or even the big LM3886-based design if I ever get around to making that work, would be quick and easy.

“But Scott,” you might ask, “are you actually any good at playing guitar?”

Ha! You’re funny. No, of course I’m not.

Shtory: update

Work on Shtory continues, but has been a bit slow. The first three weeks of January were a pretty wild year, what with the fascist coup attempt which nearly resulted in members of Congress being lynched, and in the end just barely managing to keep some semblance of a representative democracy intact long enough that now we have a chance to actually improve things. So it was a little hard to concentrate for a while there.

That said, I still expect to have Shtory up to nearly-MVP-level functionality this week, and “ready enough” to put up on github by the end of the month.

The tentative feature list for the V1 milestone is:

  • local operation only — no following remote users
  • shtory list command and lisht alias (also the default behavior of shtory with no arguments): list users with current stories, marking users with unread stories with a *
  • shtory post command and posht alias: read stdin until EOF, then post to current user’s story
  • shtory read command: read all unread posts from followed users
  • shtory read <user> command variant: read all current (read and unread) posts from specified user, whether or not current user follows them, if they have not blocked the current user
  • shtory follow <user> command: follow specified user, if user exists and has not blocked current user
  • shtory unfollow <user> command: unfollow specified user, if current user follows them
  • shtory block <user> command: block specified user from following current user or seeing their stories
  • shtory unblock <user>: remove specified user from block list, allowing them to see stories from and follow current user if they choose to

New project: Shtory

Nothing’s funnier than a joke about something people have long since stopped talking about. Introducing shtory: stories, for the Unix shell. You’re welcome, and I’m sorry.

First Snapchat introduced its “stories” feature as a broadcast alternative to its original model of sending self-destructing snaps directly to individual users; then Facebook was rebuffed in trying to buy Snapchat and Mark Zuckerberg directed Instagram to reproduce an identical “stories” feature in Instagram, prompting all kinds of jokes about what software would have “stories” next.

from Know Your Meme, a photoshopped image of Microsoft Excel with "stories" — a row of usernames and icons between the toolbars and the spreadsheet proper, just like in Snapchat and Instagram.
Excel, perhaps

Then in 2020, Twitter announced its — also identical — feature, this time called “Fleets” (see, like “tweets”, but they’re “fleeting”), and there was a whole new round of jokes.

In particular, Jef Poskanzer tweeted:

Tweet from Jef Poskanzer (@jefposk), November 18, 2020, reading “/bin/sh has stories now too.”, with an attached image of a shell prompt under a row of ASCII-art faces

And I thought, “heh. that’s pretty funny.”

And then I thought, “you know, I bet I could actually write a program to do that.”

And then I thought, “that’s a terrible idea.”

So, obviously I’m doing it. Introducing my new project: shtory — Snapchat/Instagram/Twitter-style stories, for the Unix shell. You’re welcome, and I’m sorry.

Everyone knows nothing’s funnier than a joke about something people have long since stopped talking about, so in keeping with that principle I hope to have an alpha of shtory ready to put up on github by about the end of the month. Ultimately the concept here isn’t very dissimilar to the traditional Unix utility finger, which displays the .plan and .project files, if any, that a user has in their home directory; but individual poshts in a shtory will, like their social media inspirations, only live for 24 hours before being automatically deleted, and I plan on implementing more granular privacy controls of the type we’re used to in modern social media, like follow lists, blocking, locked accounts, mutual-only posts, etc.

Cproj: v1.0.0 is live!

I’ve just pushed v1.0.0 of Cproj up to Github. Usage is very simple: source (I have a line in my .bashrc to do that when my shell starts up), then type cproj <projname>. Cproj creates a <projname> directory and populates it with a ready-to-build project skeleton including source and header files, Makefile, and basic test suite using Scuttle. If Scuttle isn’t installed on the system (Cproj checks the default installation location /usr/local), Cproj attempts to download it from the main Github repository.

You can immediately cd <projname>; make to build the project and unit tests and run the test harness.

New project: Cproj

Next I’ll start a real project. Or maybe I’ll just discover another bit of tooling that needs revision.

A couple of weeks ago, when I got started on what became Scuttle, I mentioned that the reason I wanted a simple but adequately featureful plain-C unit testing framework was that I was frustrated with the limitations of the more ad-hoc solution I’d built into my old cproj() shell script. It creates an even simpler test facility, but it was too simple.

Now that Scuttle v1.0.0 is done and published, I want to revisit Cproj. I haven’t touched this script in years, and I think there’s a lot of room to clean it up and make it more useful — and update it to use Scuttle as the testing framework it builds into the project skeletons it generates!

Much like Scuttle, the animating philosophy of Cproj is that it should be as self-contained as possible: the script contains its templates for the files it generates as here-documents, and runs pattern substitutions on them to produce the right output. The first time it’s run, the old Cproj actually writes out those templates to /etc/skel/proj/ (if run as root) or to $HOME/.skel/proj/, and on subsequent uses reads them from disk instead. I’m no longer sure there’s much value in doing that, so I’ll probably remove that functionality, and just use the here-docs directly every time, like Scuttle does.

After I finish fixing up Cproj, I should be ready to use my revised tooling to start a, you know, real project. Or maybe I’ll just discover another bit of tooling that needs revision.

Scuttle: v1.0.0 is live!

All right, it took me two weeks, but Scuttle v1.0.0 is now live. I think it’s pretty easy to use, and it’s very lightweight. The full README is over at github, of course, but it basically works as I outlined in the project introduction. No one really needed a new unit test framework, but I wrote one anyway, and now I’m going to use it in subsequent projects as often as I can.

I hope other people also find it useful!

Scuttle: pre-update update

Scuttle is now working as intended for the standard use case. As described in the introductory post, for a C project with a standard layout, adding unit testing with Scuttle requires the following steps:

  1. Install scuttle.h either locally to the project or in the system include path
  2. Install either locally to the project or somewhere on your $PATH
  3. Add test suite source files under test/, each corresponding to a module of your project, named test_<module>.c, using Scuttle’s simple macros
  4. Add a test target to your main Makefile as follows:
    bash test
    $(MAKE) -C test

Scuttle will generate test/Makefile, test/test_<module>.h and test/test_<module>_gen.h for each test/test_<module>.c suite, and the test/test_<project>.c harness; and the generated Makefile’s test target will build and run the test harness and pipe the output to test/log/test_<project>.log.

There’s a lot of room to make this more flexible and add some convenience features, but in terms of functionality it’s at MVP level now.

Tomorrow I’ll do a quick cleanup pass, make a few small tweaks I’ve already thought of, write some basic documentation, and actually push the code up to a public repo so if anyone’s interested in trying it out, it’ll be available.

New project: Scuttle

A basic principle of engineering, software or otherwise, is “don’t reinvent the wheel.” I’m gonna anyhow.

Last week I wrote about wanting a very lightweight C unit testing solution, and, in my bullheaded way, wanting to write it myself rather than learn an already-existing system. It’s a commonplace in software engineering that the impulse to start over from scratch is one of the worst habits of programmers; one way or another, reinventing the wheel means doing lots of work you didn’t have to, usually to get a less satisfactory result. Either you end up hand-rolling something you should have just used a library for; or you look at a mass of legacy code full of cryptic comments and weird edge-case handling, think “whew, what a mess, how hard to maintain! better to just scrap it all and start fresh, so it will be cleaner and more elegant,” only to discover along the way that all those weird edge cases really exist, and end up with just as patchwork a code base as the one you meant to improve.

All that said, I’m doing it anyway. It’s not like I’m busy right now, and this is what’s engaging my brain.

So, suppose you have a project projname, and you want to add unit testing. Your project layout looks like this:

~/projname$ ls
~/projname$ ls include
~/projname$ ls src

You want to add unit tests for the foo module, which is sensible because it’s pretty complex and you want to be sure all of it works exactly right. The foo module looks like this:


#ifndef _FOO_H
#define _FOO_H

int foo();

#endif /* _FOO_H */


#include "foo.h"

int foo()
    return 42;

To add unit testing with Scuttle (let’s say it stands for “simple C unit testing tool, limited edition”), you just need to put scuttle.h in your include path and in your executable path, create a test/ directory, and add test_projname_foo.c:

#include "foo.h"
#include "test_projname_foo.h"
#include "scuttle.h"
#include <stdio.h>

    printf("foo suite init\n");

    printf("foo test setup\n");

    printf("foo test teardown\n");

    int i = foo();
    SASSERT_EQ(42, i)

    int i = foo();
    SREFUTE(i == 69)

You’ll have noticed that no test_projname_foo.h header exists: will generate that, as well as a test/test_projname_foo_gen.c source file with some data structures, and test/Makefile, for you.

~/projname$ ls test/
This is Scuttle, v1.0.0.
    * found suite test/test_projname_foo.c
    * generated suite header test/test_projname_foo.h
    * generated suite data test/test_projname_foo_gen.c
    * generated harness test/test_projname.c
    * generated makefile test/Makefile
Type 'make -C test/ test' to build and run your test harness.
~/projname$ ls test/
~/projname$ make -C test/ test
[$(CC) output]
test/test_projname > test/log/test_projname.log
~/projname$ cat test/log/test_projname.log
This is Scuttle, v1.0.0.
Running test harness for: projname

Test suite projname_foo:
 *** Suite passed: 2 / 2 tests passed.
 *** 1 / 1 suites passed

Aside from the simple SASSERT(x) and SREFUTE(x), Scuttle provides convenience macros SASSERT_NULL(x), SASSERT_EQ(x,y), and SASSERT_STREQ(x,y), as well as SREFUTE versions of the same.

As of this writing, scuttle.h is at or near v1.0 completeness, targets for‘s output are determined, and I’m beginning work on itself. More on those soon.

Testing Tuesday

At this point you are no doubt thinking, “why on earth aren’t you just using one of the many existing frameworks, which would solve these problems for you?” Excellent question! Now, moving on.

I have a bad habit of getting an idea for a software project, starting to write code without much planning ahead, very quickly discovering something I hadn’t considered or some tool I’m lacking, deciding to pivot to writing a separate utility or library to address that problem so that I can come back to the original thing later, and starting to write code for the new project without much planning ahead.

You can see where this is going, and it’s not Finishedprojectsville.

Over the years, though, I have managed to accumulate some tools to help counteract these tendencies. When I’m working on hobby projects, I usually like to work in plain C, with as little scaffolding as I can get away with — Vim will do nicely, thank you — but even so it turns out there’s some boilerplate that it’s tedious to retype every time I start a new project. So I wrote a Bash script that defines a function cproj():

/home/smadin $ cproj foo
/home/smadin $ ls
/home/smadin $ cd foo
/home/smadin/foo $ ls -F
include/ Makefile src/ test/
/home/smadin/foo $ ls include
/home/smadin/foo $ ls src
foo.c main.c
/home/smadin/foo $ ls test
foo_test.c Makefile test.h
/home/smadin/foo $ make
mkdir -p obj bin
/usr/bin/gcc -Wall -Werror -Iinclude -c -o obj/main.o src/main.c
/usr/bin/gcc -Wall -Werror -Iinclude -c -o obj/foo.o src/foo.c
make -C test
make[1]: Entering directory '/home/smadi_000/dev/foo/test'
/usr/bin/gcc -g -Wall -Werror -I. -I../include -o foo_test.exe foo_test.c ../obj/foo.o
./foo_test.exe > foo_test.log
make[1]: Leaving directory '/home/smadi_000/dev/foo/test'
/usr/bin/gcc  -o bin/foo.exe obj/main.o obj/foo.o
/home/smadin/foo $ cat test/foo_test.log
initializing test data...
test 0 of 1...
foo_dummy() returned 42, expected 42
foo_test: 1 tests passed out of 1
/home/smadin/foo $

As you can see, cproj() creates a very basic skeleton for a C project, including a very primitive, hand-rolled unit-testing facility. test.h just defines an array of function pointers and manually populates it with the declared test functions:

#ifndef FOO_TEST_H
#define FOO_TEST_H

void init_test_data();

int test_foo_dummy();

int (*test_array[])() = {

#define FOO_NUM_TESTS sizeof(test_array)/sizeof(test_array[0])

#endif /* FOO_TEST_H */

And foo_test.h:main() simply iterates over the array, breaking as soon as a test returns false. This has the advantage of being very simple and completely self-contained — the cproj() function contains the complete here-documents into which the project name is interpolated, so the only dependencies are the script itself and GCC — but also the serious disadvantage of being…very simple. There are no assertions, so every test case has to do checks and any logging manually, and return a true or false value accordingly. A single failure aborts the entire test harness, preventing accurate logging of pass/fail rate. There’s no facility for defining separate suites of test cases for related functionality, all managed by the same test harness, for more helpful reporting. And worst of all, it’s completely static: the user has to (that is, I have to) manually enter each test case function name 1) in the source file where I define the test case, 2) in the header where I declare the test case function, and 3) in the header where I define the function pointer array.

At this point you are no doubt thinking, “why on earth aren’t you just using one of the many existing unit-testing frameworks, which would solve these problems for you?” Excellent question! Now, moving on.

What I want, and I suspect what most programmers want, out of a unit test framework is for it to cause as little friction as possible. I want to have to think about the framework as little as possible in order to write tests and have them run and provide me useful output, so that in writing tests I’m spending almost all my mental energy on what I’m testing, how it’s supposed to work, and how it might fail. To remedy the defects of the current cproj approach, what I want is:

  • assertions (true/false, null/not null, equal/not equal, equal/not equal for strings) with some kind of inherent expected/actual logging, so I don’t have to write those sprintf()s in each test function
  • a boilerplate test harness I don’t have to write each time, which can run all the test cases without aborting on the first failure, and report a total pass rate
  • test cases split into suites (with appropriate reporting in the harness) so I don’t have to put everything in one big source file
  • dynamic discovery of tests so I don’t have to copy and paste each function name to three different places
  • minimal extra headers or other files so that integrating testing into a project is as simple as possible

And I think I can get there, or most of the way there, with one header and one shell script: the header to define assertion macros and so forth, and the script to scan source files (test suites) in the test subdirectory, parse out the suite and test case names, and generate the per-suite headers and the harness source file. I’ve been experimenting with this idea, and I’m partway there already.

More on this soon.