I once had a spell looking at a lot of Makefiles. It prompted this ode to the Humble Make Stanza.

A make Refresher

Make is a tool to control the rebuilding of files so that unnecessary work is avoided. The simplest construct in make is a stanza:

target(s) : prerequisite(s)
    script(s)

Meaning: If the target file doesn't exist or if one of the prerequisite files is newer, then rebuild the target file(s) by running script(s). Here's a specific example:

foo.o : foo.c
    gcc -O -c foo.c -o foo.o

"If foo.c is newer than foo.o, rebuild foo.o by running gcc -O -c foo.c".

There's really no magic. You could just as easily write:

foo.o : foo.c
    echo my granny wears army boots

which is barking mad, but absolutely fine by make.

Files. Timestamps. Shell scripts (Bourne shell). What could be simpler?

make in Full Glory

The ideal make stanza does a single step in a potentially complex workflow. make stitches together many such stanzas into a beautiful harmonious whole. Let us remind ourselves of what we are hoping for:

• The make infrastructure will capture the whole workflow. Type make world and the full workflow bhoona will unfold before your eyes, the right steps in the right order.

• When something changes, the steps that must be re-done will definitely be re-run. (Anything else is a disaster.)

• When something changes, only the steps that must be re-done will be re-run. Not essential but preferred. For example: it's a pain that, when you change a comment in a header file, make will re-compile great swathes of source code. We live with it.

• When something goes wrong, make will not proceed pointlessly (e.g. do a doomed seven-hour simulation even though the preceding compilation failed) and will make it relatively clear where matters came unstuck.

• If a previous run was interrupted, make will restart at the right place (and not be confused).

• If the IT infrastructure has provision for parallel simulations (e.g. a batch queuing system), make will use that appropriately.

The Naked Stanza

You will undoubtedly not recognize your own experience with make from the preceding idyllic description. To understand why, let's look more closely at the soul of the make party, the stanza:

target(s) [hidden t's] : prerequisite(s) [hidden prereq's]
    script(s)

The core task to be performed by the script -- e.g. "run VCS with these command-line arguments" -- is often straightforward and easy to get right. So where do problems arise?

• The script(s) will read a bunch of files -- prerequisite(s) -- and probably write at least one -- target(s).

  Unless carefully done, the scripts will probably have hidden prerequisites (files read but not specifically listed in the Makefile) and hidden targets (files written but similarly unlisted).

• If you don't attend to the "hidden" prerequisites (e.g. by buying ClearCase and then using clearmake, which tracks such dependencies through "audited builds"), then you should stop using make straight away -- you have no assurance that crucial rebuilding will happen when it really needs to.

  (Of course, there is the make equivalent of the Windows re-install: in the face of the slightest hiccup, type make clean. Why someone with a lousy Makefile trusts the clean target continues to elude me, however.)

• Hidden targets are less of an issue. They may cause needless extra rebuilding by make, but that may not be a hanging offense.

• A problem that arises with some EDA tools is that they are disinclined to do a single unit of work (and then get out of the way). They often want to set up a hidden universe of their own (often with dubious Makefiles hidden somewhere), and/or they decide they should offer counseling before getting on with the job ("Mr Kelly, are you really sure you want to merge those two files? Click OK if you're absolutely certain you want to proceed, Cancel otherwise; this dialog box comes with no warranty expressed or implied.")

• The script(s) needs to offer an absolute assurance to the downstream parts of the overall "flow" that they have done their part successfully. This is usually by setting the exit status correctly (zero for OK, anything else for not).

  EDA tools are imperfect on this exit-status thing. Some just don't bother to set the exit-status to anything sensible. For others (notably simulators), the tool's idea of "not OK" may reasonably differ from your own.

  For example, a simulation may run to completion (exit 0) but it may have properties which mean its results must not be passed on to the next step of the workflow.

• The assurance of job-well-done also needs to be recorded for posterity in a file-with-timestamp (because that's make understands). This might be called a "witness" file -- it witnesses that a particular task was done successfully at a certain time.

  In many cases such as the .c-to-.o example given at the beginning, the .o file is the witness (as well as the object file that you really want). In its witness capacity, it is saying "A successful C compilation of foo.c took place on November 22 at 11:12am."

  In the more complex EDA world, I tend to favor explicit witness (or "stamp" or "sentinel") files: sim-4732-DONE witnesses that simulation 4732 completed successfully at the time indicated.

• Another thing the scripts need to do is leave a trail of what happened. This is often in the form of log files.

• Even better than "what happened" is "what happened that was interesting". A definite weakness of EDA flows is that they spew voluminously, and it is very easy to miss the One Crucial Message that shows something amiss.

The Model Stanza

The observations above hint at what a "normal" make stanza should look like in an EDA -- or similarly complicated -- flow. Here it is:

witness-file : prerequisite(s)
    /bin/rm -f witness-file log-file
    eda-tool ...args... 2>&1 | tee-grep log-file
    update-dependencies
    decide-status log-file
    /bin/touch witness-file

-include witness-file.P  # dependencies from update-dependencies

• The goal is to produce the witness-file; if it exists, it means "this steps completed successfully at the time indicated".

• eda-tool ...args... is just the normal command-line invocation of the EDA tool.

• We pipe the output(s) of the EDA tool to tee-grep, a (hypothetical) tool whose jobs are: (a) to squirrel away a copy of the log file for later perusal; and (b) to display only the log messages that are "interesting" (for the local definition of same).

• If you don't have ClearCase or equivalent, you will need some special pleading to update the dependencies related to this step; I have hypothesized an update-dependencies tool, and that it puts its output into witness-file.P, which make slurps in (last line -include).

• All of the scripts before decide-status should have completed successfully (exit status zero). (If not, something bad is wrong, e.g. out of disk space.) The question remains: can workflow steps that are relying on this one "succeeding" go ahead? That's what the decide-status script does: exit status zero will mean "go ahead" (after stamping the witness-file), non-zero exit means "go no further".

Make has a reputation for being flaky and hard to work with. If each make stanza were as bullet-proof as suggested here, most of that problem would go away.

[An earlier version of this note appeared in Verilab's internal newsletter.]