Redo implementation in Bourne Shell
I have implemented the build system redo as designed by DJB in Bourne Shell. To understand how redo can be simpler, more flexible, more powerful and more reliable than make, read Introduction to redo and/or redo: a top-down software build system. The current version of redo is redo 2.0.4.
- the main program
- marks the current target as always needing to be rebuilt
- prints redo dependency graph in DOT format (example output, example rendering)
- adds dependencies for the current target (if a dependency changes, the target will be rebuilt)
- adds non-existence dependencies for the current target (if a non-existence dependency is created, the target will be rebuilt)
- prints a list of all target files that are out of date
- prints a list of all target files that exist (a target is a file redo can build)
- prints a list of all source files that exist (a source is a dependency that is not a target)
- detects if the current target has changed (see apenwarr's documentation)
To install redo, copy the redo executables and man pages to a directory in your $PATH. For example, if /usr/local/bin exists, is listed in your $PATH and can be written to by the current user, the following commands will install redo:
wget http://news.dieweltistgarnichtso.net/bin/archives/redo-sh.tar.gz tar -oxzf redo-sh.tar.gz -C /usr/local
Frequently Asked Questions
How do I build a file with redo?
To build a file target, you have to create a dofile target.do with shell commands that write the intended result of the build either to standard output or to its parameter $3. Then run
The default target is all:
redo without arguments executes commands from all.do.
What do the redo parameters $1, $2, $3 mean?
When redo runs a dofile, it gives it three parameters:
|$1||filename of target|
|$2||basename of target, without extension if default dofile is used|
|$3||filename of temporary output file that is renamed on build success|
How does the extension removal in parameter $2 work?
Redo removes the extension that it gets from the default dofile filename:
How do I declare dependencies with redo?
Use the redo-ifchange command in a dofile:
redo-ifchange dependency inside target.do means: If the target is built, the dependency is built if it does not exist and recorded as a dependency. On subsequent builds, if dependency does not exist or has changed since the last build, both dependency and target are rebuilt.
How does this redo implementation check dependencies?
For dependency checking, this implementation of redo checks the dependencies' ctime against the stored ctime. If the ctime differs, it checks the dependencies' md5sum against the stored md5sum. This is arguably more useful than just using ctime.
How do I declare non-existence dependencies with redo?
Use the redo-ifcreate command in a dofile:
redo-ifcreate ne_dependency inside target.do means: If target is built, the non-existing file ne_dependency is recorded as a non-existence dependency. If ne_dependency exists on subsequent builds, target is rebuilt.
What are command line options for this redo implementation?
|Short option||Long option||Effect|
|-d||--debug||print dependency checks as they happen|
|-h||--help||print usage instructions and exit|
|--version||print version information and exit|
|-x||--xtrace||print commands as they are executed (variables expanded)|
How can I use this redo implementation to build in parallel?
You can use the shell builtins
wait to execute commands asynchronously. The following line in a dofile builds targets a and b in parallel:
redo-ifchange a & redo-ifchange b & ; wait
How can I make redo build everything in a different directory?
You can use union mounts to combine multiple directories. For example, on Linux with unionfs-fuse, the command unionfs -o cow 'output=RW:source=RO' /tmp/build combines the output and source directories so that /tmp/build contains all files from the source directory, but any newly created file will show up only in the output directory.
Why can a target be built twice during a run?
Some build processes require several builds of the same target. Naïve implementors of build systems disregard this possibility and do not check the validity of a target again if it was built during a run.
- A target that has a dofile that invokes redo-always will be rebuilt several more times, if several targets invoke redo-ifchange for it.
- A dofile can use a loop involving inotifywait to wait for changes to dependencies and immediately rebuild targets if a dependency changes.
- TeX documents can contain internal references. Page and equation numbers are written to an external file needed for a second build.
- TemplateHaskell with profiling needs an executable compiled twice.
- The technique detailed in David A. Wheeler's Fully Countering Trusting Trust through Diverse Double-Compiling requires two builds of a target.
Why does redo-ifchange not add a dependency if the invoked dofile does not generate a target?
redo rebuilds target files when source files have changed. Files that do not exist can never change – therefore, a non-existent file is always up to date.
How does this implementation of redo compare with other implementations?
The following table compares redo implementations. Differences from the DJB redo design and incompatibilities with existing dofiles are emphasized.
|Implementation||Programming Language||Modification Time Check||File Content Hash Check||Parallel Build||Notes, Bugs, and Incompatibilities|
|Nils Dagsson Moskopp's redo||Bourne Shell||✓ Yes||✓ Yes||✕ No||
|Alan Grosskurth's redo||Bourne Shell||✕ No||✓ Yes||✕ No||
|Avery Pennarun's do||Bourne Shell||✕ No||✕ No||✕ No||
|Avery Pennarun's redo||Python||✓ Yes||✕ No||✓ Yes||
|Christian Neukirchen's redo||C||✓ Yes||✓ Yes||✓ Yes||
|Jonathan de Boyne Pollard's redo||Bourne Again Shell||✓ Yes||✓ Yes||✕ No||
|Gyepi Sam's redux||Go||✓ Yes||✓ Yes||✕ No||
|jekor's redo||Haskell||✕ No||✓ Yes||✕ No||
|Jonathan de Boyne Pollard's redo||C++||✓ Yes||✓ Yes||✓ Yes||
|Shanti Bouchez-Mongardé's redo||Python||✓ Yes||✕ No||✓ Yes||
|Tharre's redo||C||✕ No||✓ Yes||✕ No||