Automation and Make

Key Points

Introduction
  • Make allows us to specify what depends on what and how to update things that are out of date.

Makefiles
  • Use # for comments in Makefiles.

  • Write rules as target: dependencies.

  • Specify update actions in a tab-indented block under the rule.

  • Use .PHONY to mark targets that don’t correspond to files.

Automatic Variables
  • Use $@ to refer to the target of the current rule.

  • Use $^ to refer to the dependencies of the current rule.

  • Use $< to refer to the first dependency of the current rule.

Dependencies on Data and Code
  • Make results depend on processing scripts as well as data files.

  • Dependencies are transitive: if A depends on B and B depends on C, a change to C will indirectly trigger an update to A.

Pattern Rules
  • Use the wildcard % as a placeholder in targets and dependencies.

  • Use the special variable $* to refer to matching sets of files in actions.

Variables
  • Define variables by assigning values to names.

  • Reference variables using $(...).

Functions
  • Make is actually a small programming language with many built-in functions.

  • Use wildcard function to get lists of files matching a pattern.

  • Use patsubst function to rewrite file names.

Self-Documenting Makefiles
  • Document Makefiles by adding specially-formatted comments and a target to extract and format them.

Conclusion
  • Makefiles save time by automating repetitive work, and save thinking by documenting how to reproduce results.

Running Make

To run Make:

$ make

Make will look for a Makefile called Makefile and will build the default target, the first target in the Makefile.

To use a Makefile with a different name, use the -f flag e.g.

$ make -f build-files/analyze.mk

To build a specific target, provide it as an argument e.g.

$ make isles.dat

If the target is up-to-date, Make will print a message like:

make: `isles.dat' is up to date.

To see the actions Make will run when building a target, without running the actions, use the --dry-run flag e.g.

$ make --dry-run isles.dat

Alternatively, use the abbreviation -n.

$ make -n isles.dat

Trouble Shooting

If Make prints a message like,

Makefile:3: *** missing separator.  Stop.

then check that all the actions are indented by TAB characters and not spaces.

If Make prints a message like,

No such file or directory: 'books/%.txt'
make: *** [isles.dat] Error 1

then you may have used the Make wildcard, %, in an action in a pattern rule. Make wildcards cannot be used in actions.

Makefiles

Rules:

target : dependency1 dependency2 ...
	action1
	action2
        ...

Dependencies:

Comments:

# This is a Make comment.

Line continuation character:

ARCHIVE = isles.dat isles.png \
          abyss.dat abyss.png \
          sierra.dat sierra.png

Phony targets:

.PHONY : clean
clean :
       rm -f *.dat

Automatic variables:

Pattern rules:

%.dat : books/%.txt $(COUNT_SRC)
        $(COUNT_EXE) $< $@

Defining and using variables:

COUNT_SRC=wordcount.py
COUNT_EXE=python $(COUNT_SRC)

Suppress printing of actions:

.PHONY : variables
variables:
        @echo TXT_FILES: $(TXT_FILES)

Include the contents of a Makefile in another Makefile:

include config.mk

wildcard function:

TXT_FILES=$(wildcard books/*.txt)

patsubst (‘path substitution’) function:

DAT_FILES=$(patsubst books/%.txt, %.dat, $(TXT_FILES))

Default targets:

.DEFAULT_GOAL := all

Manuals

GNU Make Manual. Reference sections include:

Glossary

action
The steps a build manager must take to create or update a file or other object.
assignment
A request that Make stores something in a variable.
automatic variable
A variable whose value is automatically redefined for each rule. Make’s automatic variables include $@, which holds the rule’s target, $^, which holds its dependencies, and, $<, which holds the first of its dependencies, and $*, which holds the stem with which the pattern was matched. Automatic variables are typically used in pattern rules.
build file
A description of dependencies and rules for a build manager.
build manager
A program, such as Make, whose main purpose is to build or update software, documentation, web sites, data files, images, and other things.
default rule
The rule that is executed if no target is specified when a build manager is run.
default target
The target of the default rule.
dependency
A file that a target depends on. If any of a target’s dependencies are newer than the target itself, the target needs to be updated. A target’s dependencies are also called its prerequisites. If a target’s dependencies do not exist, then they need to be built first.
false dependency
This can refer to a dependency that is artificial. e.g. a false dependency is introduced if a data analysis script is added as a dependency to the data files that the script analyses.
function
A built-in Make utility that performs some operation, for example gets a list of files matching a pattern.
incremental build
The feature of a build manager by which it only rebuilds files that, either directory or indirectly, depend on a file that was changed.
macro
Used as a synonym for variable in certain versions of Make.
Make
A popular build manager, from GNU, created in 1977.
Makefile
A build file used by Make, which, by default, are named Makefile.
pattern rule
A rule that specifies a general way to build or update an entire class of files that can be managed the same way. For example, a pattern rule can specify how to compile any C file rather than a single, specific C file, or, to analyze any data file rather than a single, specific data file. Pattern rules typically make use of automatic variables and wildcards.
phony target
A target that does not correspond to a file or other object. Phony targets are usually symbolic names for sequences of actions.
prerequisite
A synonym for dependency.
reference
A request that Make substitutes the name of a variable for its value.
rule
A specification of a target’s dependencies and what actions need to be executed to build or update the target.
stem
The part of the target that was matched by the pattern rule. If the target is file.dat and the target pattern was %.dat, then the stem $* is file.
target
A thing to be created or updated, for example a file. Targets can have dependencies that must exist, and be up-to-date, before the target itself can be built or updated.
variable
A symbolic name for something in a Makefile.
wildcard
A pattern that can be specified in dependencies and targets. If Make finds a dependency] matching the pattern, then the pattern is substituted into the target. wildcards are often used in pattern rules. The Make wildcard is %.