Do your job with make!

ArticleCategory: []

Software Development

AuthorImage:[Here comes a small picture of you]

[Wilbert Berendsen]

TranslationInfo:[Info concerning writer(s) and translator(s)]

original in nl Wilbert Berendsen

nl to en Philip de Groot

AboutTheAuthor:[a short biography about the author]

Wilbert Berendsen is a professional musician and a enthousiastic Linux user. Once, he intensively hacked assembly for the Z80. Nowadays, he's using Linux for all his production work. Just for fun, he writes introductory articles and he maintains a small website on http://www.xs4all.nl/~wbsoft/. Viva open source!

Abstract:[a small summary/description of this article]

This article over make shows how make works and that it can be used for more things than just software development.

ArticleIllustration:[illustration]

[Illustration]

ArticleBody:[The real article: put the text and html-codes here]

Introduction

Almost anyone who uses Linux has applied the make program sometimes. It does its job if a program or kernel is build from the source code, if a package is installed, and so on. 'Make' is an important tool for software development. However, make has much more possibilities!

In this document, we will see that make can be a powerful tool for daily jobs, such as writing articles, books or composing a nice website. During this introduction, many other 'unix tricks' will be handled. At the end of this story, some more tips will be presented on using make. Please note: we are talking about Linux, but in principle it is possible to use make on every operating system.

Example: building a website

We want to build a website that is maintained by different people. Jan takes care of two pages and maintaines them. Piet takes care of the layout.

We need a simple system to separate the layout and content from each other. A powerful solution is: reading the content from a database, everytime the page is requested. For example, PHP and Microsoft Active Server Pages work this way. However, we only have the possibility to store plain HTML (HyperText Markup Language). Furthermore, the content does not change that often to efficiently maintain a database.

Using some simple commands, a website will be constructed.

For example, Piet puts the header of the site in header.html and the footer of the site in footer.html. header.html might look like this:

<html><!-- the header -->
<head>
<title>Piet and Jan productions</title>
</head>
<body bgcolor="white">
<table border="0" width="100%"><tr>
<td bgcolor="#c040ff" valign="top">
This is our website<br>
Some rubbish is written down here.<br>
We are very interactive<br>
so this is our telephone number:<br>
<b>0123-456789</b>
</td><td valign="top">
<!-- Put the contents here -->
And this is footer.html:
<!-- the footer -->
</td></tr></table>
</body></html>
For example, the unix commands to build the final page from Jans' index.html are:
cat header.html  /home/jan/Docs/website/index.html
echo -n '<hr>Last modification: '
date '+%A %e %B'
cat footer.html
Please refer to the manual pages of these commands. The final file, as a result of the above commands, is piped to the standard output, which is grabbed to a file:
{
  cat header.html  /home/jan/Docs/website/index.html
  echo -n '<hr>Last modification: '
  date '+%A %e %B'
  cat footer.html
} > /home/piet/public_html/index.html
This procedure can be repeated with the other file, offer.html. In fact, we created a small script that enables the construction of our website.

However, executing this command by hand is not feasible. We can create a shell-script that is executed everytime as Jan has updated his index. However, if Piet decides to change the header or footer, this script should also be executed! On the other hand, if Jan has changed nothing on a day, the script should not be executed. We are using Linux, so we want to use a smart solution (read: automatically)!

At this point, make shows up.

First meeting with make

The info-manual of GNU make is a fantastic document. However, from the beginning it is focussing on a programming environment. For this reason, I am trying to indicate the functions of make in a broader sense:
    make determines whether a set of commands should be executed, 
    based on the time-stamp of the target-file and the time-stamps 
    of the source files.
In other words: if one of the source files, needed to create a target-file, is newer than the target-file, a set of commands will be executed. The purpose of these commands is to update the target-file.

The target-file is the 'target' and the source-files are `prerequisites' (first demands). The commands are executed if one of the `prerequisites' is newer than the target-file (or if the target does not exist). If all prerequisites are older than or equally old as the target, then the commands are not executed and the target is considered as being up-to-date.

In the current working directory, a file should be created that has the name Makefile. This file contains the information that is needed by make to do its job properly. Once we have the Makefile, the only thing that we need to do is: typing 'make' and the commands, needed for creating a new target-file, are executed automatically.

Make is called with the command

make target1 target2 ....

target is optional (if target is left away, the first target in the Makefile is being used). Make is always looking in the current directory for the Makefile. It is possible to supply more than one target.

Makefile syntaxis

The Makefile can be created with an editor and looks like:
# This is an example of a Makefile.
# Comments can be put after a hash (#).

target: prerequisites
	command
        
target: prerequisites
	commando 
        
# and so on and so on.
We start with a target, followed by a colon (:) and the needed prerequisites. In the presence of many prerequisites, it is possible to end the line with a backslash (\) and to continue on the next line.

On the following line(s), one or more commands are presented. Every line is considered as a standalone command. If you want to use multiple lines for one command, you should put backslashes (\\) on the end of the line. Make will connect the commands as if they were written on one line. In this situation, we have to separate the commands with a semicolon (;) in order to prevent mistakes by the executing shell.

Note: The commands should be indented with a TAB, not with 8 spaces!

Make reads the Makefile and determines for every target (starting with the first one) whether the commands should be executed. Every target, together with the prerequisites and rules, is denoted a 'rule'.

If make is executed without arguments, only the first target will be executed.

A Makefile for our example

For our example, the Makefile should look like this:

# This Makefile builds Piets' and Jans' website, the potato-eaters.

all: /home/piet/public_html/index.html /home/piet/public_html/offer.html

/home/piet/public_html/index.html:  header.html footer.html \
                                    /home/jan/Docs/website/index.html
	{ \
          cat header.html  /home/jan/Docs/website/index.html ;\
          echo -n '<hr>Last modification: '                 ;\
          date '+%A %e %B'                                   ;\
          cat footer.html                                    ;\
        } > /home/piet/public_html/index.html

/home/piet/public_html/offer.html:  header.html footer.html \
                                    /home/jan/Docs/website/offer.html
	{ \
          cat header.html  /home/jan/Docs/website/index.html ;\
          echo -n '<hr>Last modification: '                 ;\
          date '+%A %e %B'                                   ;\
          cat footer.html                                    ;\
        } > /home/piet/public_html/offer.html

# the end

Now, we have three targets, 'all' and the files index.html and offer.html from the website. The only function of the target 'all' is to have both others as prerequisites. These are both tested. Because 'all' itself is no filename, the target 'all' will always be executed. (Later, we will introduce a more elegant way of defining targets that are no file).

If the header and footer were modified, both pages will be updated. If Jan modifies one of his pages, only the modified page will be updated. Executing the 'make' command does the job!

Of course, the Makefile has a drawback: it is not easy to oversee. Fortunately, many ways are available of making things more simple!

Streamlining the Makefile

Variables

Thanks to variables, a Makefile can be simplified a lot. Variables are defined as follows:
variable = value
We refer to a variable with the expression $(variabele). If we incorporate this into the Makefile, it looks quite a lot better:
# This Makefile builds Piets' and Jans' website, the potato-eaters.

# Directory where the website is stored:
TARGETDIR = /home/piet/public_html

# Jans' directory:
JANSDIR = /home/jan/Docs/website

# Files needed for the layout:
LAYOUT = header.html footer.html

all: $(TARGETDIR)/index.html $(TARGETDIR)/offer.html

$(TARGETDIR)/index.html:  $(LAYOUT) $(JANSDIR)/index.html
	{ \
          cat header.html $(JANSDIR)/index.html     ;\
          echo -n '<hr>Last modification: '        ;\
          date '+%A %e %B'                          ;\
          cat footer.html                           ;\
        } > $(TARGETDIR)/index.html

$(TARGETDIR)/offer.html:  $(LAYOUT) $(JANSDIR)/offer.html
	{ \
          cat header.html  $(JANSDIR)/index.html    ;\
          echo -n '<hr>Last modification: '        ;\
          date '+%A %e %B'                          ;\
          cat footer.html                           ;\
        } > $(TARGETDIR)/offer.html

# the end
It is a good habit to use capital letters for the variables. Now, it is a lot easier to change, e.g., the target-directory.

If you want to, it is possible to define another method for each document that you want to put in the correct layout. What should we do if many documents must be put in the same layout? The Makefile would become very large, while many repetitions are present. This can also be simplified!

Pattern Rules

`Pattern Rules' enables us to use the same set of commands on all kinds of different targets.

If pattern rules are used, the syntaxis of a line changes; an extra pattern field is added:

Multiple targets: pattern : prerequisite prerequisite ...
	command
The pattern is an expression that should be applicable to all targets. A percent-sign is used to incorporate variable-parts of a target-name.

An example:

/home/bla/target1.html /home/bla/target2.html: /home/bla/% : %
	commands
If make reads this, the line is expanded to 2 lines. Here, the pattern determines which part of the target-name is incorporated in the percent-sign.

The percent-sign in the prerequisites-field represents the part that is copied by this percent-sign.

Make expands the above as:

/home/bla/target1.html:	target1.html
	commands
        
/home/bla/target2.html: target2.html
	commands
The percent-sign in the pattern `/home/bla/%' gets with target `/home/bla/target1.html' the value `target1.html', thus expanding the prerequisite `%' to `target1.html'.

For our website, the following rule will be incorporated:

$(TARGETDIR)/index.html $(TARGETDIR)/offer.html: \
                   $(TARGETDIR)/% : $(JANSDIR)/% \
                                         $(LAYOUT)
Now we have one problem left: how to use these variables in the commands? The commands were a little bit different for both targets?

Automatic variables

Fortunately, make defines some variables for itself. Some of these variables are called automated variables. These variables contain, during the execution of the commands (better: just prior to executing these commands), the value of the target and/or prerequisite.

The special variable $< is used to indicate the first prerequisite and the variable $@ expands always to the current target.

Using these variables, it is possible to generalise the complete rule as follows:

$(TARGETDIR)/index.html $(TARGETDIR)/offer.html: $(TARGETDIR)/% : \
                                                     $(JANSDIR)/% \
                                                          $(LAYOUT)
	{ \
          cat header.html  $<                       ;\
          echo -n '<hr>Last modification: '        ;\
          date '+%A %e %B'                          ;\
          cat footer.html                           ;\
        } > $@
Voil�! This single line functions now for both files!

For completeness, the full Makefile is presented, including some more optimalisations:

# This Makefile builds Piets' and Jans' website, the potato-eaters.

# Directory where the website is published:
TARGETDIR = /home/piet/public_html

# Jans' directory:
JANSDIR = /home/jan/Docs/website

# Files needed for the layout:
LAYOUT = header.html footer.html

# These are the webpages:
DOCS = $(TARGETDIR)/index.html $(TARGETDIR)/offer.html


# Please change nothing below this line;-)
# -------------------------------------------------------------

all: $(DOCS)

$(DOCS): $(TARGETDIR)/% : $(JANSDIR)/% $(LAYOUT)
	{ \
          cat header.html  $<                       ;\
          echo -n '<hr>Last modification: '         ;\
          date '+%A %e %B'                          ;\
          cat footer.html                           ;\
        } > $@

# the end
This starts to look like how it should be. If more documents are added, it is quite easy to incorporate them in the Makefile, using the DOCS variable, without too much typing.

In the end, the person that maintains the Makefile should easily see how it works, without puzzling on the way how it functions!

Last small optimalisations

We would prefer to mention the documents in DOCS, without including the whole directory. This can be done as follows (we change DOCS in the beginning of the makefile in TEXTS):
TEXTS = index.html  offer.html  yetanotherfile.html

# Please change nothing below this line;-)
# -------------------------------------------------------------
DOCS =  $(addprefix $(TARGETDIR)/,$(TEXTS))

all: $(DOCS)

# and so on
What we see here is a special make function: instead of a variable-name, it is possible to use a complete expression between brackets. This way, it is possible to modify texts in numerous ways.

The special command $(addprefix prefix,list) adds to each element on the list a prefix. In the example, this is the contents of the TARGETDIR variable plus a slash (/).

The listed items are separated with spaces. For this reason, it is not a good idea to process filenames that have spaces with the make command.

To conclude: at the beginning, we already mentioned that the target 'all' won't create a file with the name 'all' (this line does not contain any commands) and as a result, this target is always executed. But how to handle if <accidentally> a file does exist with this name, newer than the other files ...?

There is an easy way to tell make that a particular target always should be executed and that this target does not refer to a file on the hard disk. To achieve this, the target is marked as 'phony' (not real). This is done as follows:

.PHONY: all
Now, the whole Makefile looks like this:
# This Makefile builds Piets' and Jans' website, the potato-eaters.

# Directory where the website is published:
TARGETDIR = /home/piet/public_html

# Jans' directory:
JANSDIR = /home/jan/Docs/website

# Files needed for the layout:
LAYOUT = header.html footer.html

# These are the names of the webpages:
TEXTS = index.html  offer.html  yetanotherfile.html

# Please change nothing below this line;-)
# ------------------------------------------------------
DOCS =  $(addprefix $(TARGETDIR)/,$(TEXTS))
.PHONY: all

all: $(DOCS)

$(DOCS): $(TARGETDIR)/% : $(JANSDIR)/% $(LAYOUT)
	{ \
          cat header.html  $<                       ;\
          echo -n '<hr>Last modification: '         ;\
          date '+%A %e %B'                          ;\
          cat footer.html                           ;\
        } > $@

# the end
Store this file and forget it! From now on, it is possible to maintain your webpages, perhaps by using your crontab, and to properly separate the layout from the content!

Final remarks

Of course, it is possible to modify this example for other situations.

As an example, the simple way in which the document is generated is not error-free: if Jan ends his articles accidentally with </body></html>, most browsers won't display the footer that Piet has made. If we apply grep, perl or tcl, it is possible to put some titles from Jans documents in an clever way in the header of the site.

Of course, Jan can simply write flat text and use the sed command to change all white-lines (carriage returns) in <P>:

sed -e 's/^\s*$/<p>/g'
Furthermore, Jan can write his texts in LyX and use the a program, such as lyx2html, to convert it to HTML. Tremendous possibilities are available!

Another template-construction is also possible.

We haven't considered how possible pictures are transported (scaled, converted, or compressed) to the web directory. It is also possible to automate this process!

In this example, Piet should have read permission in Jans' website directory. The interesting thing of separating these tasks is that they can be applied in very large organisations. It is even possible for Piet to login at the other side of the world, or to mount his homedir over NFS. The examples can also be used for labour that is performed by one user.

Hopefully, it has become clear how the Makefile principles functions and how easy your daily work can become at the moment that you have written a good Makefile!

Tips

More information

More information on how make works and all other possibilities can be found in the `GNU Make Manual'. You can read this manual on your Linux system with the following command:
info make
Of course, it is possible to read the GNU Make Manual with the GNOME and KDE help browsers or the handy tkinfo program.

Links to more information about make:

Have Fun!