blob: cd672cf6183328625e005c553781189f1e717fe3 [file] [log] [blame]
Simon Glassfc3fe1c2013-04-03 11:07:16 +00001# Copyright (c) 2013 The Chromium OS Authors.
2#
Wolfgang Denk1a459662013-07-08 09:37:19 +02003# SPDX-License-Identifier: GPL-2.0+
Simon Glassfc3fe1c2013-04-03 11:07:16 +00004#
5
Simon Glass6eede342014-08-09 15:32:58 -06006(Please read 'How to change from MAKEALL' if you are used to that tool)
7
Simon Glassfc3fe1c2013-04-03 11:07:16 +00008What is this?
9=============
10
11This tool handles building U-Boot to check that you have not broken it
12with your patch series. It can build each individual commit and report
13which boards fail on which commits, and which errors come up. It aims
14to make full use of multi-processor machines.
15
16A key feature of buildman is its output summary, which allows warnings,
17errors or image size increases in a particular commit or board to be
18quickly identified and the offending commit pinpointed. This can be a big
19help for anyone working with >10 patches at a time.
20
21
22Caveats
23=======
24
Simon Glassfc3fe1c2013-04-03 11:07:16 +000025Buildman can be stopped and restarted, in which case it will continue
26where it left off. This should happen cleanly and without side-effects.
27If not, it is a bug, for which a patch would be welcome.
28
29Buildman gets so tied up in its work that it can ignore the outside world.
30You may need to press Ctrl-C several times to quit it. Also it will print
Simon Glass8ea42102016-07-27 20:32:58 -060031out various exceptions when stopped. You may have to kill it since the
32Ctrl-C handling is somewhat broken.
Simon Glassfc3fe1c2013-04-03 11:07:16 +000033
34
35Theory of Operation
36===================
37
38(please read this section in full twice or you will be perpetually confused)
39
40Buildman is a builder. It is not make, although it runs make. It does not
41produce any useful output on the terminal while building, except for
Simon Glasse5a0e5d2014-08-09 15:33:03 -060042progress information (except with -v, see below). All the output (errors,
Dirk Behme3e1ded12014-12-23 07:41:26 +010043warnings and binaries if you ask for them) is stored in output
Simon Glasse5a0e5d2014-08-09 15:33:03 -060044directories, which you can look at while the build is progressing, or when
45it is finished.
Simon Glassfc3fe1c2013-04-03 11:07:16 +000046
Simon Glass8ea42102016-07-27 20:32:58 -060047Buildman is designed to build entire git branches, i.e. muliple commits. It
48can be run repeatedly on the same branch. In this case it will automatically
49rebuild commits which have changed (and remove its old results for that
50commit). It is possible to build a branch for one board, then later build it
51for another board. If you want buildman to re-build a commit it has already
52built (e.g. because of a toolchain update), use the -f flag.
53
Simon Glassfc3fe1c2013-04-03 11:07:16 +000054Buildman produces a concise summary of which boards succeeded and failed.
55It shows which commit introduced which board failure using a simple
56red/green colour coding. Full error information can be requested, in which
57case it is de-duped and displayed against the commit that introduced the
58error. An example workflow is below.
59
60Buildman stores image size information and can report changes in image size
61from commit to commit. An example of this is below.
62
63Buildman starts multiple threads, and each thread builds for one board at
64a time. A thread starts at the first commit, configures the source for your
65board and builds it. Then it checks out the next commit and does an
66incremental build. Eventually the thread reaches the last commit and stops.
67If errors or warnings are found along the way, the thread will reconfigure
68after every commit, and your build will be very slow. This is because a
69file that produces just a warning would not normally be rebuilt in an
70incremental build.
71
72Buildman works in an entirely separate place from your U-Boot repository.
73It creates a separate working directory for each thread, and puts the
74output files in the working directory, organised by commit name and board
75name, in a two-level hierarchy.
76
77Buildman is invoked in your U-Boot directory, the one with the .git
78directory. It clones this repository into a copy for each thread, and the
79threads do not affect the state of your git repository. Any checkouts done
80by the thread affect only the working directory for that thread.
81
Simon Glasscec83c32014-08-09 15:32:57 -060082Buildman automatically selects the correct tool chain for each board. You
83must supply suitable tool chains, but buildman takes care of selecting the
Simon Glassfc3fe1c2013-04-03 11:07:16 +000084right one.
85
Simon Glasse5a0e5d2014-08-09 15:33:03 -060086Buildman generally builds a branch (with the -b flag), and in this case
87builds the upstream commit as well, for comparison. It cannot build
88individual commits at present, unless (maybe) you point it at an empty
89branch. Put all your commits in a branch, set the branch's upstream to a
90valid value, and all will be well. Otherwise buildman will perform random
91actions. Use -n to check what the random actions might be.
92
Simon Glass1d8104f2014-10-16 01:05:56 -060093If you just want to build the current source tree, leave off the -b flag
94and add -e. This will display results and errors as they happen. You can
95still look at them later using -se. Note that buildman will assume that the
96source has changed, and will build all specified boards in this case.
Simon Glassfc3fe1c2013-04-03 11:07:16 +000097
98Buildman is optimised for building many commits at once, for many boards.
99On multi-core machines, Buildman is fast because it uses most of the
100available CPU power. When it gets to the end, or if you are building just
101a few commits or boards, it will be pretty slow. As a tip, if you don't
102plan to use your machine for anything else, you can use -T to increase the
103number of threads beyond the default.
104
Stephen Warren8426d8b2013-10-10 10:00:20 -0600105Buildman lets you build all boards, or a subset. Specify the subset by passing
106command-line arguments that list the desired board name, architecture name,
107SOC name, or anything else in the boards.cfg file. Multiple arguments are
108allowed. Each argument will be interpreted as a regular expression, so
109behaviour is a superset of exact or substring matching. Examples are:
110
111* 'tegra20' All boards with a Tegra20 SoC
112* 'tegra' All boards with any Tegra Soc (Tegra20, Tegra30, Tegra114...)
113* '^tegra[23]0$' All boards with either Tegra20 or Tegra30 SoC
114* 'powerpc' All PowerPC boards
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000115
Simon Glass6131bea2014-08-09 15:33:08 -0600116While the default is to OR the terms together, you can also make use of
117the '&' operator to limit the selection:
118
119* 'freescale & arm sandbox' All Freescale boards with ARM architecture,
120 plus sandbox
121
Simon Glass3cf4ae62014-08-28 09:43:41 -0600122You can also use -x to specifically exclude some boards. For example:
123
124 buildmand arm -x nvidia,freescale,.*ball$
125
126means to build all arm boards except nvidia, freescale and anything ending
127with 'ball'.
128
Dirk Behme3e1ded12014-12-23 07:41:26 +0100129It is convenient to use the -n option to see what will be built based on
Simon Glass6131bea2014-08-09 15:33:08 -0600130the subset given.
131
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000132Buildman does not store intermediate object files. It optionally copies
133the binary output into a directory when a build is successful. Size
134information is always recorded. It needs a fair bit of disk space to work,
135typically 250MB per thread.
136
137
138Setting up
139==========
140
1411. Get the U-Boot source. You probably already have it, but if not these
142steps should get you started with a repo and some commits for testing.
143
144$ cd /path/to/u-boot
145$ git clone git://git.denx.de/u-boot.git .
146$ git checkout -b my-branch origin/master
147$ # Add some commits to the branch, reading for testing
148
Simon Glass62005342014-12-01 17:34:02 -07001492. Create ~/.buildman to tell buildman where to find tool chains (see 'The
150.buildman file' later for details). As an example:
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000151
152# Buildman settings file
153
154[toolchain]
155root: /
156rest: /toolchains/*
157eldk: /opt/eldk-4.2
Simon Glasse9569472014-08-09 15:33:07 -0600158arm: /opt/linaro/gcc-linaro-arm-linux-gnueabihf-4.8-2013.08_linux
159aarch64: /opt/linaro/gcc-linaro-aarch64-none-elf-4.8-2013.10_linux
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000160
161[toolchain-alias]
162x86: i386
163blackfin: bfin
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000164nds32: nds32le
Bin Menge8aebc42016-02-21 21:18:02 -0800165openrisc: or1k
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000166
167
168This selects the available toolchain paths. Add the base directory for
169each of your toolchains here. Buildman will search inside these directories
170and also in any '/usr' and '/usr/bin' subdirectories.
171
172Make sure the tags (here root: rest: and eldk:) are unique.
173
174The toolchain-alias section indicates that the i386 toolchain should be used
175to build x86 commits.
176
Simon Glass17bce662016-03-12 18:50:32 -0700177Note that you can also specific exactly toolchain prefixes if you like:
178
179[toolchain-prefix]
180arm: /opt/arm-eabi-4.6/bin/arm-eabi-
181
182or even:
183
184[toolchain-prefix]
185arm: /opt/arm-eabi-4.6/bin/arm-eabi-gcc
186
187This tells buildman that you want to use this exact toolchain for the arm
188architecture. This will override any toolchains found by searching using the
189[toolchain] settings.
190
191Since the toolchain prefix is an explicit request, buildman will report an
192error if a toolchain is not found with that prefix. The current PATH will be
193searched, so it is possible to use:
194
195[toolchain-prefix]
196arm: arm-none-eabi-
197
198and buildman will find arm-none-eabi-gcc in /usr/bin if you have it installed.
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000199
Simon Glass34699692014-12-01 17:34:01 -07002003. Make sure you have the require Python pre-requisites
201
Simon Glass827e37b2014-12-01 17:34:06 -0700202Buildman uses multiprocessing, Queue, shutil, StringIO, ConfigParser and
203urllib2. These should normally be available, but if you get an error like
204this then you will need to obtain those modules:
Simon Glass34699692014-12-01 17:34:01 -0700205
206 ImportError: No module named multiprocessing
207
208
2094. Check the available toolchains
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000210
211Run this check to make sure that you have a toolchain for every architecture.
212
213$ ./tools/buildman/buildman --list-tool-chains
214Scanning for tool chains
Simon Glass17bce662016-03-12 18:50:32 -0700215 - scanning prefix '/opt/gcc-4.6.3-nolibc/x86_64-linux/bin/x86_64-linux-'
216Tool chain test: OK, arch='x86', priority 1
217 - scanning prefix '/opt/arm-eabi-4.6/bin/arm-eabi-'
218Tool chain test: OK, arch='arm', priority 1
219 - scanning path '/toolchains/gcc-4.9.0-nolibc/i386-linux'
220 - looking in '/toolchains/gcc-4.9.0-nolibc/i386-linux/.'
221 - looking in '/toolchains/gcc-4.9.0-nolibc/i386-linux/bin'
222 - found '/toolchains/gcc-4.9.0-nolibc/i386-linux/bin/i386-linux-gcc'
223 - looking in '/toolchains/gcc-4.9.0-nolibc/i386-linux/usr/bin'
224Tool chain test: OK, arch='i386', priority 4
225 - scanning path '/toolchains/gcc-4.9.0-nolibc/aarch64-linux'
226 - looking in '/toolchains/gcc-4.9.0-nolibc/aarch64-linux/.'
227 - looking in '/toolchains/gcc-4.9.0-nolibc/aarch64-linux/bin'
228 - found '/toolchains/gcc-4.9.0-nolibc/aarch64-linux/bin/aarch64-linux-gcc'
229 - looking in '/toolchains/gcc-4.9.0-nolibc/aarch64-linux/usr/bin'
230Tool chain test: OK, arch='aarch64', priority 4
231 - scanning path '/toolchains/gcc-4.9.0-nolibc/microblaze-linux'
232 - looking in '/toolchains/gcc-4.9.0-nolibc/microblaze-linux/.'
233 - looking in '/toolchains/gcc-4.9.0-nolibc/microblaze-linux/bin'
234 - found '/toolchains/gcc-4.9.0-nolibc/microblaze-linux/bin/microblaze-linux-gcc'
235 - looking in '/toolchains/gcc-4.9.0-nolibc/microblaze-linux/usr/bin'
236Tool chain test: OK, arch='microblaze', priority 4
237 - scanning path '/toolchains/gcc-4.9.0-nolibc/mips64-linux'
238 - looking in '/toolchains/gcc-4.9.0-nolibc/mips64-linux/.'
239 - looking in '/toolchains/gcc-4.9.0-nolibc/mips64-linux/bin'
240 - found '/toolchains/gcc-4.9.0-nolibc/mips64-linux/bin/mips64-linux-gcc'
241 - looking in '/toolchains/gcc-4.9.0-nolibc/mips64-linux/usr/bin'
242Tool chain test: OK, arch='mips64', priority 4
243 - scanning path '/toolchains/gcc-4.9.0-nolibc/sparc64-linux'
244 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc64-linux/.'
245 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc64-linux/bin'
246 - found '/toolchains/gcc-4.9.0-nolibc/sparc64-linux/bin/sparc64-linux-gcc'
247 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc64-linux/usr/bin'
248Tool chain test: OK, arch='sparc64', priority 4
249 - scanning path '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi'
250 - looking in '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi/.'
251 - looking in '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi/bin'
252 - found '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-gcc'
253 - looking in '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi/usr/bin'
254Tool chain test: OK, arch='arm', priority 3
255Toolchain '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-gcc' at priority 3 will be ignored because another toolchain for arch 'arm' has priority 1
256 - scanning path '/toolchains/gcc-4.9.0-nolibc/sparc-linux'
257 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc-linux/.'
258 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc-linux/bin'
259 - found '/toolchains/gcc-4.9.0-nolibc/sparc-linux/bin/sparc-linux-gcc'
260 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc-linux/usr/bin'
261Tool chain test: OK, arch='sparc', priority 4
262 - scanning path '/toolchains/gcc-4.9.0-nolibc/mips-linux'
263 - looking in '/toolchains/gcc-4.9.0-nolibc/mips-linux/.'
264 - looking in '/toolchains/gcc-4.9.0-nolibc/mips-linux/bin'
265 - found '/toolchains/gcc-4.9.0-nolibc/mips-linux/bin/mips-linux-gcc'
266 - looking in '/toolchains/gcc-4.9.0-nolibc/mips-linux/usr/bin'
267Tool chain test: OK, arch='mips', priority 4
268 - scanning path '/toolchains/gcc-4.9.0-nolibc/x86_64-linux'
269 - looking in '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/.'
270 - looking in '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/bin'
271 - found '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/bin/x86_64-linux-gcc'
272 - found '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/bin/x86_64-linux-x86_64-linux-gcc'
273 - looking in '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/usr/bin'
274Tool chain test: OK, arch='x86_64', priority 4
275Tool chain test: OK, arch='x86_64', priority 4
276Toolchain '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/bin/x86_64-linux-x86_64-linux-gcc' at priority 4 will be ignored because another toolchain for arch 'x86_64' has priority 4
277 - scanning path '/toolchains/gcc-4.9.0-nolibc/m68k-linux'
278 - looking in '/toolchains/gcc-4.9.0-nolibc/m68k-linux/.'
279 - looking in '/toolchains/gcc-4.9.0-nolibc/m68k-linux/bin'
280 - found '/toolchains/gcc-4.9.0-nolibc/m68k-linux/bin/m68k-linux-gcc'
281 - looking in '/toolchains/gcc-4.9.0-nolibc/m68k-linux/usr/bin'
282Tool chain test: OK, arch='m68k', priority 4
283 - scanning path '/toolchains/gcc-4.9.0-nolibc/powerpc-linux'
284 - looking in '/toolchains/gcc-4.9.0-nolibc/powerpc-linux/.'
285 - looking in '/toolchains/gcc-4.9.0-nolibc/powerpc-linux/bin'
286 - found '/toolchains/gcc-4.9.0-nolibc/powerpc-linux/bin/powerpc-linux-gcc'
287 - looking in '/toolchains/gcc-4.9.0-nolibc/powerpc-linux/usr/bin'
288Tool chain test: OK, arch='powerpc', priority 4
289 - scanning path '/toolchains/gcc-4.6.3-nolibc/bfin-uclinux'
290 - looking in '/toolchains/gcc-4.6.3-nolibc/bfin-uclinux/.'
291 - looking in '/toolchains/gcc-4.6.3-nolibc/bfin-uclinux/bin'
292 - found '/toolchains/gcc-4.6.3-nolibc/bfin-uclinux/bin/bfin-uclinux-gcc'
293 - looking in '/toolchains/gcc-4.6.3-nolibc/bfin-uclinux/usr/bin'
294Tool chain test: OK, arch='bfin', priority 6
295 - scanning path '/toolchains/gcc-4.6.3-nolibc/sparc-linux'
296 - looking in '/toolchains/gcc-4.6.3-nolibc/sparc-linux/.'
297 - looking in '/toolchains/gcc-4.6.3-nolibc/sparc-linux/bin'
298 - found '/toolchains/gcc-4.6.3-nolibc/sparc-linux/bin/sparc-linux-gcc'
299 - looking in '/toolchains/gcc-4.6.3-nolibc/sparc-linux/usr/bin'
300Tool chain test: OK, arch='sparc', priority 4
301Toolchain '/toolchains/gcc-4.6.3-nolibc/sparc-linux/bin/sparc-linux-gcc' at priority 4 will be ignored because another toolchain for arch 'sparc' has priority 4
302 - scanning path '/toolchains/gcc-4.6.3-nolibc/mips-linux'
303 - looking in '/toolchains/gcc-4.6.3-nolibc/mips-linux/.'
304 - looking in '/toolchains/gcc-4.6.3-nolibc/mips-linux/bin'
305 - found '/toolchains/gcc-4.6.3-nolibc/mips-linux/bin/mips-linux-gcc'
306 - looking in '/toolchains/gcc-4.6.3-nolibc/mips-linux/usr/bin'
307Tool chain test: OK, arch='mips', priority 4
308Toolchain '/toolchains/gcc-4.6.3-nolibc/mips-linux/bin/mips-linux-gcc' at priority 4 will be ignored because another toolchain for arch 'mips' has priority 4
309 - scanning path '/toolchains/gcc-4.6.3-nolibc/m68k-linux'
310 - looking in '/toolchains/gcc-4.6.3-nolibc/m68k-linux/.'
311 - looking in '/toolchains/gcc-4.6.3-nolibc/m68k-linux/bin'
312 - found '/toolchains/gcc-4.6.3-nolibc/m68k-linux/bin/m68k-linux-gcc'
313 - looking in '/toolchains/gcc-4.6.3-nolibc/m68k-linux/usr/bin'
314Tool chain test: OK, arch='m68k', priority 4
315Toolchain '/toolchains/gcc-4.6.3-nolibc/m68k-linux/bin/m68k-linux-gcc' at priority 4 will be ignored because another toolchain for arch 'm68k' has priority 4
316 - scanning path '/toolchains/gcc-4.6.3-nolibc/powerpc-linux'
317 - looking in '/toolchains/gcc-4.6.3-nolibc/powerpc-linux/.'
318 - looking in '/toolchains/gcc-4.6.3-nolibc/powerpc-linux/bin'
319 - found '/toolchains/gcc-4.6.3-nolibc/powerpc-linux/bin/powerpc-linux-gcc'
320 - looking in '/toolchains/gcc-4.6.3-nolibc/powerpc-linux/usr/bin'
321Tool chain test: OK, arch='powerpc', priority 4
322Tool chain test: OK, arch='or32', priority 4
323 - scanning path '/toolchains/gcc-4.2.4-nolibc/avr32-linux'
324 - looking in '/toolchains/gcc-4.2.4-nolibc/avr32-linux/.'
325 - looking in '/toolchains/gcc-4.2.4-nolibc/avr32-linux/bin'
326 - found '/toolchains/gcc-4.2.4-nolibc/avr32-linux/bin/avr32-linux-gcc'
327 - looking in '/toolchains/gcc-4.2.4-nolibc/avr32-linux/usr/bin'
328Tool chain test: OK, arch='avr32', priority 4
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000329 - scanning path '/'
330 - looking in '/.'
331 - looking in '/bin'
332 - looking in '/usr/bin'
Simon Glass17bce662016-03-12 18:50:32 -0700333 - found '/usr/bin/i586-mingw32msvc-gcc'
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000334 - found '/usr/bin/c89-gcc'
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000335 - found '/usr/bin/x86_64-linux-gnu-gcc'
Simon Glass17bce662016-03-12 18:50:32 -0700336 - found '/usr/bin/gcc'
337 - found '/usr/bin/c99-gcc'
338 - found '/usr/bin/arm-linux-gnueabi-gcc'
339 - found '/usr/bin/aarch64-linux-gnu-gcc'
340 - found '/usr/bin/winegcc'
341 - found '/usr/bin/arm-linux-gnueabihf-gcc'
342Tool chain test: OK, arch='i586', priority 11
343Tool chain test: OK, arch='c89', priority 11
344Tool chain test: OK, arch='x86_64', priority 4
345Toolchain '/usr/bin/x86_64-linux-gnu-gcc' at priority 4 will be ignored because another toolchain for arch 'x86_64' has priority 4
346Tool chain test: OK, arch='sandbox', priority 11
347Tool chain test: OK, arch='c99', priority 11
348Tool chain test: OK, arch='arm', priority 4
349Toolchain '/usr/bin/arm-linux-gnueabi-gcc' at priority 4 will be ignored because another toolchain for arch 'arm' has priority 1
350Tool chain test: OK, arch='aarch64', priority 4
351Toolchain '/usr/bin/aarch64-linux-gnu-gcc' at priority 4 will be ignored because another toolchain for arch 'aarch64' has priority 4
352Tool chain test: OK, arch='sandbox', priority 11
353Toolchain '/usr/bin/winegcc' at priority 11 will be ignored because another toolchain for arch 'sandbox' has priority 11
354Tool chain test: OK, arch='arm', priority 4
355Toolchain '/usr/bin/arm-linux-gnueabihf-gcc' at priority 4 will be ignored because another toolchain for arch 'arm' has priority 1
356List of available toolchains (34):
357aarch64 : /toolchains/gcc-4.9.0-nolibc/aarch64-linux/bin/aarch64-linux-gcc
358alpha : /toolchains/gcc-4.9.0-nolibc/alpha-linux/bin/alpha-linux-gcc
359am33_2.0 : /toolchains/gcc-4.9.0-nolibc/am33_2.0-linux/bin/am33_2.0-linux-gcc
360arm : /opt/arm-eabi-4.6/bin/arm-eabi-gcc
361avr32 : /toolchains/gcc-4.2.4-nolibc/avr32-linux/bin/avr32-linux-gcc
362bfin : /toolchains/gcc-4.6.3-nolibc/bfin-uclinux/bin/bfin-uclinux-gcc
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000363c89 : /usr/bin/c89-gcc
364c99 : /usr/bin/c99-gcc
Simon Glass17bce662016-03-12 18:50:32 -0700365frv : /toolchains/gcc-4.9.0-nolibc/frv-linux/bin/frv-linux-gcc
366h8300 : /toolchains/gcc-4.9.0-nolibc/h8300-elf/bin/h8300-elf-gcc
367hppa : /toolchains/gcc-4.9.0-nolibc/hppa-linux/bin/hppa-linux-gcc
368hppa64 : /toolchains/gcc-4.9.0-nolibc/hppa64-linux/bin/hppa64-linux-gcc
369i386 : /toolchains/gcc-4.9.0-nolibc/i386-linux/bin/i386-linux-gcc
370i586 : /usr/bin/i586-mingw32msvc-gcc
371ia64 : /toolchains/gcc-4.9.0-nolibc/ia64-linux/bin/ia64-linux-gcc
372m32r : /toolchains/gcc-4.9.0-nolibc/m32r-linux/bin/m32r-linux-gcc
373m68k : /toolchains/gcc-4.9.0-nolibc/m68k-linux/bin/m68k-linux-gcc
374microblaze: /toolchains/gcc-4.9.0-nolibc/microblaze-linux/bin/microblaze-linux-gcc
375mips : /toolchains/gcc-4.9.0-nolibc/mips-linux/bin/mips-linux-gcc
376mips64 : /toolchains/gcc-4.9.0-nolibc/mips64-linux/bin/mips64-linux-gcc
377or32 : /toolchains/gcc-4.5.1-nolibc/or32-linux/bin/or32-linux-gcc
378powerpc : /toolchains/gcc-4.9.0-nolibc/powerpc-linux/bin/powerpc-linux-gcc
379powerpc64 : /toolchains/gcc-4.9.0-nolibc/powerpc64-linux/bin/powerpc64-linux-gcc
380ppc64le : /toolchains/gcc-4.9.0-nolibc/ppc64le-linux/bin/ppc64le-linux-gcc
381s390x : /toolchains/gcc-4.9.0-nolibc/s390x-linux/bin/s390x-linux-gcc
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000382sandbox : /usr/bin/gcc
Simon Glass17bce662016-03-12 18:50:32 -0700383sh4 : /toolchains/gcc-4.6.3-nolibc/sh4-linux/bin/sh4-linux-gcc
384sparc : /toolchains/gcc-4.9.0-nolibc/sparc-linux/bin/sparc-linux-gcc
385sparc64 : /toolchains/gcc-4.9.0-nolibc/sparc64-linux/bin/sparc64-linux-gcc
386tilegx : /toolchains/gcc-4.6.2-nolibc/tilegx-linux/bin/tilegx-linux-gcc
387x86 : /opt/gcc-4.6.3-nolibc/x86_64-linux/bin/x86_64-linux-gcc
388x86_64 : /toolchains/gcc-4.9.0-nolibc/x86_64-linux/bin/x86_64-linux-gcc
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000389
390
391You can see that everything is covered, even some strange ones that won't
392be used (c88 and c99). This is a feature.
393
394
Simon Glass827e37b2014-12-01 17:34:06 -07003955. Install new toolchains if needed
396
397You can download toolchains and update the [toolchain] section of the
398settings file to find them.
399
400To make this easier, buildman can automatically download and install
401toolchains from kernel.org. First list the available architectures:
402
Bin Meng9f244b22015-07-16 19:43:46 -0700403$ ./tools/buildman/buildman --fetch-arch list
Simon Glass827e37b2014-12-01 17:34:06 -0700404Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.6.3/
405Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.6.2/
406Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.5.1/
407Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.2.4/
408Available architectures: alpha am33_2.0 arm avr32 bfin cris crisv32 frv h8300
409hppa hppa64 i386 ia64 m32r m68k mips mips64 or32 powerpc powerpc64 s390x sh4
410sparc sparc64 tilegx x86_64 xtensa
411
412Then pick one and download it:
413
Bin Meng9f244b22015-07-16 19:43:46 -0700414$ ./tools/buildman/buildman --fetch-arch or32
Simon Glass827e37b2014-12-01 17:34:06 -0700415Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.6.3/
416Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.6.2/
417Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.5.1/
418Downloading: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.5.1//x86_64-gcc-4.5.1-nolibc_or32-linux.tar.xz
419Unpacking to: /home/sjg/.buildman-toolchains
420Testing
421 - looking in '/home/sjg/.buildman-toolchains/gcc-4.5.1-nolibc/or32-linux/.'
422 - looking in '/home/sjg/.buildman-toolchains/gcc-4.5.1-nolibc/or32-linux/bin'
423 - found '/home/sjg/.buildman-toolchains/gcc-4.5.1-nolibc/or32-linux/bin/or32-linux-gcc'
424Tool chain test: OK
425
Thomas Chou89515232015-11-12 09:29:09 +0800426Or download them all from kernel.org and move them to /toolchains directory,
427
Simon Glass8ea42102016-07-27 20:32:58 -0600428$ ./tools/buildman/buildman --fetch-arch all
Thomas Chou89515232015-11-12 09:29:09 +0800429$ sudo mkdir -p /toolchains
430$ sudo mv ~/.buildman-toolchains/*/* /toolchains/
431
432For those not available from kernel.org, download from the following links.
433
434arc: https://github.com/foss-for-synopsys-dwc-arc-processors/toolchain/releases/
435 arc_gnu_2015.06_prebuilt_uclibc_le_archs_linux_install.tar.gz
436blackfin: http://sourceforge.net/projects/adi-toolchain/files/
437 blackfin-toolchain-elf-gcc-4.5-2014R1_45-RC2.x86_64.tar.bz2
438nds32: http://osdk.andestech.com/packages/
439 nds32le-linux-glibc-v1.tgz
440nios2: http://sourcery.mentor.com/public/gnu_toolchain/nios2-linux-gnu/
441 sourceryg++-2015.11-27-nios2-linux-gnu-i686-pc-linux-gnu.tar.bz2
442sh: http://sourcery.mentor.com/public/gnu_toolchain/sh-linux-gnu/
443 renesas-4.4-200-sh-linux-gnu-i686-pc-linux-gnu.tar.bz2
444
Simon Glass8ea42102016-07-27 20:32:58 -0600445Note openrisc kernel.org toolchain is out of date. Download the latest one from
446http://opencores.org/or1k/OpenRISC_GNU_tool_chain#Prebuilt_versions - eg:
Bin Menge8aebc42016-02-21 21:18:02 -0800447ftp://ocuser:ocuser@openrisc.opencores.org/toolchain/gcc-or1k-elf-4.8.1-x86.tar.bz2.
448
Simon Glass827e37b2014-12-01 17:34:06 -0700449Buildman should now be set up to use your new toolchain.
450
451At the time of writing, U-Boot has these architectures:
452
453 arc, arm, avr32, blackfin, m68k, microblaze, mips, nds32, nios2, openrisc
454 powerpc, sandbox, sh, sparc, x86
455
Michal Simek12462312015-04-20 11:46:24 +0200456Of these, only arc and nds32 are not available at kernel.org..
Simon Glass827e37b2014-12-01 17:34:06 -0700457
458
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000459How to run it
460=============
461
462First do a dry run using the -n flag: (replace <branch> with a real, local
463branch with a valid upstream)
464
465$ ./tools/buildman/buildman -b <branch> -n
466
467If it can't detect the upstream branch, try checking out the branch, and
Simon Glass2a9e2c62014-12-01 17:33:54 -0700468doing something like 'git branch --set-upstream-to upstream/master'
469or something similar. Buildman will try to guess a suitable upstream branch
470if it can't find one (you will see a message like" Guessing upstream as ...).
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000471
Simon Glasscec83c32014-08-09 15:32:57 -0600472As an example:
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000473
474Dry run, so not doing much. But I would do this:
475
476Building 18 commits for 1059 boards (4 threads, 1 job per thread)
477Build directory: ../lcd9b
478 5bb3505 Merge branch 'master' of git://git.denx.de/u-boot-arm
479 c18f1b4 tegra: Use const for pinmux_config_pingroup/table()
480 2f043ae tegra: Add display support to funcmux
481 e349900 tegra: fdt: Add pwm binding and node
482 424a5f0 tegra: fdt: Add LCD definitions for Tegra
483 0636ccf tegra: Add support for PWM
484 a994fe7 tegra: Add SOC support for display/lcd
485 fcd7350 tegra: Add LCD driver
486 4d46e9d tegra: Add LCD support to Nvidia boards
487 991bd48 arm: Add control over cachability of memory regions
488 54e8019 lcd: Add CONFIG_LCD_ALIGNMENT to select frame buffer alignment
489 d92aff7 lcd: Add support for flushing LCD fb from dcache after update
490 dbd0677 tegra: Align LCD frame buffer to section boundary
491 0cff9b8 tegra: Support control of cache settings for LCD
492 9c56900 tegra: fdt: Add LCD definitions for Seaboard
493 5cc29db lcd: Add CONFIG_CONSOLE_SCROLL_LINES option to speed console
494 cac5a23 tegra: Enable display/lcd support on Seaboard
495 49ff541 wip
496
497Total boards to build for each commit: 1059
498
499This shows that it will build all 1059 boards, using 4 threads (because
500we have a 4-core CPU). Each thread will run with -j1, meaning that each
501make job will use a single CPU. The list of commits to be built helps you
502confirm that things look about right. Notice that buildman has chosen a
503'base' directory for you, immediately above your source tree.
504
505Buildman works entirely inside the base directory, here ../lcd9b,
506creating a working directory for each thread, and creating output
507directories for each commit and board.
508
509
510Suggested Workflow
511==================
512
513To run the build for real, take off the -n:
514
515$ ./tools/buildman/buildman -b <branch>
516
517Buildman will set up some working directories, and get started. After a
518minute or so it will settle down to a steady pace, with a display like this:
519
520Building 18 commits for 1059 boards (4 threads, 1 job per thread)
521 528 36 124 /19062 1:13:30 : SIMPC8313_SP
522
523This means that it is building 19062 board/commit combinations. So far it
Simon Glasscec83c32014-08-09 15:32:57 -0600524has managed to successfully build 528. Another 36 have built with warnings,
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000525and 124 more didn't build at all. Buildman expects to complete the process
Simon Glass8ea42102016-07-27 20:32:58 -0600526in around an hour and a quarter. Use this time to buy a faster computer.
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000527
528
529To find out how the build went, ask for a summary with -s. You can do this
Dirk Behme3e1ded12014-12-23 07:41:26 +0100530either before the build completes (presumably in another terminal) or
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000531afterwards. Let's work through an example of how this is used:
532
533$ ./tools/buildman/buildman -b lcd9b -s
534...
53501: Merge branch 'master' of git://git.denx.de/u-boot-arm
536 powerpc: + galaxy5200_LOWBOOT
53702: tegra: Use const for pinmux_config_pingroup/table()
53803: tegra: Add display support to funcmux
53904: tegra: fdt: Add pwm binding and node
54005: tegra: fdt: Add LCD definitions for Tegra
54106: tegra: Add support for PWM
54207: tegra: Add SOC support for display/lcd
54308: tegra: Add LCD driver
54409: tegra: Add LCD support to Nvidia boards
54510: arm: Add control over cachability of memory regions
54611: lcd: Add CONFIG_LCD_ALIGNMENT to select frame buffer alignment
54712: lcd: Add support for flushing LCD fb from dcache after update
548 arm: + lubbock
54913: tegra: Align LCD frame buffer to section boundary
55014: tegra: Support control of cache settings for LCD
55115: tegra: fdt: Add LCD definitions for Seaboard
55216: lcd: Add CONFIG_CONSOLE_SCROLL_LINES option to speed console
55317: tegra: Enable display/lcd support on Seaboard
55418: wip
555
556This shows which commits have succeeded and which have failed. In this case
557the build is still in progress so many boards are not built yet (use -u to
558see which ones). But still we can see a few failures. The galaxy5200_LOWBOOT
559never builds correctly. This could be a problem with our toolchain, or it
560could be a bug in the upstream. The good news is that we probably don't need
Simon Glass8ea42102016-07-27 20:32:58 -0600561to blame our commits. The bad news is that our commits are not tested on that
562board.
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000563
564Commit 12 broke lubbock. That's what the '+ lubbock' means. The failure
565is never fixed by a later commit, or you would see lubbock again, in green,
566without the +.
567
568To see the actual error:
569
570$ ./tools/buildman/buildman -b <branch> -se lubbock
571...
57212: lcd: Add support for flushing LCD fb from dcache after update
573 arm: + lubbock
574+common/libcommon.o: In function `lcd_sync':
575+/u-boot/lcd9b/.bm-work/00/common/lcd.c:120: undefined reference to `flush_dcache_range'
576+arm-none-linux-gnueabi-ld: BFD (Sourcery G++ Lite 2010q1-202) 2.19.51.20090709 assertion fail /scratch/julian/2010q1-release-linux-lite/obj/binutils-src-2010q1-202-arm-none-linux-gnueabi-i686-pc-linux-gnu/bfd/elf32-arm.c:12572
577+make: *** [/u-boot/lcd9b/.bm-work/00/build/u-boot] Error 139
57813: tegra: Align LCD frame buffer to section boundary
57914: tegra: Support control of cache settings for LCD
58015: tegra: fdt: Add LCD definitions for Seaboard
58116: lcd: Add CONFIG_CONSOLE_SCROLL_LINES option to speed console
582-/u-boot/lcd9b/.bm-work/00/common/lcd.c:120: undefined reference to `flush_dcache_range'
583+/u-boot/lcd9b/.bm-work/00/common/lcd.c:125: undefined reference to `flush_dcache_range'
58417: tegra: Enable display/lcd support on Seaboard
58518: wip
586
587So the problem is in lcd.c, due to missing cache operations. This information
588should be enough to work out what that commit is doing to break these
589boards. (In this case pxa did not have cache operations defined).
590
Simon Glass8ea42102016-07-27 20:32:58 -0600591If you see error lines marked with '-', that means that the errors were fixed
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000592by that commit. Sometimes commits can be in the wrong order, so that a
593breakage is introduced for a few commits and fixed by later commits. This
594shows up clearly with buildman. You can then reorder the commits and try
595again.
596
Simon Glass8ea42102016-07-27 20:32:58 -0600597At commit 16, the error moves: you can see that the old error at line 120
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000598is fixed, but there is a new one at line 126. This is probably only because
Dirk Behme3e1ded12014-12-23 07:41:26 +0100599we added some code and moved the broken line further down the file.
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000600
601If many boards have the same error, then -e will display the error only
Simon Glassed966652014-08-28 09:43:43 -0600602once. This makes the output as concise as possible. To see which boards have
Simon Glass8ea42102016-07-27 20:32:58 -0600603each error, use -l. So it is safe to omit the board name - you will not get
604lots of repeated output for every board.
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000605
Simon Glasse30965d2014-08-28 09:43:44 -0600606Buildman tries to distinguish warnings from errors, and shows warning lines
607separately with a 'w' prefix.
608
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000609The full build output in this case is available in:
610
611../lcd9b/12_of_18_gd92aff7_lcd--Add-support-for/lubbock/
612
613 done: Indicates the build was done, and holds the return code from make.
614 This is 0 for a good build, typically 2 for a failure.
615
616 err: Output from stderr, if any. Errors and warnings appear here.
617
618 log: Output from stdout. Normally there isn't any since buildman runs
Simon Glassc81d0d22016-03-12 18:50:33 -0700619 in silent mode. Use -V to force a verbose build (this passes V=1
620 to 'make')
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000621
622 toolchain: Shows information about the toolchain used for the build.
623
624 sizes: Shows image size information.
625
Simon Glass8ea42102016-07-27 20:32:58 -0600626It is possible to get the build binary output there also. Use the -k option
627for this. In that case you will also see some output files, like:
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000628
629 System.map toolchain u-boot u-boot.bin u-boot.map autoconf.mk
630 (also SPL versions u-boot-spl and u-boot-spl.bin if available)
631
632
633Checking Image Sizes
634====================
635
636A key requirement for U-Boot is that you keep code/data size to a minimum.
637Where a new feature increases this noticeably it should normally be put
Simon Glass8ea42102016-07-27 20:32:58 -0600638behind a CONFIG flag so that boards can leave it disabled and keep the image
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000639size more or less the same with each new release.
640
641To check the impact of your commits on image size, use -S. For example:
642
643$ ./tools/buildman/buildman -b us-x86 -sS
644Summary of 10 commits for 1066 boards (4 threads, 1 job per thread)
64501: MAKEALL: add support for per architecture toolchains
64602: x86: Add function to get top of usable ram
647 x86: (for 1/3 boards) text -272.0 rodata +41.0
64803: x86: Add basic cache operations
64904: x86: Permit bootstage and timer data to be used prior to relocation
650 x86: (for 1/3 boards) data +16.0
65105: x86: Add an __end symbol to signal the end of the U-Boot binary
652 x86: (for 1/3 boards) text +76.0
65306: x86: Rearrange the output input to remove BSS
654 x86: (for 1/3 boards) bss -2140.0
65507: x86: Support relocation of FDT on start-up
656 x86: + coreboot-x86
65708: x86: Add error checking to x86 relocation code
65809: x86: Adjust link device tree include file
65910: x86: Enable CONFIG_OF_CONTROL on coreboot
660
661
662You can see that image size only changed on x86, which is good because this
663series is not supposed to change any other board. From commit 7 onwards the
664build fails so we don't get code size numbers. The numbers are fractional
665because they are an average of all boards for that architecture. The
666intention is to allow you to quickly find image size problems introduced by
667your commits.
668
669Note that the 'text' region and 'rodata' are split out. You should add the
670two together to get the total read-only size (reported as the first column
671in the output from binutil's 'size' utility).
672
673A useful option is --step which lets you skip some commits. For example
674--step 2 will show the image sizes for only every 2nd commit (so it will
675compare the image sizes of the 1st, 3rd, 5th... commits). You can also use
676--step 0 which will compare only the first and last commits. This is useful
Simon Glass8ea42102016-07-27 20:32:58 -0600677for an overview of how your entire series affects code size. It will build
678only the upstream commit and your final branch commit.
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000679
680You can also use -d to see a detailed size breakdown for each board. This
681list is sorted in order from largest growth to largest reduction.
682
Simon Glass8ea42102016-07-27 20:32:58 -0600683It is even possible to go a little further with the -B option (--bloat). This
Simon Glasscec83c32014-08-09 15:32:57 -0600684shows where U-Boot has bloated, breaking the size change down to the function
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000685level. Example output is below:
686
687$ ./tools/buildman/buildman -b us-mem4 -sSdB
688...
68919: Roll crc32 into hash infrastructure
690 arm: (for 10/10 boards) all -143.4 bss +1.2 data -4.8 rodata -48.2 text -91.6
691 paz00 : all +23 bss -4 rodata -29 text +56
692 u-boot: add: 1/0, grow: 3/-2 bytes: 168/-104 (64)
693 function old new delta
694 hash_command 80 160 +80
695 crc32_wd_buf - 56 +56
696 ext4fs_read_file 540 568 +28
697 insert_var_value_sub 688 692 +4
698 run_list_real 1996 1992 -4
699 do_mem_crc 168 68 -100
700 trimslice : all -9 bss +16 rodata -29 text +4
701 u-boot: add: 1/0, grow: 1/-3 bytes: 136/-124 (12)
702 function old new delta
703 hash_command 80 160 +80
704 crc32_wd_buf - 56 +56
705 ext4fs_iterate_dir 672 668 -4
706 ext4fs_read_file 568 548 -20
707 do_mem_crc 168 68 -100
708 whistler : all -9 bss +16 rodata -29 text +4
709 u-boot: add: 1/0, grow: 1/-3 bytes: 136/-124 (12)
710 function old new delta
711 hash_command 80 160 +80
712 crc32_wd_buf - 56 +56
713 ext4fs_iterate_dir 672 668 -4
714 ext4fs_read_file 568 548 -20
715 do_mem_crc 168 68 -100
716 seaboard : all -9 bss -28 rodata -29 text +48
717 u-boot: add: 1/0, grow: 3/-2 bytes: 160/-104 (56)
718 function old new delta
719 hash_command 80 160 +80
720 crc32_wd_buf - 56 +56
721 ext4fs_read_file 548 568 +20
722 run_list_real 1996 2000 +4
723 do_nandboot 760 756 -4
724 do_mem_crc 168 68 -100
Marcel Ziswilere57c6e52015-03-26 01:31:54 +0100725 colibri_t20 : all -9 rodata -29 text +20
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000726 u-boot: add: 1/0, grow: 2/-3 bytes: 140/-112 (28)
727 function old new delta
728 hash_command 80 160 +80
729 crc32_wd_buf - 56 +56
730 read_abs_bbt 204 208 +4
731 do_nandboot 760 756 -4
732 ext4fs_read_file 576 568 -8
733 do_mem_crc 168 68 -100
734 ventana : all -37 bss -12 rodata -29 text +4
735 u-boot: add: 1/0, grow: 1/-3 bytes: 136/-124 (12)
736 function old new delta
737 hash_command 80 160 +80
738 crc32_wd_buf - 56 +56
739 ext4fs_iterate_dir 672 668 -4
740 ext4fs_read_file 568 548 -20
741 do_mem_crc 168 68 -100
742 harmony : all -37 bss -16 rodata -29 text +8
743 u-boot: add: 1/0, grow: 2/-3 bytes: 140/-124 (16)
744 function old new delta
745 hash_command 80 160 +80
746 crc32_wd_buf - 56 +56
747 nand_write_oob_syndrome 428 432 +4
748 ext4fs_iterate_dir 672 668 -4
749 ext4fs_read_file 568 548 -20
750 do_mem_crc 168 68 -100
751 medcom-wide : all -417 bss +28 data -16 rodata -93 text -336
752 u-boot: add: 1/-1, grow: 1/-2 bytes: 88/-376 (-288)
753 function old new delta
754 crc32_wd_buf - 56 +56
755 do_fat_read_at 2872 2904 +32
756 hash_algo 16 - -16
757 do_mem_crc 168 68 -100
758 hash_command 420 160 -260
759 tec : all -449 bss -4 data -16 rodata -93 text -336
760 u-boot: add: 1/-1, grow: 1/-2 bytes: 88/-376 (-288)
761 function old new delta
762 crc32_wd_buf - 56 +56
763 do_fat_read_at 2872 2904 +32
764 hash_algo 16 - -16
765 do_mem_crc 168 68 -100
766 hash_command 420 160 -260
767 plutux : all -481 bss +16 data -16 rodata -93 text -388
768 u-boot: add: 1/-1, grow: 1/-3 bytes: 68/-408 (-340)
769 function old new delta
770 crc32_wd_buf - 56 +56
771 do_load_serial_bin 1688 1700 +12
772 hash_algo 16 - -16
773 do_fat_read_at 2904 2872 -32
774 do_mem_crc 168 68 -100
775 hash_command 420 160 -260
776 powerpc: (for 5/5 boards) all +37.4 data -3.2 rodata -41.8 text +82.4
777 MPC8610HPCD : all +55 rodata -29 text +84
778 u-boot: add: 1/0, grow: 0/-1 bytes: 176/-96 (80)
779 function old new delta
780 hash_command - 176 +176
781 do_mem_crc 184 88 -96
782 MPC8641HPCN : all +55 rodata -29 text +84
783 u-boot: add: 1/0, grow: 0/-1 bytes: 176/-96 (80)
784 function old new delta
785 hash_command - 176 +176
786 do_mem_crc 184 88 -96
787 MPC8641HPCN_36BIT: all +55 rodata -29 text +84
788 u-boot: add: 1/0, grow: 0/-1 bytes: 176/-96 (80)
789 function old new delta
790 hash_command - 176 +176
791 do_mem_crc 184 88 -96
792 sbc8641d : all +55 rodata -29 text +84
793 u-boot: add: 1/0, grow: 0/-1 bytes: 176/-96 (80)
794 function old new delta
795 hash_command - 176 +176
796 do_mem_crc 184 88 -96
797 xpedite517x : all -33 data -16 rodata -93 text +76
798 u-boot: add: 1/-1, grow: 0/-1 bytes: 176/-112 (64)
799 function old new delta
800 hash_command - 176 +176
801 hash_algo 16 - -16
802 do_mem_crc 184 88 -96
803...
804
805
Simon Glass8ea42102016-07-27 20:32:58 -0600806This shows that commit 19 has reduced codesize for arm slightly and increased
807it for powerpc. This increase was offset in by reductions in rodata and
808data/bss.
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000809
Dirk Behme3e1ded12014-12-23 07:41:26 +0100810Shown below the summary lines are the sizes for each board. Below each board
811are the sizes for each function. This information starts with:
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000812
813 add - number of functions added / removed
814 grow - number of functions which grew / shrunk
815 bytes - number of bytes of code added to / removed from all functions,
816 plus the total byte change in brackets
817
818The change seems to be that hash_command() has increased by more than the
819do_mem_crc() function has decreased. The function sizes typically add up to
820roughly the text area size, but note that every read-only section except
821rodata is included in 'text', so the function total does not exactly
822correspond.
823
824It is common when refactoring code for the rodata to decrease as the text size
825increases, and vice versa.
826
827
Simon Glass62005342014-12-01 17:34:02 -0700828The .buildman file
829==================
Simon Glass4281ad82013-09-23 17:35:17 -0600830
Simon Glass62005342014-12-01 17:34:02 -0700831The .buildman file provides information about the available toolchains and
832also allows build flags to be passed to 'make'. It consists of several
833sections, with the section name in square brackets. Within each section are
834a set of (tag, value) pairs.
Simon Glass4281ad82013-09-23 17:35:17 -0600835
Simon Glass62005342014-12-01 17:34:02 -0700836'[toolchain]' section
Simon Glass4281ad82013-09-23 17:35:17 -0600837
Simon Glass62005342014-12-01 17:34:02 -0700838 This lists the available toolchains. The tag here doesn't matter, but
839 make sure it is unique. The value is the path to the toolchain. Buildman
840 will look in that path for a file ending in 'gcc'. It will then execute
841 it to check that it is a C compiler, passing only the --version flag to
842 it. If the return code is 0, buildman assumes that it is a valid C
843 compiler. It uses the first part of the name as the architecture and
844 strips off the last part when setting the CROSS_COMPILE environment
845 variable (parts are delimited with a hyphen).
Simon Glass4281ad82013-09-23 17:35:17 -0600846
Simon Glass62005342014-12-01 17:34:02 -0700847 For example powerpc-linux-gcc will be noted as a toolchain for 'powerpc'
848 and CROSS_COMPILE will be set to powerpc-linux- when using it.
849
850'[toolchain-alias]' section
851
852 This converts toolchain architecture names to U-Boot names. For example,
853 if an x86 toolchains is called i386-linux-gcc it will not normally be
Simon Glass9b83bfd2014-12-01 17:34:05 -0700854 used for architecture 'x86'. Adding 'x86: i386 x86_64' to this section
855 will tell buildman that the i386 and x86_64 toolchains can be used for
856 the x86 architecture.
Simon Glass62005342014-12-01 17:34:02 -0700857
858'[make-flags]' section
859
860 U-Boot's build system supports a few flags (such as BUILD_TAG) which
861 affect the build product. These flags can be specified in the buildman
862 settings file. They can also be useful when building U-Boot against other
863 open source software.
864
865 [make-flags]
866 at91-boards=ENABLE_AT91_TEST=1
867 snapper9260=${at91-boards} BUILD_TAG=442
868 snapper9g45=${at91-boards} BUILD_TAG=443
869
870 This will use 'make ENABLE_AT91_TEST=1 BUILD_TAG=442' for snapper9260
871 and 'make ENABLE_AT91_TEST=1 BUILD_TAG=443' for snapper9g45. A special
872 variable ${target} is available to access the target name (snapper9260
873 and snapper9g20 in this case). Variables are resolved recursively. Note
874 that variables can only contain the characters A-Z, a-z, 0-9, hyphen (-)
875 and underscore (_).
876
877 It is expected that any variables added are dealt with in U-Boot's
878 config.mk file and documented in the README.
879
880 Note that you can pass ad-hoc options to the build using environment
881 variables, for example:
882
883 SOME_OPTION=1234 ./tools/buildman/buildman my_board
Simon Glass4281ad82013-09-23 17:35:17 -0600884
885
Simon Glasse5a0e5d2014-08-09 15:33:03 -0600886Quick Sanity Check
887==================
888
889If you have made changes and want to do a quick sanity check of the
Simon Glass1d8104f2014-10-16 01:05:56 -0600890currently checked-out source, run buildman without the -b flag. This will
891build the selected boards and display build status as it runs (i.e. -v is
892enabled automatically). Use -e to see errors/warnings as well.
Simon Glasse5a0e5d2014-08-09 15:33:03 -0600893
894
Simon Glass5abab202014-12-01 17:33:57 -0700895Building Ranges
896===============
897
898You can build a range of commits by specifying a range instead of a branch
899when using the -b flag. For example:
900
901 upstream/master..us-buildman
902
903will build commits in us-buildman that are not in upstream/master.
904
905
Stephen Warrenf79f1e02016-04-11 10:48:44 -0600906Building Faster
907===============
908
909By default, buildman executes 'make mrproper' prior to building the first
910commit for each board. This causes everything to be built from scratch. If you
911trust the build system's incremental build capabilities, you can pass the -I
912flag to skip the 'make mproper' invocation, which will reduce the amount of
913work 'make' does, and hence speed up the build. This flag will speed up any
914buildman invocation, since it reduces the amount of work done on any build.
915
916One possible application of buildman is as part of a continual edit, build,
917edit, build, ... cycle; repeatedly applying buildman to the same change or
918series of changes while making small incremental modifications to the source
919each time. This provides quick feedback regarding the correctness of recent
920modifications. In this scenario, buildman's default choice of build directory
921causes more build work to be performed than strictly necessary.
922
923By default, each buildman thread uses a single directory for all builds. When a
924thread builds multiple boards, the configuration built in this directory will
925cycle through various different configurations, one per board built by the
926thread. Variations in the configuration will force a rebuild of affected source
927files when a thread switches between boards. Ideally, such buildman-induced
928rebuilds would not happen, thus allowing the build to operate as efficiently as
929the build system and source changes allow. buildman's -P flag may be used to
930enable this; -P causes each board to be built in a separate (board-specific)
931directory, thus avoiding any buildman-induced configuration changes in any
932build directory.
933
934U-Boot's build system embeds information such as a build timestamp into the
935final binary. This information varies each time U-Boot is built. This causes
936various files to be rebuilt even if no source changes are made, which in turn
937requires that the final U-Boot binary be re-linked. This unnecessary work can
938be avoided by turning off the timestamp feature. This can be achieved by
939setting the SOURCE_DATE_EPOCH environment variable to 0.
940
941Combining all of these options together yields the command-line shown below.
942This will provide the quickest possible feedback regarding the current content
943of the source tree, thus allowing rapid tested evolution of the code.
944
945 SOURCE_DATE_EPOCH=0 ./tools/buildman/buildman -I -P tegra
946
947
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000948Other options
949=============
950
951Buildman has various other command line options. Try --help to see them.
952
Simon Glass2c3deb92014-08-28 09:43:39 -0600953When doing builds, Buildman's return code will reflect the overall result:
954
955 0 (success) No errors or warnings found
956 128 Errors found
957 129 Warnings found
958
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000959
Simon Glass6eede342014-08-09 15:32:58 -0600960How to change from MAKEALL
961==========================
962
963Buildman includes most of the features of MAKEALL and is generally faster
964and easier to use. In particular it builds entire branches: if a particular
965commit introduces an error in a particular board, buildman can easily show
966you this, even if a later commit fixes that error.
967
968The reasons to deprecate MAKEALL are:
969- We don't want to maintain two build systems
970- Buildman is typically faster
971- Buildman has a lot more features
972
973But still, many people will be sad to lose MAKEALL. If you are used to
974MAKEALL, here are a few pointers.
975
976First you need to set up your tool chains - see the 'Setting up' section
977for details. Once you have your required toolchain(s) detected then you are
978ready to go.
979
Simon Glasse5a0e5d2014-08-09 15:33:03 -0600980To build the current source tree, run buildman without a -b flag:
981
982 ./tools/buildman/buildman <list of things to build>
983
984This will build the current source tree for the given boards and display
985the results and errors.
986
987However buildman usually works on entire branches, and for that you must
988specify a board flag:
Simon Glass6eede342014-08-09 15:32:58 -0600989
990 ./tools/buildman/buildman -b <branch_name> <list of things to build>
991
992followed by (afterwards, or perhaps concurrently in another terminal):
993
994 ./tools/buildman/buildman -b <branch_name> -s <list of things to build>
995
996to see the results of the build. Rather than showing you all the output,
997buildman just shows a summary, with red indicating that a commit introduced
998an error and green indicating that a commit fixed an error. Use the -e
Simon Glassed966652014-08-28 09:43:43 -0600999flag to see the full errors and -l to see which boards caused which errors.
Simon Glass6eede342014-08-09 15:32:58 -06001000
Simon Glasse5a0e5d2014-08-09 15:33:03 -06001001If you really want to see build results as they happen, use -v when doing a
Simon Glass1d8104f2014-10-16 01:05:56 -06001002build (and -e to see the errors/warnings too).
Simon Glasse5a0e5d2014-08-09 15:33:03 -06001003
Simon Glass6eede342014-08-09 15:32:58 -06001004You don't need to stick around on that branch while buildman is running. It
1005checks out its own copy of the source code, so you can change branches,
1006add commits, etc. without affecting the build in progress.
1007
1008The <list of things to build> can include board names, architectures or the
1009like. There are no flags to disambiguate since ambiguities are rare. Using
1010the examples from MAKEALL:
1011
1012Examples:
1013 - build all Power Architecture boards:
1014 MAKEALL -a powerpc
1015 MAKEALL --arch powerpc
1016 MAKEALL powerpc
1017 ** buildman -b <branch> powerpc
1018 - build all PowerPC boards manufactured by vendor "esd":
1019 MAKEALL -a powerpc -v esd
1020 ** buildman -b <branch> esd
1021 - build all PowerPC boards manufactured either by "keymile" or "siemens":
1022 MAKEALL -a powerpc -v keymile -v siemens
1023 ** buildman -b <branch> keymile siemens
1024 - build all Freescale boards with MPC83xx CPUs, plus all 4xx boards:
1025 MAKEALL -c mpc83xx -v freescale 4xx
1026 ** buildman -b <branch> mpc83xx freescale 4xx
1027
1028Buildman automatically tries to use all the CPUs in your machine. If you
1029are building a lot of boards it will use one thread for every CPU core
1030it detects in your machine. This is like MAKEALL's BUILD_NBUILDS option.
1031You can use the -T flag to change the number of threads. If you are only
1032building a few boards, buildman will automatically run make with the -j
1033flag to increase the number of concurrent make tasks. It isn't normally
1034that helpful to fiddle with this option, but if you use the BUILD_NCPUS
1035option in MAKEALL then -j is the equivalent in buildman.
1036
1037Buildman puts its output in ../<branch_name> by default but you can change
1038this with the -o option. Buildman normally does out-of-tree builds: use -i
1039to disable that if you really want to. But be careful that once you have
1040used -i you pollute buildman's copies of the source tree, and you will need
1041to remove the build directory (normally ../<branch_name>) to run buildman
1042in normal mode (without -i).
1043
1044Buildman doesn't keep the output result normally, but use the -k option to
1045do this.
1046
1047Please read 'Theory of Operation' a few times as it will make a lot of
1048things clearer.
1049
1050Some options you might like are:
1051
1052 -B shows which functions are growing/shrinking in which commit - great
1053 for finding code bloat.
1054 -S shows image sizes for each commit (just an overall summary)
1055 -u shows boards that you haven't built yet
1056 --step 0 will build just the upstream commit and the last commit of your
1057 branch. This is often a quick sanity check that your branch doesn't
1058 break anything. But note this does not check bisectability!
1059
1060
Simon Glassfc3fe1c2013-04-03 11:07:16 +00001061TODO
1062====
1063
1064This has mostly be written in my spare time as a response to my difficulties
1065in testing large series of patches. Apart from tidying up there is quite a
Simon Glass1d8104f2014-10-16 01:05:56 -06001066bit of scope for improvement. Things like better error diffs and easier
Dirk Behme3e1ded12014-12-23 07:41:26 +01001067access to log files. Also it would be nice if buildman could 'hunt' for
Simon Glass1d8104f2014-10-16 01:05:56 -06001068problems, perhaps by building a few boards for each arch, or checking
1069commits for changed files and building only boards which use those files.
Simon Glassfc3fe1c2013-04-03 11:07:16 +00001070
Simon Glass8ea42102016-07-27 20:32:58 -06001071A specific problem to fix is that Ctrl-C does not exit buildman cleanly when
1072multiple builder threads are active.
Simon Glassfc3fe1c2013-04-03 11:07:16 +00001073
1074Credits
1075=======
1076
1077Thanks to Grant Grundler <grundler@chromium.org> for his ideas for improving
1078the build speed by building all commits for a board instead of the other
1079way around.
1080
1081
Simon Glassfc3fe1c2013-04-03 11:07:16 +00001082Simon Glass
1083sjg@chromium.org
1084Halloween 2012
1085Updated 12-12-12
1086Updated 23-02-13