blob: ca0d1f64460e825621a34cab244161bba8170f65 [file] [log] [blame]
Tom Rini83d290c2018-05-06 17:58:06 -04001# SPDX-License-Identifier: GPL-2.0+
Simon Glassfc3fe1c2013-04-03 11:07:16 +00002# Copyright (c) 2013 The Chromium OS Authors.
Simon Glassfc3fe1c2013-04-03 11:07:16 +00003
Simon Glass6eede342014-08-09 15:32:58 -06004(Please read 'How to change from MAKEALL' if you are used to that tool)
5
Simon Glassc8d73932016-07-27 20:33:06 -06006Quick-start
7===========
8
9If you just want to quickly set up buildman so you can build something (for
10example Raspberry Pi 2):
11
12 cd /path/to/u-boot
13 PATH=$PATH:`pwd`/tools/buildman
14 buildman --fetch-arch arm
15 buildman -k rpi_2
16 ls ../current/rpi_2
17 # u-boot.bin is the output image
18
19
Simon Glassfc3fe1c2013-04-03 11:07:16 +000020What is this?
21=============
22
23This tool handles building U-Boot to check that you have not broken it
24with your patch series. It can build each individual commit and report
25which boards fail on which commits, and which errors come up. It aims
26to make full use of multi-processor machines.
27
28A key feature of buildman is its output summary, which allows warnings,
29errors or image size increases in a particular commit or board to be
30quickly identified and the offending commit pinpointed. This can be a big
31help for anyone working with >10 patches at a time.
32
33
34Caveats
35=======
36
Simon Glassfc3fe1c2013-04-03 11:07:16 +000037Buildman can be stopped and restarted, in which case it will continue
38where it left off. This should happen cleanly and without side-effects.
39If not, it is a bug, for which a patch would be welcome.
40
41Buildman gets so tied up in its work that it can ignore the outside world.
42You may need to press Ctrl-C several times to quit it. Also it will print
Simon Glass8ea42102016-07-27 20:32:58 -060043out various exceptions when stopped. You may have to kill it since the
44Ctrl-C handling is somewhat broken.
Simon Glassfc3fe1c2013-04-03 11:07:16 +000045
46
47Theory of Operation
48===================
49
50(please read this section in full twice or you will be perpetually confused)
51
52Buildman is a builder. It is not make, although it runs make. It does not
53produce any useful output on the terminal while building, except for
Simon Glassf9728362020-04-09 15:08:48 -060054progress information (but see -v below). All the output (errors, warnings and
55binaries if you ask for them) is stored in output directories, which you can
56look at from a separate 'buildman -s' instance while the build is progressing,
57or when it is finished.
Simon Glassfc3fe1c2013-04-03 11:07:16 +000058
Simon Glass8ea42102016-07-27 20:32:58 -060059Buildman is designed to build entire git branches, i.e. muliple commits. It
Simon Glassf9728362020-04-09 15:08:48 -060060can be run repeatedly on the same branch after making changes to commits on
61that branch. In this case it will automatically rebuild commits which have
62changed (and remove its old results for that commit). It is possible to build
63a branch for one board, then later build it for another board. This adds to
64the output, so now you have results for two boards. If you want buildman to
65re-build a commit it has already built (e.g. because of a toolchain update),
66use the -f flag.
Simon Glass8ea42102016-07-27 20:32:58 -060067
Simon Glassfc3fe1c2013-04-03 11:07:16 +000068Buildman produces a concise summary of which boards succeeded and failed.
69It shows which commit introduced which board failure using a simple
Simon Glassf9728362020-04-09 15:08:48 -060070red/green colour coding (with yellow/cyan for warnings). Full error
71information can be requested, in which case it is de-duped and displayed
72against the commit that introduced the error. An example workflow is below.
Simon Glassfc3fe1c2013-04-03 11:07:16 +000073
74Buildman stores image size information and can report changes in image size
75from commit to commit. An example of this is below.
76
77Buildman starts multiple threads, and each thread builds for one board at
78a time. A thread starts at the first commit, configures the source for your
79board and builds it. Then it checks out the next commit and does an
Simon Glassf9728362020-04-09 15:08:48 -060080incremental build (i.e. not using 'make xxx_defconfig' unless you use -C).
81Eventually the thread reaches the last commit and stops. If a commit causes
82an error or warning, buildman will try it again after reconfiguring (but see
83-Q). Thus some commits may be built twice, with the first result silently
84discarded. Lots of errors and warnings will causes lots of reconfigures and your
85build will be very slow. This is because a file that produces just a warning
86would not normally be rebuilt in an incremental build. Once a thread finishes
87building all the commits for a board, it starts on the commits for another
88board.
Simon Glassfc3fe1c2013-04-03 11:07:16 +000089
90Buildman works in an entirely separate place from your U-Boot repository.
91It creates a separate working directory for each thread, and puts the
92output files in the working directory, organised by commit name and board
Simon Glassf9728362020-04-09 15:08:48 -060093name, in a two-level hierarchy (but see -P).
Simon Glassfc3fe1c2013-04-03 11:07:16 +000094
95Buildman is invoked in your U-Boot directory, the one with the .git
96directory. It clones this repository into a copy for each thread, and the
97threads do not affect the state of your git repository. Any checkouts done
98by the thread affect only the working directory for that thread.
99
Simon Glasscec83c32014-08-09 15:32:57 -0600100Buildman automatically selects the correct tool chain for each board. You
Simon Glassf9728362020-04-09 15:08:48 -0600101must supply suitable tool chains (see --fetch-arch), but buildman takes care
102of selecting the right one.
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000103
Simon Glasse5a0e5d2014-08-09 15:33:03 -0600104Buildman generally builds a branch (with the -b flag), and in this case
Simon Glassf9728362020-04-09 15:08:48 -0600105builds the upstream commit as well, for comparison. So even if you have one
106commit in your branch, two commits will be built. Put all your commits in a
107branch, set the branch's upstream to a valid value, and all will be well.
108Otherwise buildman will perform random actions. Use -n to check what the
109random actions might be.
Simon Glasse5a0e5d2014-08-09 15:33:03 -0600110
Simon Glassf9728362020-04-09 15:08:48 -0600111Buildman effectively has two modes: without -s it builds, with -s it
112summarises the results of previous (or active) builds.
113
114If you just want to build the current source tree, leave off the -b flag.
115This will display results and errors as they happen. You can still look at
116them later using -se. Note that buildman will assume that the source has
117changed, and will build all specified boards in this case.
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000118
119Buildman is optimised for building many commits at once, for many boards.
120On multi-core machines, Buildman is fast because it uses most of the
121available CPU power. When it gets to the end, or if you are building just
122a few commits or boards, it will be pretty slow. As a tip, if you don't
123plan to use your machine for anything else, you can use -T to increase the
124number of threads beyond the default.
125
Simon Glass06890362018-06-11 23:26:46 -0600126
127Selecting which boards to build
128===============================
129
Stephen Warren8426d8b2013-10-10 10:00:20 -0600130Buildman lets you build all boards, or a subset. Specify the subset by passing
131command-line arguments that list the desired board name, architecture name,
132SOC name, or anything else in the boards.cfg file. Multiple arguments are
133allowed. Each argument will be interpreted as a regular expression, so
134behaviour is a superset of exact or substring matching. Examples are:
135
136* 'tegra20' All boards with a Tegra20 SoC
137* 'tegra' All boards with any Tegra Soc (Tegra20, Tegra30, Tegra114...)
138* '^tegra[23]0$' All boards with either Tegra20 or Tegra30 SoC
139* 'powerpc' All PowerPC boards
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000140
Simon Glass6131bea2014-08-09 15:33:08 -0600141While the default is to OR the terms together, you can also make use of
142the '&' operator to limit the selection:
143
144* 'freescale & arm sandbox' All Freescale boards with ARM architecture,
145 plus sandbox
146
Simon Glass3cf4ae62014-08-28 09:43:41 -0600147You can also use -x to specifically exclude some boards. For example:
148
Baruch Siach24296132019-07-11 12:57:41 +0300149 buildman arm -x nvidia,freescale,.*ball$
Simon Glass3cf4ae62014-08-28 09:43:41 -0600150
151means to build all arm boards except nvidia, freescale and anything ending
152with 'ball'.
153
Simon Glass9e040b52020-04-09 15:08:49 -0600154For building specific boards you can use the --boards (or --bo) option, which
155takes a comma-separated list of board target names and be used multiple times
156on the command line:
Simon Glass06890362018-06-11 23:26:46 -0600157
Baruch Siach24296132019-07-11 12:57:41 +0300158 buildman --boards sandbox,snow --boards
Simon Glass06890362018-06-11 23:26:46 -0600159
Dirk Behme3e1ded12014-12-23 07:41:26 +0100160It is convenient to use the -n option to see what will be built based on
Simon Glass8d7523c2017-01-23 05:38:56 -0700161the subset given. Use -v as well to get an actual list of boards.
Simon Glass6131bea2014-08-09 15:33:08 -0600162
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000163Buildman does not store intermediate object files. It optionally copies
Simon Glass06890362018-06-11 23:26:46 -0600164the binary output into a directory when a build is successful (-k). Size
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000165information is always recorded. It needs a fair bit of disk space to work,
166typically 250MB per thread.
167
168
169Setting up
170==========
171
1721. Get the U-Boot source. You probably already have it, but if not these
173steps should get you started with a repo and some commits for testing.
174
175$ cd /path/to/u-boot
176$ git clone git://git.denx.de/u-boot.git .
177$ git checkout -b my-branch origin/master
178$ # Add some commits to the branch, reading for testing
179
Simon Glass62005342014-12-01 17:34:02 -07001802. Create ~/.buildman to tell buildman where to find tool chains (see 'The
181.buildman file' later for details). As an example:
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000182
183# Buildman settings file
184
185[toolchain]
186root: /
187rest: /toolchains/*
188eldk: /opt/eldk-4.2
Simon Glasse9569472014-08-09 15:33:07 -0600189arm: /opt/linaro/gcc-linaro-arm-linux-gnueabihf-4.8-2013.08_linux
190aarch64: /opt/linaro/gcc-linaro-aarch64-none-elf-4.8-2013.10_linux
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000191
192[toolchain-alias]
193x86: i386
194blackfin: bfin
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000195nds32: nds32le
Bin Menge8aebc42016-02-21 21:18:02 -0800196openrisc: or1k
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000197
198
199This selects the available toolchain paths. Add the base directory for
200each of your toolchains here. Buildman will search inside these directories
201and also in any '/usr' and '/usr/bin' subdirectories.
202
203Make sure the tags (here root: rest: and eldk:) are unique.
204
205The toolchain-alias section indicates that the i386 toolchain should be used
206to build x86 commits.
207
Simon Glass17bce662016-03-12 18:50:32 -0700208Note that you can also specific exactly toolchain prefixes if you like:
209
210[toolchain-prefix]
211arm: /opt/arm-eabi-4.6/bin/arm-eabi-
212
213or even:
214
215[toolchain-prefix]
216arm: /opt/arm-eabi-4.6/bin/arm-eabi-gcc
217
218This tells buildman that you want to use this exact toolchain for the arm
219architecture. This will override any toolchains found by searching using the
220[toolchain] settings.
221
222Since the toolchain prefix is an explicit request, buildman will report an
223error if a toolchain is not found with that prefix. The current PATH will be
224searched, so it is possible to use:
225
226[toolchain-prefix]
227arm: arm-none-eabi-
228
229and buildman will find arm-none-eabi-gcc in /usr/bin if you have it installed.
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000230
York Sund5fe0132016-10-04 14:33:51 -0700231[toolchain-wrapper]
232wrapper: ccache
233
234This tells buildman to use a compiler wrapper in front of CROSS_COMPILE. In
235this example, ccache. It doesn't affect the toolchain scan. The wrapper is
236added when CROSS_COMPILE environtal variable is set. The name in this
237section is ignored. If more than one line is provided, only the last one
238is taken.
239
Simon Glass34699692014-12-01 17:34:01 -07002403. Make sure you have the require Python pre-requisites
241
Simon Glass827e37b2014-12-01 17:34:06 -0700242Buildman uses multiprocessing, Queue, shutil, StringIO, ConfigParser and
243urllib2. These should normally be available, but if you get an error like
244this then you will need to obtain those modules:
Simon Glass34699692014-12-01 17:34:01 -0700245
246 ImportError: No module named multiprocessing
247
248
2494. Check the available toolchains
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000250
251Run this check to make sure that you have a toolchain for every architecture.
252
253$ ./tools/buildman/buildman --list-tool-chains
254Scanning for tool chains
Simon Glass17bce662016-03-12 18:50:32 -0700255 - scanning prefix '/opt/gcc-4.6.3-nolibc/x86_64-linux/bin/x86_64-linux-'
256Tool chain test: OK, arch='x86', priority 1
257 - scanning prefix '/opt/arm-eabi-4.6/bin/arm-eabi-'
258Tool chain test: OK, arch='arm', priority 1
259 - scanning path '/toolchains/gcc-4.9.0-nolibc/i386-linux'
260 - looking in '/toolchains/gcc-4.9.0-nolibc/i386-linux/.'
261 - looking in '/toolchains/gcc-4.9.0-nolibc/i386-linux/bin'
262 - found '/toolchains/gcc-4.9.0-nolibc/i386-linux/bin/i386-linux-gcc'
263 - looking in '/toolchains/gcc-4.9.0-nolibc/i386-linux/usr/bin'
264Tool chain test: OK, arch='i386', priority 4
265 - scanning path '/toolchains/gcc-4.9.0-nolibc/aarch64-linux'
266 - looking in '/toolchains/gcc-4.9.0-nolibc/aarch64-linux/.'
267 - looking in '/toolchains/gcc-4.9.0-nolibc/aarch64-linux/bin'
268 - found '/toolchains/gcc-4.9.0-nolibc/aarch64-linux/bin/aarch64-linux-gcc'
269 - looking in '/toolchains/gcc-4.9.0-nolibc/aarch64-linux/usr/bin'
270Tool chain test: OK, arch='aarch64', priority 4
271 - scanning path '/toolchains/gcc-4.9.0-nolibc/microblaze-linux'
272 - looking in '/toolchains/gcc-4.9.0-nolibc/microblaze-linux/.'
273 - looking in '/toolchains/gcc-4.9.0-nolibc/microblaze-linux/bin'
274 - found '/toolchains/gcc-4.9.0-nolibc/microblaze-linux/bin/microblaze-linux-gcc'
275 - looking in '/toolchains/gcc-4.9.0-nolibc/microblaze-linux/usr/bin'
276Tool chain test: OK, arch='microblaze', priority 4
277 - scanning path '/toolchains/gcc-4.9.0-nolibc/mips64-linux'
278 - looking in '/toolchains/gcc-4.9.0-nolibc/mips64-linux/.'
279 - looking in '/toolchains/gcc-4.9.0-nolibc/mips64-linux/bin'
280 - found '/toolchains/gcc-4.9.0-nolibc/mips64-linux/bin/mips64-linux-gcc'
281 - looking in '/toolchains/gcc-4.9.0-nolibc/mips64-linux/usr/bin'
282Tool chain test: OK, arch='mips64', priority 4
283 - scanning path '/toolchains/gcc-4.9.0-nolibc/sparc64-linux'
284 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc64-linux/.'
285 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc64-linux/bin'
286 - found '/toolchains/gcc-4.9.0-nolibc/sparc64-linux/bin/sparc64-linux-gcc'
287 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc64-linux/usr/bin'
288Tool chain test: OK, arch='sparc64', priority 4
289 - scanning path '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi'
290 - looking in '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi/.'
291 - looking in '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi/bin'
292 - found '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-gcc'
293 - looking in '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi/usr/bin'
294Tool chain test: OK, arch='arm', priority 3
295Toolchain '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-gcc' at priority 3 will be ignored because another toolchain for arch 'arm' has priority 1
296 - scanning path '/toolchains/gcc-4.9.0-nolibc/sparc-linux'
297 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc-linux/.'
298 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc-linux/bin'
299 - found '/toolchains/gcc-4.9.0-nolibc/sparc-linux/bin/sparc-linux-gcc'
300 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc-linux/usr/bin'
301Tool chain test: OK, arch='sparc', priority 4
302 - scanning path '/toolchains/gcc-4.9.0-nolibc/mips-linux'
303 - looking in '/toolchains/gcc-4.9.0-nolibc/mips-linux/.'
304 - looking in '/toolchains/gcc-4.9.0-nolibc/mips-linux/bin'
305 - found '/toolchains/gcc-4.9.0-nolibc/mips-linux/bin/mips-linux-gcc'
306 - looking in '/toolchains/gcc-4.9.0-nolibc/mips-linux/usr/bin'
307Tool chain test: OK, arch='mips', priority 4
308 - scanning path '/toolchains/gcc-4.9.0-nolibc/x86_64-linux'
309 - looking in '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/.'
310 - looking in '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/bin'
311 - found '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/bin/x86_64-linux-gcc'
312 - found '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/bin/x86_64-linux-x86_64-linux-gcc'
313 - looking in '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/usr/bin'
314Tool chain test: OK, arch='x86_64', priority 4
315Tool chain test: OK, arch='x86_64', priority 4
316Toolchain '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/bin/x86_64-linux-x86_64-linux-gcc' at priority 4 will be ignored because another toolchain for arch 'x86_64' has priority 4
317 - scanning path '/toolchains/gcc-4.9.0-nolibc/m68k-linux'
318 - looking in '/toolchains/gcc-4.9.0-nolibc/m68k-linux/.'
319 - looking in '/toolchains/gcc-4.9.0-nolibc/m68k-linux/bin'
320 - found '/toolchains/gcc-4.9.0-nolibc/m68k-linux/bin/m68k-linux-gcc'
321 - looking in '/toolchains/gcc-4.9.0-nolibc/m68k-linux/usr/bin'
322Tool chain test: OK, arch='m68k', priority 4
323 - scanning path '/toolchains/gcc-4.9.0-nolibc/powerpc-linux'
324 - looking in '/toolchains/gcc-4.9.0-nolibc/powerpc-linux/.'
325 - looking in '/toolchains/gcc-4.9.0-nolibc/powerpc-linux/bin'
326 - found '/toolchains/gcc-4.9.0-nolibc/powerpc-linux/bin/powerpc-linux-gcc'
327 - looking in '/toolchains/gcc-4.9.0-nolibc/powerpc-linux/usr/bin'
328Tool chain test: OK, arch='powerpc', priority 4
329 - scanning path '/toolchains/gcc-4.6.3-nolibc/bfin-uclinux'
330 - looking in '/toolchains/gcc-4.6.3-nolibc/bfin-uclinux/.'
331 - looking in '/toolchains/gcc-4.6.3-nolibc/bfin-uclinux/bin'
332 - found '/toolchains/gcc-4.6.3-nolibc/bfin-uclinux/bin/bfin-uclinux-gcc'
333 - looking in '/toolchains/gcc-4.6.3-nolibc/bfin-uclinux/usr/bin'
334Tool chain test: OK, arch='bfin', priority 6
335 - scanning path '/toolchains/gcc-4.6.3-nolibc/sparc-linux'
336 - looking in '/toolchains/gcc-4.6.3-nolibc/sparc-linux/.'
337 - looking in '/toolchains/gcc-4.6.3-nolibc/sparc-linux/bin'
338 - found '/toolchains/gcc-4.6.3-nolibc/sparc-linux/bin/sparc-linux-gcc'
339 - looking in '/toolchains/gcc-4.6.3-nolibc/sparc-linux/usr/bin'
340Tool chain test: OK, arch='sparc', priority 4
341Toolchain '/toolchains/gcc-4.6.3-nolibc/sparc-linux/bin/sparc-linux-gcc' at priority 4 will be ignored because another toolchain for arch 'sparc' has priority 4
342 - scanning path '/toolchains/gcc-4.6.3-nolibc/mips-linux'
343 - looking in '/toolchains/gcc-4.6.3-nolibc/mips-linux/.'
344 - looking in '/toolchains/gcc-4.6.3-nolibc/mips-linux/bin'
345 - found '/toolchains/gcc-4.6.3-nolibc/mips-linux/bin/mips-linux-gcc'
346 - looking in '/toolchains/gcc-4.6.3-nolibc/mips-linux/usr/bin'
347Tool chain test: OK, arch='mips', priority 4
348Toolchain '/toolchains/gcc-4.6.3-nolibc/mips-linux/bin/mips-linux-gcc' at priority 4 will be ignored because another toolchain for arch 'mips' has priority 4
349 - scanning path '/toolchains/gcc-4.6.3-nolibc/m68k-linux'
350 - looking in '/toolchains/gcc-4.6.3-nolibc/m68k-linux/.'
351 - looking in '/toolchains/gcc-4.6.3-nolibc/m68k-linux/bin'
352 - found '/toolchains/gcc-4.6.3-nolibc/m68k-linux/bin/m68k-linux-gcc'
353 - looking in '/toolchains/gcc-4.6.3-nolibc/m68k-linux/usr/bin'
354Tool chain test: OK, arch='m68k', priority 4
355Toolchain '/toolchains/gcc-4.6.3-nolibc/m68k-linux/bin/m68k-linux-gcc' at priority 4 will be ignored because another toolchain for arch 'm68k' has priority 4
356 - scanning path '/toolchains/gcc-4.6.3-nolibc/powerpc-linux'
357 - looking in '/toolchains/gcc-4.6.3-nolibc/powerpc-linux/.'
358 - looking in '/toolchains/gcc-4.6.3-nolibc/powerpc-linux/bin'
359 - found '/toolchains/gcc-4.6.3-nolibc/powerpc-linux/bin/powerpc-linux-gcc'
360 - looking in '/toolchains/gcc-4.6.3-nolibc/powerpc-linux/usr/bin'
361Tool chain test: OK, arch='powerpc', priority 4
362Tool chain test: OK, arch='or32', priority 4
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000363 - scanning path '/'
364 - looking in '/.'
365 - looking in '/bin'
366 - looking in '/usr/bin'
Simon Glass17bce662016-03-12 18:50:32 -0700367 - found '/usr/bin/i586-mingw32msvc-gcc'
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000368 - found '/usr/bin/c89-gcc'
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000369 - found '/usr/bin/x86_64-linux-gnu-gcc'
Simon Glass17bce662016-03-12 18:50:32 -0700370 - found '/usr/bin/gcc'
371 - found '/usr/bin/c99-gcc'
372 - found '/usr/bin/arm-linux-gnueabi-gcc'
373 - found '/usr/bin/aarch64-linux-gnu-gcc'
374 - found '/usr/bin/winegcc'
375 - found '/usr/bin/arm-linux-gnueabihf-gcc'
376Tool chain test: OK, arch='i586', priority 11
377Tool chain test: OK, arch='c89', priority 11
378Tool chain test: OK, arch='x86_64', priority 4
379Toolchain '/usr/bin/x86_64-linux-gnu-gcc' at priority 4 will be ignored because another toolchain for arch 'x86_64' has priority 4
380Tool chain test: OK, arch='sandbox', priority 11
381Tool chain test: OK, arch='c99', priority 11
382Tool chain test: OK, arch='arm', priority 4
383Toolchain '/usr/bin/arm-linux-gnueabi-gcc' at priority 4 will be ignored because another toolchain for arch 'arm' has priority 1
384Tool chain test: OK, arch='aarch64', priority 4
385Toolchain '/usr/bin/aarch64-linux-gnu-gcc' at priority 4 will be ignored because another toolchain for arch 'aarch64' has priority 4
386Tool chain test: OK, arch='sandbox', priority 11
387Toolchain '/usr/bin/winegcc' at priority 11 will be ignored because another toolchain for arch 'sandbox' has priority 11
388Tool chain test: OK, arch='arm', priority 4
389Toolchain '/usr/bin/arm-linux-gnueabihf-gcc' at priority 4 will be ignored because another toolchain for arch 'arm' has priority 1
390List of available toolchains (34):
391aarch64 : /toolchains/gcc-4.9.0-nolibc/aarch64-linux/bin/aarch64-linux-gcc
392alpha : /toolchains/gcc-4.9.0-nolibc/alpha-linux/bin/alpha-linux-gcc
393am33_2.0 : /toolchains/gcc-4.9.0-nolibc/am33_2.0-linux/bin/am33_2.0-linux-gcc
394arm : /opt/arm-eabi-4.6/bin/arm-eabi-gcc
Simon Glass17bce662016-03-12 18:50:32 -0700395bfin : /toolchains/gcc-4.6.3-nolibc/bfin-uclinux/bin/bfin-uclinux-gcc
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000396c89 : /usr/bin/c89-gcc
397c99 : /usr/bin/c99-gcc
Simon Glass17bce662016-03-12 18:50:32 -0700398frv : /toolchains/gcc-4.9.0-nolibc/frv-linux/bin/frv-linux-gcc
399h8300 : /toolchains/gcc-4.9.0-nolibc/h8300-elf/bin/h8300-elf-gcc
400hppa : /toolchains/gcc-4.9.0-nolibc/hppa-linux/bin/hppa-linux-gcc
401hppa64 : /toolchains/gcc-4.9.0-nolibc/hppa64-linux/bin/hppa64-linux-gcc
402i386 : /toolchains/gcc-4.9.0-nolibc/i386-linux/bin/i386-linux-gcc
403i586 : /usr/bin/i586-mingw32msvc-gcc
404ia64 : /toolchains/gcc-4.9.0-nolibc/ia64-linux/bin/ia64-linux-gcc
405m32r : /toolchains/gcc-4.9.0-nolibc/m32r-linux/bin/m32r-linux-gcc
406m68k : /toolchains/gcc-4.9.0-nolibc/m68k-linux/bin/m68k-linux-gcc
407microblaze: /toolchains/gcc-4.9.0-nolibc/microblaze-linux/bin/microblaze-linux-gcc
408mips : /toolchains/gcc-4.9.0-nolibc/mips-linux/bin/mips-linux-gcc
409mips64 : /toolchains/gcc-4.9.0-nolibc/mips64-linux/bin/mips64-linux-gcc
410or32 : /toolchains/gcc-4.5.1-nolibc/or32-linux/bin/or32-linux-gcc
411powerpc : /toolchains/gcc-4.9.0-nolibc/powerpc-linux/bin/powerpc-linux-gcc
412powerpc64 : /toolchains/gcc-4.9.0-nolibc/powerpc64-linux/bin/powerpc64-linux-gcc
413ppc64le : /toolchains/gcc-4.9.0-nolibc/ppc64le-linux/bin/ppc64le-linux-gcc
414s390x : /toolchains/gcc-4.9.0-nolibc/s390x-linux/bin/s390x-linux-gcc
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000415sandbox : /usr/bin/gcc
Simon Glass17bce662016-03-12 18:50:32 -0700416sh4 : /toolchains/gcc-4.6.3-nolibc/sh4-linux/bin/sh4-linux-gcc
417sparc : /toolchains/gcc-4.9.0-nolibc/sparc-linux/bin/sparc-linux-gcc
418sparc64 : /toolchains/gcc-4.9.0-nolibc/sparc64-linux/bin/sparc64-linux-gcc
419tilegx : /toolchains/gcc-4.6.2-nolibc/tilegx-linux/bin/tilegx-linux-gcc
420x86 : /opt/gcc-4.6.3-nolibc/x86_64-linux/bin/x86_64-linux-gcc
421x86_64 : /toolchains/gcc-4.9.0-nolibc/x86_64-linux/bin/x86_64-linux-gcc
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000422
423
424You can see that everything is covered, even some strange ones that won't
425be used (c88 and c99). This is a feature.
426
427
Simon Glass827e37b2014-12-01 17:34:06 -07004285. Install new toolchains if needed
429
430You can download toolchains and update the [toolchain] section of the
431settings file to find them.
432
433To make this easier, buildman can automatically download and install
434toolchains from kernel.org. First list the available architectures:
435
Bin Meng9f244b22015-07-16 19:43:46 -0700436$ ./tools/buildman/buildman --fetch-arch list
Simon Glass827e37b2014-12-01 17:34:06 -0700437Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.6.3/
438Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.6.2/
439Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.5.1/
440Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.2.4/
Andy Shevchenkodaab59a2017-07-05 16:25:22 +0300441Available architectures: alpha am33_2.0 arm bfin cris crisv32 frv h8300
Simon Glass827e37b2014-12-01 17:34:06 -0700442hppa hppa64 i386 ia64 m32r m68k mips mips64 or32 powerpc powerpc64 s390x sh4
443sparc sparc64 tilegx x86_64 xtensa
444
445Then pick one and download it:
446
Bin Meng9f244b22015-07-16 19:43:46 -0700447$ ./tools/buildman/buildman --fetch-arch or32
Simon Glass827e37b2014-12-01 17:34:06 -0700448Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.6.3/
449Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.6.2/
450Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.5.1/
451Downloading: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.5.1//x86_64-gcc-4.5.1-nolibc_or32-linux.tar.xz
452Unpacking to: /home/sjg/.buildman-toolchains
453Testing
454 - looking in '/home/sjg/.buildman-toolchains/gcc-4.5.1-nolibc/or32-linux/.'
455 - looking in '/home/sjg/.buildman-toolchains/gcc-4.5.1-nolibc/or32-linux/bin'
456 - found '/home/sjg/.buildman-toolchains/gcc-4.5.1-nolibc/or32-linux/bin/or32-linux-gcc'
457Tool chain test: OK
458
Thomas Chou89515232015-11-12 09:29:09 +0800459Or download them all from kernel.org and move them to /toolchains directory,
460
Simon Glass8ea42102016-07-27 20:32:58 -0600461$ ./tools/buildman/buildman --fetch-arch all
Thomas Chou89515232015-11-12 09:29:09 +0800462$ sudo mkdir -p /toolchains
463$ sudo mv ~/.buildman-toolchains/*/* /toolchains/
464
465For those not available from kernel.org, download from the following links.
466
467arc: https://github.com/foss-for-synopsys-dwc-arc-processors/toolchain/releases/
Alexey Brodkina55bed12017-01-26 17:25:59 +0300468 download/arc-2016.09-release/arc_gnu_2016.09_prebuilt_uclibc_le_archs_linux_install.tar.gz
Thomas Chou89515232015-11-12 09:29:09 +0800469blackfin: http://sourceforge.net/projects/adi-toolchain/files/
470 blackfin-toolchain-elf-gcc-4.5-2014R1_45-RC2.x86_64.tar.bz2
471nds32: http://osdk.andestech.com/packages/
472 nds32le-linux-glibc-v1.tgz
473nios2: http://sourcery.mentor.com/public/gnu_toolchain/nios2-linux-gnu/
474 sourceryg++-2015.11-27-nios2-linux-gnu-i686-pc-linux-gnu.tar.bz2
475sh: http://sourcery.mentor.com/public/gnu_toolchain/sh-linux-gnu/
476 renesas-4.4-200-sh-linux-gnu-i686-pc-linux-gnu.tar.bz2
477
Simon Glass8ea42102016-07-27 20:32:58 -0600478Note openrisc kernel.org toolchain is out of date. Download the latest one from
479http://opencores.org/or1k/OpenRISC_GNU_tool_chain#Prebuilt_versions - eg:
Bin Menge8aebc42016-02-21 21:18:02 -0800480ftp://ocuser:ocuser@openrisc.opencores.org/toolchain/gcc-or1k-elf-4.8.1-x86.tar.bz2.
481
Simon Glass827e37b2014-12-01 17:34:06 -0700482Buildman should now be set up to use your new toolchain.
483
484At the time of writing, U-Boot has these architectures:
485
Andy Shevchenkodaab59a2017-07-05 16:25:22 +0300486 arc, arm, blackfin, m68k, microblaze, mips, nds32, nios2, openrisc
Simon Glass827e37b2014-12-01 17:34:06 -0700487 powerpc, sandbox, sh, sparc, x86
488
Michal Simek12462312015-04-20 11:46:24 +0200489Of these, only arc and nds32 are not available at kernel.org..
Simon Glass827e37b2014-12-01 17:34:06 -0700490
491
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000492How to run it
493=============
494
495First do a dry run using the -n flag: (replace <branch> with a real, local
496branch with a valid upstream)
497
498$ ./tools/buildman/buildman -b <branch> -n
499
500If it can't detect the upstream branch, try checking out the branch, and
Simon Glass2a9e2c62014-12-01 17:33:54 -0700501doing something like 'git branch --set-upstream-to upstream/master'
502or something similar. Buildman will try to guess a suitable upstream branch
503if it can't find one (you will see a message like" Guessing upstream as ...).
Simon Glassea09fb52020-04-09 15:08:50 -0600504You can also use the -c option to manually specify the number of commits to
505build.
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000506
Simon Glasscec83c32014-08-09 15:32:57 -0600507As an example:
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000508
509Dry run, so not doing much. But I would do this:
510
511Building 18 commits for 1059 boards (4 threads, 1 job per thread)
512Build directory: ../lcd9b
513 5bb3505 Merge branch 'master' of git://git.denx.de/u-boot-arm
514 c18f1b4 tegra: Use const for pinmux_config_pingroup/table()
515 2f043ae tegra: Add display support to funcmux
516 e349900 tegra: fdt: Add pwm binding and node
517 424a5f0 tegra: fdt: Add LCD definitions for Tegra
518 0636ccf tegra: Add support for PWM
519 a994fe7 tegra: Add SOC support for display/lcd
520 fcd7350 tegra: Add LCD driver
521 4d46e9d tegra: Add LCD support to Nvidia boards
522 991bd48 arm: Add control over cachability of memory regions
523 54e8019 lcd: Add CONFIG_LCD_ALIGNMENT to select frame buffer alignment
524 d92aff7 lcd: Add support for flushing LCD fb from dcache after update
525 dbd0677 tegra: Align LCD frame buffer to section boundary
526 0cff9b8 tegra: Support control of cache settings for LCD
527 9c56900 tegra: fdt: Add LCD definitions for Seaboard
528 5cc29db lcd: Add CONFIG_CONSOLE_SCROLL_LINES option to speed console
529 cac5a23 tegra: Enable display/lcd support on Seaboard
530 49ff541 wip
531
532Total boards to build for each commit: 1059
533
534This shows that it will build all 1059 boards, using 4 threads (because
535we have a 4-core CPU). Each thread will run with -j1, meaning that each
536make job will use a single CPU. The list of commits to be built helps you
537confirm that things look about right. Notice that buildman has chosen a
538'base' directory for you, immediately above your source tree.
539
540Buildman works entirely inside the base directory, here ../lcd9b,
541creating a working directory for each thread, and creating output
542directories for each commit and board.
543
544
545Suggested Workflow
546==================
547
548To run the build for real, take off the -n:
549
550$ ./tools/buildman/buildman -b <branch>
551
552Buildman will set up some working directories, and get started. After a
553minute or so it will settle down to a steady pace, with a display like this:
554
555Building 18 commits for 1059 boards (4 threads, 1 job per thread)
Simon Glassea09fb52020-04-09 15:08:50 -0600556 528 36 124 /19062 -18374 1:13:30 : SIMPC8313_SP
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000557
558This means that it is building 19062 board/commit combinations. So far it
Simon Glasscec83c32014-08-09 15:32:57 -0600559has managed to successfully build 528. Another 36 have built with warnings,
Simon Glassea09fb52020-04-09 15:08:50 -0600560and 124 more didn't build at all. It has 18374 builds left to complete.
561Buildman expects to complete the process in around an hour and a quarter.
562Use this time to buy a faster computer.
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000563
564
565To find out how the build went, ask for a summary with -s. You can do this
Dirk Behme3e1ded12014-12-23 07:41:26 +0100566either before the build completes (presumably in another terminal) or
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000567afterwards. Let's work through an example of how this is used:
568
569$ ./tools/buildman/buildman -b lcd9b -s
570...
57101: Merge branch 'master' of git://git.denx.de/u-boot-arm
572 powerpc: + galaxy5200_LOWBOOT
57302: tegra: Use const for pinmux_config_pingroup/table()
57403: tegra: Add display support to funcmux
57504: tegra: fdt: Add pwm binding and node
57605: tegra: fdt: Add LCD definitions for Tegra
57706: tegra: Add support for PWM
57807: tegra: Add SOC support for display/lcd
57908: tegra: Add LCD driver
58009: tegra: Add LCD support to Nvidia boards
58110: arm: Add control over cachability of memory regions
58211: lcd: Add CONFIG_LCD_ALIGNMENT to select frame buffer alignment
58312: lcd: Add support for flushing LCD fb from dcache after update
584 arm: + lubbock
58513: tegra: Align LCD frame buffer to section boundary
58614: tegra: Support control of cache settings for LCD
58715: tegra: fdt: Add LCD definitions for Seaboard
58816: lcd: Add CONFIG_CONSOLE_SCROLL_LINES option to speed console
58917: tegra: Enable display/lcd support on Seaboard
59018: wip
591
592This shows which commits have succeeded and which have failed. In this case
593the build is still in progress so many boards are not built yet (use -u to
Simon Glassea09fb52020-04-09 15:08:50 -0600594see which ones). But already we can see a few failures. The galaxy5200_LOWBOOT
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000595never builds correctly. This could be a problem with our toolchain, or it
596could be a bug in the upstream. The good news is that we probably don't need
Simon Glass8ea42102016-07-27 20:32:58 -0600597to blame our commits. The bad news is that our commits are not tested on that
598board.
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000599
Simon Glassea09fb52020-04-09 15:08:50 -0600600Commit 12 broke lubbock. That's what the '+ lubbock', in red, means. The
601failure is never fixed by a later commit, or you would see lubbock again, in
602green, without the +.
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000603
604To see the actual error:
605
Simon Glassea09fb52020-04-09 15:08:50 -0600606$ ./tools/buildman/buildman -b <branch> -se
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000607...
60812: lcd: Add support for flushing LCD fb from dcache after update
609 arm: + lubbock
610+common/libcommon.o: In function `lcd_sync':
Simon Glassea09fb52020-04-09 15:08:50 -0600611+common/lcd.c:120: undefined reference to `flush_dcache_range'
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000612+arm-none-linux-gnueabi-ld: BFD (Sourcery G++ Lite 2010q1-202) 2.19.51.20090709 assertion fail /scratch/julian/2010q1-release-linux-lite/obj/binutils-src-2010q1-202-arm-none-linux-gnueabi-i686-pc-linux-gnu/bfd/elf32-arm.c:12572
Simon Glassea09fb52020-04-09 15:08:50 -0600613+make: *** [build/u-boot] Error 139
Simon Glassfc3fe1c2013-04-03 11:07:16 +000061413: tegra: Align LCD frame buffer to section boundary
61514: tegra: Support control of cache settings for LCD
61615: tegra: fdt: Add LCD definitions for Seaboard
61716: lcd: Add CONFIG_CONSOLE_SCROLL_LINES option to speed console
Simon Glassea09fb52020-04-09 15:08:50 -0600618-common/lcd.c:120: undefined reference to `flush_dcache_range'
619+common/lcd.c:125: undefined reference to `flush_dcache_range'
Simon Glassfc3fe1c2013-04-03 11:07:16 +000062017: tegra: Enable display/lcd support on Seaboard
62118: wip
622
623So the problem is in lcd.c, due to missing cache operations. This information
624should be enough to work out what that commit is doing to break these
625boards. (In this case pxa did not have cache operations defined).
626
Simon Glassea09fb52020-04-09 15:08:50 -0600627Note that if there were other boards with errors, the above command would
628show their errors also. Each line is shown only once. So if lubbock and snow
629produce the same error, we just see:
630
63112: lcd: Add support for flushing LCD fb from dcache after update
632 arm: + lubbock snow
633+common/libcommon.o: In function `lcd_sync':
634+common/lcd.c:120: undefined reference to `flush_dcache_range'
635+arm-none-linux-gnueabi-ld: BFD (Sourcery G++ Lite 2010q1-202) 2.19.51.20090709 assertion fail /scratch/julian/2010q1-release-linux-lite/obj/binutils-src-2010q1-202-arm-none-linux-gnueabi-i686-pc-linux-gnu/bfd/elf32-arm.c:12572
636+make: *** [build/u-boot] Error 139
637
638But if you did want to see just the errors for lubbock, use:
639
640$ ./tools/buildman/buildman -b <branch> -se lubbock
641
Simon Glass8ea42102016-07-27 20:32:58 -0600642If you see error lines marked with '-', that means that the errors were fixed
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000643by that commit. Sometimes commits can be in the wrong order, so that a
644breakage is introduced for a few commits and fixed by later commits. This
645shows up clearly with buildman. You can then reorder the commits and try
646again.
647
Simon Glass8ea42102016-07-27 20:32:58 -0600648At commit 16, the error moves: you can see that the old error at line 120
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000649is fixed, but there is a new one at line 126. This is probably only because
Dirk Behme3e1ded12014-12-23 07:41:26 +0100650we added some code and moved the broken line further down the file.
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000651
Simon Glassea09fb52020-04-09 15:08:50 -0600652As mentioned, if many boards have the same error, then -e will display the
653error only once. This makes the output as concise as possible. To see which
654boards have each error, use -l. So it is safe to omit the board name - you
655will not get lots of repeated output for every board.
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000656
Simon Glasse30965d2014-08-28 09:43:44 -0600657Buildman tries to distinguish warnings from errors, and shows warning lines
Simon Glassea09fb52020-04-09 15:08:50 -0600658separately with a 'w' prefix. Warnings introduced show as yellow. Warnings
659fixed show as cyan.
Simon Glasse30965d2014-08-28 09:43:44 -0600660
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000661The full build output in this case is available in:
662
663../lcd9b/12_of_18_gd92aff7_lcd--Add-support-for/lubbock/
664
665 done: Indicates the build was done, and holds the return code from make.
666 This is 0 for a good build, typically 2 for a failure.
667
668 err: Output from stderr, if any. Errors and warnings appear here.
669
670 log: Output from stdout. Normally there isn't any since buildman runs
Simon Glassc81d0d22016-03-12 18:50:33 -0700671 in silent mode. Use -V to force a verbose build (this passes V=1
672 to 'make')
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000673
674 toolchain: Shows information about the toolchain used for the build.
675
676 sizes: Shows image size information.
677
Simon Glass8ea42102016-07-27 20:32:58 -0600678It is possible to get the build binary output there also. Use the -k option
679for this. In that case you will also see some output files, like:
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000680
681 System.map toolchain u-boot u-boot.bin u-boot.map autoconf.mk
682 (also SPL versions u-boot-spl and u-boot-spl.bin if available)
683
684
685Checking Image Sizes
686====================
687
688A key requirement for U-Boot is that you keep code/data size to a minimum.
689Where a new feature increases this noticeably it should normally be put
Simon Glass8ea42102016-07-27 20:32:58 -0600690behind a CONFIG flag so that boards can leave it disabled and keep the image
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000691size more or less the same with each new release.
692
693To check the impact of your commits on image size, use -S. For example:
694
695$ ./tools/buildman/buildman -b us-x86 -sS
696Summary of 10 commits for 1066 boards (4 threads, 1 job per thread)
69701: MAKEALL: add support for per architecture toolchains
69802: x86: Add function to get top of usable ram
699 x86: (for 1/3 boards) text -272.0 rodata +41.0
70003: x86: Add basic cache operations
70104: x86: Permit bootstage and timer data to be used prior to relocation
702 x86: (for 1/3 boards) data +16.0
70305: x86: Add an __end symbol to signal the end of the U-Boot binary
704 x86: (for 1/3 boards) text +76.0
70506: x86: Rearrange the output input to remove BSS
706 x86: (for 1/3 boards) bss -2140.0
70707: x86: Support relocation of FDT on start-up
708 x86: + coreboot-x86
70908: x86: Add error checking to x86 relocation code
71009: x86: Adjust link device tree include file
71110: x86: Enable CONFIG_OF_CONTROL on coreboot
712
713
714You can see that image size only changed on x86, which is good because this
715series is not supposed to change any other board. From commit 7 onwards the
716build fails so we don't get code size numbers. The numbers are fractional
717because they are an average of all boards for that architecture. The
718intention is to allow you to quickly find image size problems introduced by
719your commits.
720
721Note that the 'text' region and 'rodata' are split out. You should add the
722two together to get the total read-only size (reported as the first column
723in the output from binutil's 'size' utility).
724
725A useful option is --step which lets you skip some commits. For example
726--step 2 will show the image sizes for only every 2nd commit (so it will
727compare the image sizes of the 1st, 3rd, 5th... commits). You can also use
728--step 0 which will compare only the first and last commits. This is useful
Simon Glass8ea42102016-07-27 20:32:58 -0600729for an overview of how your entire series affects code size. It will build
730only the upstream commit and your final branch commit.
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000731
732You can also use -d to see a detailed size breakdown for each board. This
733list is sorted in order from largest growth to largest reduction.
734
Simon Glass8ea42102016-07-27 20:32:58 -0600735It is even possible to go a little further with the -B option (--bloat). This
Simon Glasscec83c32014-08-09 15:32:57 -0600736shows where U-Boot has bloated, breaking the size change down to the function
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000737level. Example output is below:
738
739$ ./tools/buildman/buildman -b us-mem4 -sSdB
740...
74119: Roll crc32 into hash infrastructure
742 arm: (for 10/10 boards) all -143.4 bss +1.2 data -4.8 rodata -48.2 text -91.6
743 paz00 : all +23 bss -4 rodata -29 text +56
744 u-boot: add: 1/0, grow: 3/-2 bytes: 168/-104 (64)
745 function old new delta
746 hash_command 80 160 +80
747 crc32_wd_buf - 56 +56
748 ext4fs_read_file 540 568 +28
749 insert_var_value_sub 688 692 +4
750 run_list_real 1996 1992 -4
751 do_mem_crc 168 68 -100
752 trimslice : all -9 bss +16 rodata -29 text +4
753 u-boot: add: 1/0, grow: 1/-3 bytes: 136/-124 (12)
754 function old new delta
755 hash_command 80 160 +80
756 crc32_wd_buf - 56 +56
757 ext4fs_iterate_dir 672 668 -4
758 ext4fs_read_file 568 548 -20
759 do_mem_crc 168 68 -100
760 whistler : all -9 bss +16 rodata -29 text +4
761 u-boot: add: 1/0, grow: 1/-3 bytes: 136/-124 (12)
762 function old new delta
763 hash_command 80 160 +80
764 crc32_wd_buf - 56 +56
765 ext4fs_iterate_dir 672 668 -4
766 ext4fs_read_file 568 548 -20
767 do_mem_crc 168 68 -100
768 seaboard : all -9 bss -28 rodata -29 text +48
769 u-boot: add: 1/0, grow: 3/-2 bytes: 160/-104 (56)
770 function old new delta
771 hash_command 80 160 +80
772 crc32_wd_buf - 56 +56
773 ext4fs_read_file 548 568 +20
774 run_list_real 1996 2000 +4
775 do_nandboot 760 756 -4
776 do_mem_crc 168 68 -100
Marcel Ziswilere57c6e52015-03-26 01:31:54 +0100777 colibri_t20 : all -9 rodata -29 text +20
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000778 u-boot: add: 1/0, grow: 2/-3 bytes: 140/-112 (28)
779 function old new delta
780 hash_command 80 160 +80
781 crc32_wd_buf - 56 +56
782 read_abs_bbt 204 208 +4
783 do_nandboot 760 756 -4
784 ext4fs_read_file 576 568 -8
785 do_mem_crc 168 68 -100
786 ventana : all -37 bss -12 rodata -29 text +4
787 u-boot: add: 1/0, grow: 1/-3 bytes: 136/-124 (12)
788 function old new delta
789 hash_command 80 160 +80
790 crc32_wd_buf - 56 +56
791 ext4fs_iterate_dir 672 668 -4
792 ext4fs_read_file 568 548 -20
793 do_mem_crc 168 68 -100
794 harmony : all -37 bss -16 rodata -29 text +8
795 u-boot: add: 1/0, grow: 2/-3 bytes: 140/-124 (16)
796 function old new delta
797 hash_command 80 160 +80
798 crc32_wd_buf - 56 +56
799 nand_write_oob_syndrome 428 432 +4
800 ext4fs_iterate_dir 672 668 -4
801 ext4fs_read_file 568 548 -20
802 do_mem_crc 168 68 -100
803 medcom-wide : all -417 bss +28 data -16 rodata -93 text -336
804 u-boot: add: 1/-1, grow: 1/-2 bytes: 88/-376 (-288)
805 function old new delta
806 crc32_wd_buf - 56 +56
807 do_fat_read_at 2872 2904 +32
808 hash_algo 16 - -16
809 do_mem_crc 168 68 -100
810 hash_command 420 160 -260
811 tec : all -449 bss -4 data -16 rodata -93 text -336
812 u-boot: add: 1/-1, grow: 1/-2 bytes: 88/-376 (-288)
813 function old new delta
814 crc32_wd_buf - 56 +56
815 do_fat_read_at 2872 2904 +32
816 hash_algo 16 - -16
817 do_mem_crc 168 68 -100
818 hash_command 420 160 -260
819 plutux : all -481 bss +16 data -16 rodata -93 text -388
820 u-boot: add: 1/-1, grow: 1/-3 bytes: 68/-408 (-340)
821 function old new delta
822 crc32_wd_buf - 56 +56
823 do_load_serial_bin 1688 1700 +12
824 hash_algo 16 - -16
825 do_fat_read_at 2904 2872 -32
826 do_mem_crc 168 68 -100
827 hash_command 420 160 -260
828 powerpc: (for 5/5 boards) all +37.4 data -3.2 rodata -41.8 text +82.4
829 MPC8610HPCD : all +55 rodata -29 text +84
830 u-boot: add: 1/0, grow: 0/-1 bytes: 176/-96 (80)
831 function old new delta
832 hash_command - 176 +176
833 do_mem_crc 184 88 -96
834 MPC8641HPCN : all +55 rodata -29 text +84
835 u-boot: add: 1/0, grow: 0/-1 bytes: 176/-96 (80)
836 function old new delta
837 hash_command - 176 +176
838 do_mem_crc 184 88 -96
839 MPC8641HPCN_36BIT: all +55 rodata -29 text +84
840 u-boot: add: 1/0, grow: 0/-1 bytes: 176/-96 (80)
841 function old new delta
842 hash_command - 176 +176
843 do_mem_crc 184 88 -96
844 sbc8641d : all +55 rodata -29 text +84
845 u-boot: add: 1/0, grow: 0/-1 bytes: 176/-96 (80)
846 function old new delta
847 hash_command - 176 +176
848 do_mem_crc 184 88 -96
849 xpedite517x : all -33 data -16 rodata -93 text +76
850 u-boot: add: 1/-1, grow: 0/-1 bytes: 176/-112 (64)
851 function old new delta
852 hash_command - 176 +176
853 hash_algo 16 - -16
854 do_mem_crc 184 88 -96
855...
856
857
Simon Glass8ea42102016-07-27 20:32:58 -0600858This shows that commit 19 has reduced codesize for arm slightly and increased
859it for powerpc. This increase was offset in by reductions in rodata and
860data/bss.
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000861
Dirk Behme3e1ded12014-12-23 07:41:26 +0100862Shown below the summary lines are the sizes for each board. Below each board
863are the sizes for each function. This information starts with:
Simon Glassfc3fe1c2013-04-03 11:07:16 +0000864
865 add - number of functions added / removed
866 grow - number of functions which grew / shrunk
867 bytes - number of bytes of code added to / removed from all functions,
868 plus the total byte change in brackets
869
870The change seems to be that hash_command() has increased by more than the
871do_mem_crc() function has decreased. The function sizes typically add up to
872roughly the text area size, but note that every read-only section except
873rodata is included in 'text', so the function total does not exactly
874correspond.
875
876It is common when refactoring code for the rodata to decrease as the text size
877increases, and vice versa.
878
879
Simon Glass62005342014-12-01 17:34:02 -0700880The .buildman file
881==================
Simon Glass4281ad82013-09-23 17:35:17 -0600882
Simon Glass62005342014-12-01 17:34:02 -0700883The .buildman file provides information about the available toolchains and
884also allows build flags to be passed to 'make'. It consists of several
885sections, with the section name in square brackets. Within each section are
886a set of (tag, value) pairs.
Simon Glass4281ad82013-09-23 17:35:17 -0600887
Simon Glass62005342014-12-01 17:34:02 -0700888'[toolchain]' section
Simon Glass4281ad82013-09-23 17:35:17 -0600889
Simon Glass62005342014-12-01 17:34:02 -0700890 This lists the available toolchains. The tag here doesn't matter, but
891 make sure it is unique. The value is the path to the toolchain. Buildman
892 will look in that path for a file ending in 'gcc'. It will then execute
893 it to check that it is a C compiler, passing only the --version flag to
894 it. If the return code is 0, buildman assumes that it is a valid C
895 compiler. It uses the first part of the name as the architecture and
896 strips off the last part when setting the CROSS_COMPILE environment
897 variable (parts are delimited with a hyphen).
Simon Glass4281ad82013-09-23 17:35:17 -0600898
Simon Glass62005342014-12-01 17:34:02 -0700899 For example powerpc-linux-gcc will be noted as a toolchain for 'powerpc'
900 and CROSS_COMPILE will be set to powerpc-linux- when using it.
901
902'[toolchain-alias]' section
903
904 This converts toolchain architecture names to U-Boot names. For example,
905 if an x86 toolchains is called i386-linux-gcc it will not normally be
Simon Glass9b83bfd2014-12-01 17:34:05 -0700906 used for architecture 'x86'. Adding 'x86: i386 x86_64' to this section
907 will tell buildman that the i386 and x86_64 toolchains can be used for
908 the x86 architecture.
Simon Glass62005342014-12-01 17:34:02 -0700909
910'[make-flags]' section
911
912 U-Boot's build system supports a few flags (such as BUILD_TAG) which
913 affect the build product. These flags can be specified in the buildman
914 settings file. They can also be useful when building U-Boot against other
915 open source software.
916
917 [make-flags]
918 at91-boards=ENABLE_AT91_TEST=1
919 snapper9260=${at91-boards} BUILD_TAG=442
920 snapper9g45=${at91-boards} BUILD_TAG=443
921
922 This will use 'make ENABLE_AT91_TEST=1 BUILD_TAG=442' for snapper9260
923 and 'make ENABLE_AT91_TEST=1 BUILD_TAG=443' for snapper9g45. A special
924 variable ${target} is available to access the target name (snapper9260
925 and snapper9g20 in this case). Variables are resolved recursively. Note
926 that variables can only contain the characters A-Z, a-z, 0-9, hyphen (-)
927 and underscore (_).
928
929 It is expected that any variables added are dealt with in U-Boot's
930 config.mk file and documented in the README.
931
932 Note that you can pass ad-hoc options to the build using environment
933 variables, for example:
934
935 SOME_OPTION=1234 ./tools/buildman/buildman my_board
Simon Glass4281ad82013-09-23 17:35:17 -0600936
937
Simon Glasse5a0e5d2014-08-09 15:33:03 -0600938Quick Sanity Check
939==================
940
941If you have made changes and want to do a quick sanity check of the
Simon Glass1d8104f2014-10-16 01:05:56 -0600942currently checked-out source, run buildman without the -b flag. This will
943build the selected boards and display build status as it runs (i.e. -v is
944enabled automatically). Use -e to see errors/warnings as well.
Simon Glasse5a0e5d2014-08-09 15:33:03 -0600945
946
Simon Glass5abab202014-12-01 17:33:57 -0700947Building Ranges
948===============
949
950You can build a range of commits by specifying a range instead of a branch
951when using the -b flag. For example:
952
953 upstream/master..us-buildman
954
955will build commits in us-buildman that are not in upstream/master.
956
957
Stephen Warrenf79f1e02016-04-11 10:48:44 -0600958Building Faster
959===============
960
961By default, buildman executes 'make mrproper' prior to building the first
962commit for each board. This causes everything to be built from scratch. If you
963trust the build system's incremental build capabilities, you can pass the -I
964flag to skip the 'make mproper' invocation, which will reduce the amount of
965work 'make' does, and hence speed up the build. This flag will speed up any
966buildman invocation, since it reduces the amount of work done on any build.
967
968One possible application of buildman is as part of a continual edit, build,
969edit, build, ... cycle; repeatedly applying buildman to the same change or
970series of changes while making small incremental modifications to the source
971each time. This provides quick feedback regarding the correctness of recent
972modifications. In this scenario, buildman's default choice of build directory
973causes more build work to be performed than strictly necessary.
974
975By default, each buildman thread uses a single directory for all builds. When a
976thread builds multiple boards, the configuration built in this directory will
977cycle through various different configurations, one per board built by the
978thread. Variations in the configuration will force a rebuild of affected source
979files when a thread switches between boards. Ideally, such buildman-induced
980rebuilds would not happen, thus allowing the build to operate as efficiently as
981the build system and source changes allow. buildman's -P flag may be used to
982enable this; -P causes each board to be built in a separate (board-specific)
983directory, thus avoiding any buildman-induced configuration changes in any
984build directory.
985
986U-Boot's build system embeds information such as a build timestamp into the
987final binary. This information varies each time U-Boot is built. This causes
988various files to be rebuilt even if no source changes are made, which in turn
989requires that the final U-Boot binary be re-linked. This unnecessary work can
990be avoided by turning off the timestamp feature. This can be achieved by
991setting the SOURCE_DATE_EPOCH environment variable to 0.
992
993Combining all of these options together yields the command-line shown below.
994This will provide the quickest possible feedback regarding the current content
995of the source tree, thus allowing rapid tested evolution of the code.
996
997 SOURCE_DATE_EPOCH=0 ./tools/buildman/buildman -I -P tegra
998
999
Simon Glass94d2ebe2016-11-13 14:25:52 -07001000Checking configuration
1001======================
1002
1003A common requirement when converting CONFIG options to Kconfig is to check
1004that the effective configuration has not changed due to the conversion.
1005Buildman supports this with the -K option, used after a build. This shows
1006differences in effective configuration between one commit and the next.
1007
1008For example:
1009
1010 $ buildman -b kc4 -sK
1011 ...
1012 43: Convert CONFIG_SPL_USBETH_SUPPORT to Kconfig
1013 arm:
1014 + u-boot.cfg: CONFIG_SPL_ENV_SUPPORT=1 CONFIG_SPL_NET_SUPPORT=1
1015 + u-boot-spl.cfg: CONFIG_SPL_MMC_SUPPORT=1 CONFIG_SPL_NAND_SUPPORT=1
1016 + all: CONFIG_SPL_ENV_SUPPORT=1 CONFIG_SPL_MMC_SUPPORT=1 CONFIG_SPL_NAND_SUPPORT=1 CONFIG_SPL_NET_SUPPORT=1
1017 am335x_evm_usbspl :
1018 + u-boot.cfg: CONFIG_SPL_ENV_SUPPORT=1 CONFIG_SPL_NET_SUPPORT=1
1019 + u-boot-spl.cfg: CONFIG_SPL_MMC_SUPPORT=1 CONFIG_SPL_NAND_SUPPORT=1
1020 + all: CONFIG_SPL_ENV_SUPPORT=1 CONFIG_SPL_MMC_SUPPORT=1 CONFIG_SPL_NAND_SUPPORT=1 CONFIG_SPL_NET_SUPPORT=1
1021 44: Convert CONFIG_SPL_USB_HOST_SUPPORT to Kconfig
1022 ...
1023
1024This shows that commit 44 enabled three new options for the board
1025am335x_evm_usbspl which were not enabled in commit 43. There is also a
1026summary for 'arm' showing all the changes detected for that architecture.
1027In this case there is only one board with changes, so 'arm' output is the
1028same as 'am335x_evm_usbspl'/
1029
1030The -K option uses the u-boot.cfg, spl/u-boot-spl.cfg and tpl/u-boot-tpl.cfg
1031files which are produced by a build. If all you want is to check the
1032configuration you can in fact avoid doing a full build, using -D. This tells
1033buildman to configuration U-Boot and create the .cfg files, but not actually
1034build the source. This is 5-10 times faster than doing a full build.
1035
Simon Glassb464f8e2016-11-13 14:25:53 -07001036By default buildman considers the follow two configuration methods
1037equivalent:
1038
1039 #define CONFIG_SOME_OPTION
1040
1041 CONFIG_SOME_OPTION=y
1042
1043The former would appear in a header filer and the latter in a defconfig
1044file. The achieve this, buildman considers 'y' to be '1' in configuration
1045variables. This avoids lots of useless output when converting a CONFIG
1046option to Kconfig. To disable this behaviour, use --squash-config-y.
1047
Simon Glass94d2ebe2016-11-13 14:25:52 -07001048
Alex Kiernan48ae4122018-05-31 04:48:34 +00001049Checking the environment
1050========================
1051
1052When converting CONFIG options which manipulate the default environment,
1053a common requirement is to check that the default environment has not
1054changed due to the conversion. Buildman supports this with the -U option,
1055used after a build. This shows differences in the default environment
1056between one commit and the next.
1057
1058For example:
1059
1060$ buildman -b squash brppt1 -sU
1061boards.cfg is up to date. Nothing to do.
1062Summary of 2 commits for 3 boards (3 threads, 3 jobs per thread)
106301: Migrate bootlimit to Kconfig
106402: Squashed commit of the following:
1065 c brppt1_mmc: altbootcmd=mmc dev 1; run mmcboot0; -> mmc dev 1; run mmcboot0
1066 c brppt1_spi: altbootcmd=mmc dev 1; run mmcboot0; -> mmc dev 1; run mmcboot0
1067 + brppt1_nand: altbootcmd=run usbscript
1068 - brppt1_nand: altbootcmd=run usbscript
1069(no errors to report)
1070
1071This shows that commit 2 modified the value of 'altbootcmd' for 'brppt1_mmc'
1072and 'brppt1_spi', removing a trailing semicolon. 'brppt1_nand' gained an a
1073value for 'altbootcmd', but lost one for ' altbootcmd'.
1074
1075The -U option uses the u-boot.env files which are produced by a build.
1076
Simon Glass00beb242019-01-07 16:44:20 -07001077
1078Building with clang
1079===================
1080
1081To build with clang (sandbox only), use the -O option to override the
1082toolchain. For example:
1083
1084 buildman -O clang-7 --board sandbox
1085
1086
Simon Glassd829f122020-03-18 09:42:42 -06001087Doing a simple build
1088====================
1089
1090In some cases you just want to build a single board and get the full output, use
1091the -w option, for example:
1092
1093 buildman -o /tmp/build --board sandbox -w
1094
1095This will write the full build into /tmp/build including object files.
1096
1097
Simon Glassfc3fe1c2013-04-03 11:07:16 +00001098Other options
1099=============
1100
Simon Glass7beb43c2020-03-18 09:42:44 -06001101Buildman has various other command-line options. Try --help to see them.
Simon Glassfc3fe1c2013-04-03 11:07:16 +00001102
Simon Glass4e9162d2020-03-18 09:42:47 -06001103To find out what toolchain prefix buildman will use for a build, use the -A
1104option.
Simon Glass57cb9d52019-12-05 15:59:14 -07001105
Simon Glass7beb43c2020-03-18 09:42:44 -06001106To request that compiler warnings be promoted to errors, use -E. This passes the
1107-Werror flag to the compiler. Note that the build can still produce warnings
1108with -E, e.g. the migration warnings:
1109
1110 ===================== WARNING ======================
1111 This board does not use CONFIG_DM_MMC. Please update
1112 ...
1113 ====================================================
1114
Simon Glass2c3deb92014-08-28 09:43:39 -06001115When doing builds, Buildman's return code will reflect the overall result:
1116
1117 0 (success) No errors or warnings found
1118 128 Errors found
Simon Glass7beb43c2020-03-18 09:42:44 -06001119 129 Warnings found (only if no -W)
1120
1121You can use -W to tell Buildman to return 0 (success) instead of 129 when
1122warnings are found. Note that it can be useful to combine -E and -W. This means
1123that all compiler warnings will produce failures (code 128) and all other
1124warnings will produce success (since 129 is changed to 0).
1125
1126If there are both warnings and errors, errors win, so buildman returns 128.
Simon Glass2c3deb92014-08-28 09:43:39 -06001127
Simon Glassfc3fe1c2013-04-03 11:07:16 +00001128
Simon Glass6eede342014-08-09 15:32:58 -06001129How to change from MAKEALL
1130==========================
1131
1132Buildman includes most of the features of MAKEALL and is generally faster
1133and easier to use. In particular it builds entire branches: if a particular
1134commit introduces an error in a particular board, buildman can easily show
1135you this, even if a later commit fixes that error.
1136
1137The reasons to deprecate MAKEALL are:
1138- We don't want to maintain two build systems
1139- Buildman is typically faster
1140- Buildman has a lot more features
1141
1142But still, many people will be sad to lose MAKEALL. If you are used to
1143MAKEALL, here are a few pointers.
1144
1145First you need to set up your tool chains - see the 'Setting up' section
1146for details. Once you have your required toolchain(s) detected then you are
1147ready to go.
1148
Simon Glasse5a0e5d2014-08-09 15:33:03 -06001149To build the current source tree, run buildman without a -b flag:
1150
1151 ./tools/buildman/buildman <list of things to build>
1152
1153This will build the current source tree for the given boards and display
1154the results and errors.
1155
1156However buildman usually works on entire branches, and for that you must
1157specify a board flag:
Simon Glass6eede342014-08-09 15:32:58 -06001158
1159 ./tools/buildman/buildman -b <branch_name> <list of things to build>
1160
1161followed by (afterwards, or perhaps concurrently in another terminal):
1162
1163 ./tools/buildman/buildman -b <branch_name> -s <list of things to build>
1164
1165to see the results of the build. Rather than showing you all the output,
1166buildman just shows a summary, with red indicating that a commit introduced
1167an error and green indicating that a commit fixed an error. Use the -e
Simon Glassed966652014-08-28 09:43:43 -06001168flag to see the full errors and -l to see which boards caused which errors.
Simon Glass6eede342014-08-09 15:32:58 -06001169
Simon Glasse5a0e5d2014-08-09 15:33:03 -06001170If you really want to see build results as they happen, use -v when doing a
Simon Glass1d8104f2014-10-16 01:05:56 -06001171build (and -e to see the errors/warnings too).
Simon Glasse5a0e5d2014-08-09 15:33:03 -06001172
Simon Glass6eede342014-08-09 15:32:58 -06001173You don't need to stick around on that branch while buildman is running. It
1174checks out its own copy of the source code, so you can change branches,
1175add commits, etc. without affecting the build in progress.
1176
1177The <list of things to build> can include board names, architectures or the
1178like. There are no flags to disambiguate since ambiguities are rare. Using
1179the examples from MAKEALL:
1180
1181Examples:
1182 - build all Power Architecture boards:
1183 MAKEALL -a powerpc
1184 MAKEALL --arch powerpc
1185 MAKEALL powerpc
1186 ** buildman -b <branch> powerpc
1187 - build all PowerPC boards manufactured by vendor "esd":
1188 MAKEALL -a powerpc -v esd
1189 ** buildman -b <branch> esd
1190 - build all PowerPC boards manufactured either by "keymile" or "siemens":
1191 MAKEALL -a powerpc -v keymile -v siemens
1192 ** buildman -b <branch> keymile siemens
1193 - build all Freescale boards with MPC83xx CPUs, plus all 4xx boards:
1194 MAKEALL -c mpc83xx -v freescale 4xx
1195 ** buildman -b <branch> mpc83xx freescale 4xx
1196
1197Buildman automatically tries to use all the CPUs in your machine. If you
1198are building a lot of boards it will use one thread for every CPU core
1199it detects in your machine. This is like MAKEALL's BUILD_NBUILDS option.
1200You can use the -T flag to change the number of threads. If you are only
1201building a few boards, buildman will automatically run make with the -j
1202flag to increase the number of concurrent make tasks. It isn't normally
1203that helpful to fiddle with this option, but if you use the BUILD_NCPUS
1204option in MAKEALL then -j is the equivalent in buildman.
1205
1206Buildman puts its output in ../<branch_name> by default but you can change
1207this with the -o option. Buildman normally does out-of-tree builds: use -i
1208to disable that if you really want to. But be careful that once you have
1209used -i you pollute buildman's copies of the source tree, and you will need
1210to remove the build directory (normally ../<branch_name>) to run buildman
1211in normal mode (without -i).
1212
1213Buildman doesn't keep the output result normally, but use the -k option to
1214do this.
1215
1216Please read 'Theory of Operation' a few times as it will make a lot of
1217things clearer.
1218
1219Some options you might like are:
1220
1221 -B shows which functions are growing/shrinking in which commit - great
1222 for finding code bloat.
1223 -S shows image sizes for each commit (just an overall summary)
1224 -u shows boards that you haven't built yet
1225 --step 0 will build just the upstream commit and the last commit of your
1226 branch. This is often a quick sanity check that your branch doesn't
1227 break anything. But note this does not check bisectability!
1228
1229
Simon Glassfc3fe1c2013-04-03 11:07:16 +00001230TODO
1231====
1232
1233This has mostly be written in my spare time as a response to my difficulties
1234in testing large series of patches. Apart from tidying up there is quite a
Simon Glass1d8104f2014-10-16 01:05:56 -06001235bit of scope for improvement. Things like better error diffs and easier
Dirk Behme3e1ded12014-12-23 07:41:26 +01001236access to log files. Also it would be nice if buildman could 'hunt' for
Simon Glass1d8104f2014-10-16 01:05:56 -06001237problems, perhaps by building a few boards for each arch, or checking
1238commits for changed files and building only boards which use those files.
Simon Glassfc3fe1c2013-04-03 11:07:16 +00001239
1240
1241Credits
1242=======
1243
1244Thanks to Grant Grundler <grundler@chromium.org> for his ideas for improving
1245the build speed by building all commits for a board instead of the other
1246way around.
1247
1248
Simon Glassfc3fe1c2013-04-03 11:07:16 +00001249Simon Glass
1250sjg@chromium.org
1251Halloween 2012
1252Updated 12-12-12
1253Updated 23-02-13