blob: 3a12438f909de909ac0511e5fc9efad348aef46d [file] [log] [blame]
Heinrich Schuchardtb9896482018-12-01 10:07:10 +01001# iSCSI booting with U-Boot and iPXE
Heinrich Schuchardt29a8a282018-01-28 15:26:02 +01002
Heinrich Schuchardtb9896482018-12-01 10:07:10 +01003## Motivation
Heinrich Schuchardt29a8a282018-01-28 15:26:02 +01004
5U-Boot has only a reduced set of supported network protocols. The focus for
6network booting has been on UDP based protocols. A TCP stack and HTTP support
7are expected to be integrated in 2018 together with a wget command.
8
9For booting a diskless computer this leaves us with BOOTP or DHCP to get the
10address of a boot script. TFTP or NFS can be used to load the boot script, the
11operating system kernel and the initial file system (initrd).
12
13These protocols are insecure. The client cannot validate the authenticity
14of the contacted servers. And the server cannot verify the identity of the
15client.
16
17Furthermore the services providing the operating system loader or kernel are
18not the ones that the operating system typically will use. Especially in a SAN
19environment this makes updating the operating system a hassle. After installing
20a new kernel version the boot files have to be copied to the TFTP server
21directory.
22
23The HTTPS protocol provides certificate based validation of servers. Sensitive
24data like passwords can be securely transmitted.
25
26The iSCSI protocol is used for connecting storage attached networks. It
27provides mutual authentication using the CHAP protocol. It typically runs on
28a TCP transport.
29
30Thus a better solution than DHCP/TFTP/NFS boot would be to load a boot script
31via HTTPS and to download any other files needed for booting via iSCSI from the
32same target where the operating system is installed.
33
34An alternative to implementing these protocols in U-Boot is to use an existing
Heinrich Schuchardt76a472d2018-07-29 13:50:50 +020035software that can run on top of U-Boot. iPXE[1] is the "swiss army knife" of
Heinrich Schuchardt29a8a282018-01-28 15:26:02 +010036network booting. It supports both HTTPS and iSCSI. It has a scripting engine for
37fine grained control of the boot process and can provide a command shell.
38
39iPXE can be built as an EFI application (named snp.efi) which can be loaded and
40run by U-Boot.
41
Heinrich Schuchardtb9896482018-12-01 10:07:10 +010042## Boot sequence
Heinrich Schuchardt29a8a282018-01-28 15:26:02 +010043
44U-Boot loads the EFI application iPXE snp.efi using the bootefi command. This
45application has network access via the simple network protocol offered by
46U-Boot.
47
48iPXE executes its internal script. This script may optionally chain load a
49secondary boot script via HTTPS or open a shell.
50
51For the further boot process iPXE connects to the iSCSI server. This includes
52the mutual authentication using the CHAP protocol. After the authentication iPXE
53has access to the iSCSI targets.
54
55For a selected iSCSI target iPXE sets up a handle with the block IO protocol. It
56uses the ConnectController boot service of U-Boot to request U-Boot to connect a
57file system driver. U-Boot reads from the iSCSI drive via the block IO protocol
58offered by iPXE. It creates the partition handles and installs the simple file
Heinrich Schuchardt76a472d2018-07-29 13:50:50 +020059protocol. Now iPXE can call the simple file protocol to load GRUB[2]. U-Boot
60uses the block IO protocol offered by iPXE to fulfill the request.
Heinrich Schuchardt29a8a282018-01-28 15:26:02 +010061
Heinrich Schuchardt76a472d2018-07-29 13:50:50 +020062Once GRUB is started it uses the same block IO protocol to load Linux. Via
63the EFI stub Linux is called as an EFI application::
Heinrich Schuchardt29a8a282018-01-28 15:26:02 +010064
Heinrich Schuchardt76a472d2018-07-29 13:50:50 +020065 +--------+ +--------+
66 | | Runs | |
67 | U-Boot |========>| iPXE |
68 | EFI | | snp.efi|
69 +--------+ | | DHCP | |
70 | |<===|********|<========| |
71 | DHCP | | | Get IP | |
72 | Server | | | Address | |
73 | |===>|********|========>| |
74 +--------+ | | Response| |
75 | | | |
76 | | | |
77 +--------+ | | HTTPS | |
78 | |<===|********|<========| |
79 | HTTPS | | | Load | |
80 | Server | | | Script | |
81 | |===>|********|========>| |
82 +--------+ | | | |
83 | | | |
84 | | | |
85 +--------+ | | iSCSI | |
86 | |<===|********|<========| |
87 | iSCSI | | | Auth | |
88 | Server |===>|********|========>| |
89 | | | | | |
90 | | | | Loads | |
91 | |<===|********|<========| | +--------+
92 | | | | GRUB | | Runs | |
93 | |===>|********|========>| |======>| GRUB |
94 | | | | | | | |
95 | | | | | | | |
96 | | | | | | Loads | |
97 | |<===|********|<========|********|<======| | +--------+
98 | | | | | | Linux | | Runs | |
99 | |===>|********|========>|********|======>| |=====>| Linux |
100 | | | | | | | | | |
101 +--------+ +--------+ +--------+ +--------+ | |
102 | |
103 | |
104 | ~ ~ ~ ~|
Heinrich Schuchardt29a8a282018-01-28 15:26:02 +0100105
Heinrich Schuchardtb9896482018-12-01 10:07:10 +0100106## Security
Heinrich Schuchardt29a8a282018-01-28 15:26:02 +0100107
108The iSCSI protocol is not encrypted. The traffic could be secured using IPsec
109but neither U-Boot nor iPXE does support this. So we should at least separate
110the iSCSI traffic from all other network traffic. This can be achieved using a
111virtual local area network (VLAN).
112
Heinrich Schuchardtb9896482018-12-01 10:07:10 +0100113## Configuration
Heinrich Schuchardt29a8a282018-01-28 15:26:02 +0100114
Heinrich Schuchardtb9896482018-12-01 10:07:10 +0100115### iPXE
Heinrich Schuchardt29a8a282018-01-28 15:26:02 +0100116
Heinrich Schuchardt76a472d2018-07-29 13:50:50 +0200117For running iPXE on arm64 the bin-arm64-efi/snp.efi build target is needed::
Heinrich Schuchardt29a8a282018-01-28 15:26:02 +0100118
119 git clone http://git.ipxe.org/ipxe.git
120 cd ipxe/src
121 make bin-arm64-efi/snp.efi -j6 EMBED=myscript.ipxe
122
123The available commands for the boot script are documented at:
124
125http://ipxe.org/cmd
126
127Credentials are managed as environment variables. These are described here:
128
129http://ipxe.org/cfg
130
131iPXE by default will put the CPU to rest when waiting for input. U-Boot does
132not wake it up due to missing interrupt support. To avoid this behavior create
Heinrich Schuchardt76a472d2018-07-29 13:50:50 +0200133file src/config/local/nap.h::
Heinrich Schuchardt29a8a282018-01-28 15:26:02 +0100134
135 /* nap.h */
136 #undef NAP_EFIX86
137 #undef NAP_EFIARM
138 #define NAP_NULL
139
140The supported commands in iPXE are controlled by an include, too. Putting the
Heinrich Schuchardt76a472d2018-07-29 13:50:50 +0200141following into src/config/local/general.h is sufficient for most use cases::
Heinrich Schuchardt29a8a282018-01-28 15:26:02 +0100142
143 /* general.h */
144 #define NSLOOKUP_CMD /* Name resolution command */
145 #define PING_CMD /* Ping command */
146 #define NTP_CMD /* NTP commands */
147 #define VLAN_CMD /* VLAN commands */
148 #define IMAGE_EFI /* EFI image support */
149 #define DOWNLOAD_PROTO_HTTPS /* Secure Hypertext Transfer Protocol */
150 #define DOWNLOAD_PROTO_FTP /* File Transfer Protocol */
151 #define DOWNLOAD_PROTO_NFS /* Network File System Protocol */
152 #define DOWNLOAD_PROTO_FILE /* Local file system access */
153
Heinrich Schuchardtb9896482018-12-01 10:07:10 +0100154### Open-iSCSI
155
156When the root file system is on an iSCSI drive you should disable pings and set
157the replacement timer to a high value [3]:
158
159 node.conn[0].timeo.noop_out_interval = 0
160 node.conn[0].timeo.noop_out_timeout = 0
161 node.session.timeo.replacement_timeout = 86400
162
163## Links
Heinrich Schuchardt29a8a282018-01-28 15:26:02 +0100164
Heinrich Schuchardt76a472d2018-07-29 13:50:50 +0200165* [1](https://ipxe.org) https://ipxe.org - iPXE open source boot firmware
166* [2](https://www.gnu.org/software/grub/) https://www.gnu.org/software/grub/ -
Heinrich Schuchardtb9896482018-12-01 10:07:10 +0100167 GNU GRUB (Grand Unified Bootloader)
168* [3](https://github.com/open-iscsi/open-iscsi/blob/master/README)
169 https://github.com/open-iscsi/open-iscsi/blob/master/README -
170 Open-iSCSI README