Rendered at 12:54:33 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
nijave 19 hours ago [-]
Having http as an alternative to tftp is a nice win. The range of things that can run an http server is much bigger than tftp
>Additionally, adding the TLS layer brings back the missing integrity and confidentiality guarantees and thus paves the way to move critical boot components out of the trusted network, possibly even to a remote location/Cloud.
Doesn't secure boot already provide this or am I misunderstanding something? I suppose secure boot only provides integrity but not confidentiality although I'm not sure how much confidentiality matters given we're just talking about the kernel here
naturalmovement 17 hours ago [-]
> > adding the TLS layer brings back the missing integrity
A foolish interpretation of what TLS does and I see this every day. Integrity of the bits and bytes in transit is unimportant here. Validation of the signed software after you have received it is everything. TLS integrity is at best redundant and at worst — the interpretation made here — leaves you vulnerable and with a false sense of security.
Anyone who has gone to the trouble to modify software to inject malware would certainly happily serve it to you over TLS.
rwmj 15 hours ago [-]
In theory the client could validate a specific server with a pinned certificate, although TLS implementations can make this difficult to do in practice. TLS also lets you use client certificates to authenticate the client to the server, which could be a win in some situations (although also a PITA to set up).
naturalmovement 14 hours ago [-]
I can guarantee you with nearly 100% certainty that UEFI TLS clients are bound to be buggy garbage broken in not-insignificant ways.
bigfatkitten 7 hours ago [-]
The IP stack and HTTP clients are problematic enough without adding the enormous complexity of a TLS implementation on top.
naturalmovement 5 hours ago [-]
They have a hard enough time managing the relatively few certificates for secure boot.
You want me to believe all the various BIOS manufacturers are going to competently manage a WebPKI root certificate program?
nijave 10 hours ago [-]
From the article, it's using OpenSSL in EDK II
In fact, a whole section of the article is dedicated to talking about how they got tripped up by OpenSSL security level 3 rejecting 2048 bit RSA key
eggnet 16 hours ago [-]
TLS also allows for the contents of a boot image to be hidden from others.
naturalmovement 14 hours ago [-]
Ok, but so what?
You guys are out here protecting against ghosts but at the same time making the really important stuff 10x harder and more vulnerable to bugs.
pzmarzly 17 hours ago [-]
> The range of things that can run an http server is much bigger than tftp
Don't go too crazy though, these UEFI HTTP clients are not well behaved. For example, last time I checked, EDK required the URL to end with ".efi", instead of checking Content-Type header.
I have an HTTP netboot flow that worked on older AMI Aptio firmwares, but fails on anything newer to even fetch the bootfile after a successful DHCP cycle, so I heartily agree with your sentiment that these are not necessarily well adjusted clients!
LooseMarmoset 19 hours ago [-]
Secure boot is designed to verify software signatures. The UEFI bios might support loading software over https, but it isn't part of secure boot. Secure boot would verify any kernels/etc loaded from https.
RulerOf 18 hours ago [-]
That was the point as I read it. Payload signature verification is a good and sometimes desirable alternative to transport encryption when the payload itself isn't secret.
Highly-cacheable resources like game and OS updates are often intentionally delivered over http as signed payloads to facilitate middlebox caching.
naturalmovement 17 hours ago [-]
> Secure boot is designed to verify software signatures
aka integrity.
HTTPS is a useless gesture here, adding complexity to critical software that needs to be as simple and auditable as possible. Confidentiality is essentially unimportant to anyone but the most autistic of by-the-book nerds. It buys you nothing in a practical sense. Most netbooting happens over closed networks anyway.
robertlagrant 17 hours ago [-]
I agree that integrity can be done by secure boot, but HTTPS does mean that someone can't intercept your request and serve you valid, signed, older software that has a known security flaw in it.
nijave 9 hours ago [-]
An LLM pointed this out to me as well which I think is a fair point.
However, in practice it doesn't matter for any machine that has persistence since it only needs to netboot once to transfer an image to local storage. Besides that, you can also invert and bootstrap with BMC or even a flash drive and skip the whole network anyway.
Finally, you can reduce risk if you only bootstrap a minimal executable which itself has a robust bootstrapping mechanism. In the post, they're jumping to iPXE from UEFI so the concern would be loading an old iPXE version.
rwmj 15 hours ago [-]
NBD (over TLS) would be even better because it loads on demand. You could ship large full-featured OS images and only the parts/features you used would be loaded. One day when I'm retired I plan to add this to OVMF.
naturalmovement 17 hours ago [-]
BTW Apple has been doing HTTP boot for like two decades at this point. How do you think Internet Recovery works? It leverages a dusty old Apple netbooting spec.
my-boot 17 hours ago [-]
Unfortunately, Apple seems to be alone in having a good implementation. Using the old PXE, you expect to see some dos-like screens and slow loading but with HTTP the experience is not much better. Any decently sized bootloader is downloaded at a snails-pace and the user is presented with a very technical screen. Fine for rescue-boot like purposes but not fine when your daily driver is expected to be booted from network. I had especially expected better from Dell but the experience is... lacking.
mschuster91 15 hours ago [-]
> Unfortunately, Apple seems to be alone in having a good implementation.
Well, Apple is in full control over their entire stack, down to the firmware on the embedded parts.
In the non-Apple world, no way, simply due to the sheer insane amount of different ethernet and wireless chipsets, with many of them shipping binary blobs. The mediatek blobs alone expand to 64MB [1], Intel clocks in a further 24 MB [2], and then there's all the other firmware stuff.
Unfortunately, there is nothing in the "physical world" that comes even close to USB-CDC in its versatility.
I'd guess that the mediatek blobs are exactly 64k, and mostly 0s?
naikrovek 13 hours ago [-]
> slow loading
Not necessarily. When I worked at Yahoo, in Australia (what a glorious time that was) 25 years ago, I built servers in the datacenter using PXE. It was anything but slow.
Unbox a server, plug it into the PXE network, boot it. It boots to a miniscule FreeBSD distribution and you use a common user/pass to log in. Then, you type clone -h <fqdn> and it mounted an NFS share and installed packages and config files for that hostname. In three minutes or so your server shut down and you racked it up, plugged it into the production network and it started accepting work, or it notified the engineers in the US that it was ready for use and they’d add it to a pool, then it would start handling work.
It was extremely slick.
The build network was secure because you could only access it in secure areas which had the build network and a build server deployed to it. So security was not a problem.
Why can’t I set up Windows or MacOS like that? I know the answer I just find the answer annoying.
RulerOf 7 hours ago [-]
> Why can’t I set up Windows or MacOS like that? I know the answer I just find the answer annoying.
I wish I still cared about this. I had intended to build an iPXE boot menu via a small web service that would act as a windows install XML template editor/selector, but I never got around to doing it after learning enough web dev to pull it off.
I built a few similar things that worked inside of WinPE, but the slowness of waiting for it to boot was always what drove me to do as much config as possible in the PXE boot menu—you can get into that in seconds versus minutes for the PE.
I used to install Windows a lot, and found a lot of tech around it to be a little too opinionated. SMS/WDS were just too legacy-leaning and Microsoft Enterprise-flavored. FOG was a little too heavy-handed (though very good). Glazier excited me but I never actually used it to determine if it has the flexibility I wanted...
But I digress. OS installs should be a lot easier and faster to accept your configuration preferences and get to work when the goal is "erase this machine and reinstall" than they are even today.
12 hours ago [-]
noodlesUK 19 hours ago [-]
To what extent is this possible for actual metal hardware? I'm sure lots of us are running PXE/TFTP systems and HTTP would be a heck of a lot simpler.
ValdikSS 16 hours ago [-]
It works on the majority of UEFI implementations
1. Just a matter of a boot entry. However adding the boot entry is not trivial: efibootmgr used to have implementation which have been reverted (it was incomplete, works only with full device path unlike just MAC which the original code added).
I don't know any utilities to do that, ended patching efibootmgr myself.
There's still the tftp->ipxe->http->??? path. TFTP only needs to serve a 300kb file which can then switch to more robust transport like http for the kernel/OS
You could bypass that by shipping iPXE on USB tho
On metal you also commonly have a BMC so generally that lets you attach an ISO or other storage you can boot from to bypass UEFI primitive PXE. This is probably the biggest one--use BMC functionality instead of UEFI PXE
At home, I use JetKVM or GL.iNet Comet network KVM to bootstrap commodity hardware without BMC (by attaching an ISO). Probably could make a cheap commodity device with Raspberry Pi Zero that does that same thing at lower cost although at that point you're back to "just use USB storage"
The worst thing about UEFI HTTP boot is the utter lack of information to debug anything that's gone wrong. Whether that's the DHCP filename option is some wrong format for whatever stupid mode the UEFI is in, or there's some dhcp relay issue. It literally tells you almost nothing besides "can't get NBP file size".
The error messages seem to be written by people on a happy path who don't know how utterly broken almost everything about networking and DHCP even is.
And this is all IPv4! The IPv6 stuff is even more cryptic with different DHCP options and dealing with RAs and managed-flag, other-flag, etc.
It's infuriating. And I work on a team that writes code to generate all these things for automating bare metal for a living.
wmf 14 hours ago [-]
It's probably a good idea to follow the path of this article and get everything working in a VM first where booting is faster and it's easier to sniff packets. Once you have vanilla OVMF working you can try booting real servers.
jeffrallen 18 hours ago [-]
Hey, I'd say "hire this guy", but we already did. This is an excellent write up of excellent work by an excellent colleague. Thanks yadutaf!
>Additionally, adding the TLS layer brings back the missing integrity and confidentiality guarantees and thus paves the way to move critical boot components out of the trusted network, possibly even to a remote location/Cloud.
Doesn't secure boot already provide this or am I misunderstanding something? I suppose secure boot only provides integrity but not confidentiality although I'm not sure how much confidentiality matters given we're just talking about the kernel here
A foolish interpretation of what TLS does and I see this every day. Integrity of the bits and bytes in transit is unimportant here. Validation of the signed software after you have received it is everything. TLS integrity is at best redundant and at worst — the interpretation made here — leaves you vulnerable and with a false sense of security.
Anyone who has gone to the trouble to modify software to inject malware would certainly happily serve it to you over TLS.
You want me to believe all the various BIOS manufacturers are going to competently manage a WebPKI root certificate program?
In fact, a whole section of the article is dedicated to talking about how they got tripped up by OpenSSL security level 3 rejecting 2048 bit RSA key
You guys are out here protecting against ghosts but at the same time making the really important stuff 10x harder and more vulnerable to bugs.
Don't go too crazy though, these UEFI HTTP clients are not well behaved. For example, last time I checked, EDK required the URL to end with ".efi", instead of checking Content-Type header.
I have an HTTP netboot flow that worked on older AMI Aptio firmwares, but fails on anything newer to even fetch the bootfile after a successful DHCP cycle, so I heartily agree with your sentiment that these are not necessarily well adjusted clients!
Highly-cacheable resources like game and OS updates are often intentionally delivered over http as signed payloads to facilitate middlebox caching.
aka integrity.
HTTPS is a useless gesture here, adding complexity to critical software that needs to be as simple and auditable as possible. Confidentiality is essentially unimportant to anyone but the most autistic of by-the-book nerds. It buys you nothing in a practical sense. Most netbooting happens over closed networks anyway.
However, in practice it doesn't matter for any machine that has persistence since it only needs to netboot once to transfer an image to local storage. Besides that, you can also invert and bootstrap with BMC or even a flash drive and skip the whole network anyway.
Finally, you can reduce risk if you only bootstrap a minimal executable which itself has a robust bootstrapping mechanism. In the post, they're jumping to iPXE from UEFI so the concern would be loading an old iPXE version.
Well, Apple is in full control over their entire stack, down to the firmware on the embedded parts.
In the non-Apple world, no way, simply due to the sheer insane amount of different ethernet and wireless chipsets, with many of them shipping binary blobs. The mediatek blobs alone expand to 64MB [1], Intel clocks in a further 24 MB [2], and then there's all the other firmware stuff.
Unfortunately, there is nothing in the "physical world" that comes even close to USB-CDC in its versatility.
[1] https://packages.debian.org/forky/firmware-mediatek
[2] https://packages.debian.org/forky/firmware-intel-misc
Not necessarily. When I worked at Yahoo, in Australia (what a glorious time that was) 25 years ago, I built servers in the datacenter using PXE. It was anything but slow.
Unbox a server, plug it into the PXE network, boot it. It boots to a miniscule FreeBSD distribution and you use a common user/pass to log in. Then, you type clone -h <fqdn> and it mounted an NFS share and installed packages and config files for that hostname. In three minutes or so your server shut down and you racked it up, plugged it into the production network and it started accepting work, or it notified the engineers in the US that it was ready for use and they’d add it to a pool, then it would start handling work.
It was extremely slick.
The build network was secure because you could only access it in secure areas which had the build network and a build server deployed to it. So security was not a problem.
Why can’t I set up Windows or MacOS like that? I know the answer I just find the answer annoying.
I wish I still cared about this. I had intended to build an iPXE boot menu via a small web service that would act as a windows install XML template editor/selector, but I never got around to doing it after learning enough web dev to pull it off.
I built a few similar things that worked inside of WinPE, but the slowness of waiting for it to boot was always what drove me to do as much config as possible in the PXE boot menu—you can get into that in seconds versus minutes for the PE.
I used to install Windows a lot, and found a lot of tech around it to be a little too opinionated. SMS/WDS were just too legacy-leaning and Microsoft Enterprise-flavored. FOG was a little too heavy-handed (though very good). Glazier excited me but I never actually used it to determine if it has the flexibility I wanted...
But I digress. OS installs should be a lot easier and faster to accept your configuration preferences and get to work when the goal is "erase this machine and reinstall" than they are even today.
1. Just a matter of a boot entry. However adding the boot entry is not trivial: efibootmgr used to have implementation which have been reverted (it was incomplete, works only with full device path unlike just MAC which the original code added).
I don't know any utilities to do that, ended patching efibootmgr myself.
Learned about it from this talk: https://youtu.be/EtGhHCr3VLE?t=567
2. HTTP Boot is also available as a DHCP option 67 without boot entry:
* https://github.com/tianocore/tianocore.github.io/wiki/HTTP-B...
* https://documentation.suse.com/sles/15-SP6/html/SLES-all/cha...
You could bypass that by shipping iPXE on USB tho
On metal you also commonly have a BMC so generally that lets you attach an ISO or other storage you can boot from to bypass UEFI primitive PXE. This is probably the biggest one--use BMC functionality instead of UEFI PXE
At home, I use JetKVM or GL.iNet Comet network KVM to bootstrap commodity hardware without BMC (by attaching an ISO). Probably could make a cheap commodity device with Raspberry Pi Zero that does that same thing at lower cost although at that point you're back to "just use USB storage"
The error messages seem to be written by people on a happy path who don't know how utterly broken almost everything about networking and DHCP even is.
And this is all IPv4! The IPv6 stuff is even more cryptic with different DHCP options and dealing with RAs and managed-flag, other-flag, etc.
It's infuriating. And I work on a team that writes code to generate all these things for automating bare metal for a living.