MCU Bootloaders: Boxing up
This article is the third one in my series about bootloaders for microcontrollers, where I’m exploring the clever algorithms that make those pieces of software possible, as well as the ones I developped for Polyboot (a bootloader geared towards updating devices that contain multiple microcontrollers).
If this is your first time here, then I strongly recommend you to read the opening article of this series beforehand, where I explain the basics of what a bootloader for microcontroller is and does.
In the last article, I showed how one can carefully design an algorithm to be resistant to sudden shutdowns — a situation that our little bootloader can unfortunately find itself in. With this algorithm, we now know how a bootloader can update the application from one version to another.
Don’t worry if you didn’t catch everything, I prepared a recap :)
There’s still a bit of work left, though, before we can say that our bootloader can reliably update a microcontroller. It still doesn’t know when to update, or where the application is. Today, we’re going to teach it exactly that!
There isn’t going to be any algorithm in this article, so you should be good if you made it so far.
Partitioning the flash
Just like on regular computers, we need to impose some structure on how we are using the internal storage. On regular computers, the internal storage is split into multiple partitions. For instance, you might have a partition dedicated to the read-only system files, one for the bootloader and one for your files. This is commonly done using MBR or GPT.
On a microcontroller, it would be very wasteful to use these partition schemes: we don’t need all of their features and storage space is a premium. Instead, we are going to let the developer choose the partitions ahead of time and bake it directly into the bootloader’s source code.
Which partitions do they have to choose?
We are going to need 4 different kinds of partition, each serving a different purpose:
- The bootloader partition will contain the code for the bootloader.
- The primary partition will contain the current version of the application. This is where the application will be run from.
- Secondary partitions will contain the next or the previous versions of the application.
- The scratch partition is used by our bootloader to always keep a copy around of the data it is processing, to ensure that it is resistant to sudden shutdowns.
The partitions must, naturally, not overlap and be big enough for what we want to store in them.
They also need to be aligned to a page of the flash: it must be made up of a whole number of pages, otherwise we risk loosing information from another partition if we try to erase the memory allocated to a partition.
Oh, wait! My recap!
Moxie's flash memory cheat sheet
Welcome to my small recap about how flash memory works!
Flash memory is very cheap but a bit difficult to write to. In microcontrollers, we are using a variant called “NOR” flash memory, which is fast enough to read that we can directly execute the code from it.
We can read from flash memory with little to no restriction, but for economical and physical reasons, writing to it is trickier:
- By default, all bits are set to
1
. - We can write to a word, setting the bits we want to
0
and leaving the others untouched. - A word is usually
2
to8
bytes long. - We can erase an entire page, setting all bits back to
1
. - A page is quite big, usually something like
1024
to4096
bytes! - We can only write each word a set number of times.
- On some devices, we can only write each word once! Gasp!
A few examples
For our simple, example bootloader, we will require the flash memory to be split into four different partitions: one for the bootloader, a primary and secondary partitions (of similar sizes) and a small scratch partition. This is the most basic layout required for the scratch-swap algorithm to work, but it can get the job done:
MCUBoot gives you some more freedom: it lets you have multiple primary and secondary partitions, but each secondary partition must be paired with a primary partition. The only way to update the code within a primary partition is to put its new version within its associated secondary partition:
Polyboot, on the other hand, lets you have fewer secondary partitions than primary partitions. As long as there is one secondary partitions, new versions of any of the applications running across the device can be placed in any secondary partition, and the bootloader will take care of the rest:
This means that you can dedicate almost all of a microcontroller’s internal storage to the application it needs to run, allowing board designers to use a cheaper microcontroller!
Nifty!
Images
Okay, now we know where the application and its new version are, but how do we know when to update?
The bootloader indeed cannot just guess when there is an update available. It needs to be able to read what’s inside the primary and secondary partitions to gather several pieces of information:
- Whether or not there is an application within that partition.
- What version of the software it is.
- How many bytes that version of the software uses.
- Some kind of checksum, to ensure that nothing got corrupted.
- Other metadata, like cryptographic keys or, in the case of Polyboot, to which microcontroller the application belongs.
There is also some information that the bootloader needs to modify at runtime:
- Whether or not an update is under way.
- The journal for the scratch-swap algorithm.
That’s a lot to manage!
Clearly, we need a way to neatly organize all of this information into an image: a “box” in which we can place the application’s code and the metadata our bootloader needs, that can easily be handled by a machine (which here is our bootloader).
This is not to be confused with the images that a camera takes!
There are many, many ways to do design an image format, but I chose for Polyboot to use an extension of MCUBoot’s image format. This lets developers using my bootloader reuse the tools they already built around MCUBoot for creating, flashing or analyzing images.
MCUBoot’s image format looks something like this:
The exact details of this format are not important for us today, but we can look at the more interesting aspects:
- The first few bytes of the image are called “magic bytes”, which were chosen randomly. Their presence indicates that we are, in fact, looking at an MCUBoot image.
- The application’s code and metadata is protected by a checksum, which is stored near the end of the image.
- TLV stands for Type-Length-Value: it’s a way to efficiently encode data of various types and sizes.
The trailer
What’s that thing at the very end of your image?
This is called the trailer, it also contains a few magic bytes, but it most importantly contains the data that the bootloader will modify.
The data stored in this trailer is different between MCUBoot and Polyboot, but both variants contain the journal and a flag (is_updating
), indicating whether the scratch-swap algorithm is being ran.
Unlike the rest of the image, there is no checksum mechanism in place to ensure that the data remains sound. We also still need to abide by the flash storage medium’s rules, so that means pulling off all the tricks we can!
What kind of tricks?
You might have already noticed an issue with the is_updating
flag.
During an update, the bootloader would have to set that flag twice:
once to turn it on, right before the scratch-swap algorithm begins, and once to turn it back off.
If you remembered well, some microcontrollers make it impossible to set such a flag twice within a single word!
The trick is to store this flag into two words: one indicating that the update of that image started, and one that it finished.
Encoding of the is_updating
flag using two words.
When to update
With all of this in place, we can finally teach our bootloader when to update!
Obviously, we want it to update if the version read in the secondary partition is newer than the version read in the primary partition. But, if you recall from the last article, we also would like for it to revert a faulty update.
How is it going to know when an update is faulty?
This is a surprisingly tricky problem. The solution that MCUBoot uses and that I have decided to keep is to ask the application to tell the bootloader when it thinks it is “good”. If any new version is faulty, then it will crash before telling the bootloader that it is good. The next time the device reboots, the bootloader can detect that it happened, and revert the faulty update!
We store the fact that an image is good with a flag called image_good
inside of the trailer, as it will need to be modified.
The reset value of image_good
is interpreted as false
, and once the application notifies us that it is good, we program it to a value interpreted as true
.
Note that this isn’t a perfect solution: a critical bug could very well appear after the application notifies the bootloader. There is sadly not much else that one can do to prevent this kind of scenario, besides notifying the bootloader as late as possible.
Here are the different rules that we can teach to our bootloader so it finally knows when to update:
- If the secondary partition contains a newer version, then update the application.
- If the primary partition was not flagged as good when the device booted up, then revert the update.
A typical update sequence using the rules above.
Letting developers skip bumping the version
We can also choose to add the rule that if the secondary partition contains the same version as the primary partition, but isn’t flagged as good, then we update the application.
This scenario doesn’t happen normally (see the graph above), but it makes it so that developers don’t need to bump the version number whenever they want to try out a new version locally.
I implemented it by comparing the tuple (version, !good)
of each partition, instead of just comparing their version
.
A working bootloader
One last step is to give our bootloader the ability to, well, boot the application. Thankfully, on Cortex-M microcontrollers, this is pretty easy :)
In C, you will have to write some assembly to read and set the starting address and the stack address, while in Rust the authors of cortex_m
lovingly put together cortex_m::asm::bootload
, which does this exact thing for you.
With this out the way, we have a working bootloader that is almost resilient to sudden shutdowns!
“almost”?! I thought we were fully resilient!
Well, the scratch-swap algorithm from the last article is resilient, but it can only be used to update the body of the image, not the trailer.
In the next article, we will see how MCUBoot handles this problem, as well as my different attempts at solving this issue in Polyboot, where there isn’t just one primary and one secondary partition. You will also finally get to see a new algorithm :D
Congratulations on making it this far! If you are reading this, then I’m still writing the next article in this series (or you’re looking at an archive of this page; oops). Follow me on BlueSky if you want to be notified of when the next article is ready!