Small USB HID bootloader for BluePill and other STM32F10X devices

[Vassilis]

Advantages

• The HID-BL firmware can be easily updated without affecting the boards.txt or linker script

Normally, it is possible in a bootloader firmware update to exceed the 2 KBytes of flash memory that we have allocated. That will change the HID bootloader start address. The free flash memory will also be reduced. The upload.maximum_size in boards.txt does not have the real free flash memory value.
So, there is a danger to compile a large sketch and the produced file will be larger from the available free flash memory. As a result, a part of the bootloader will be overwritten.

I spent some time figuring out how to solve that problem.

I slightly modified the CLI and the HID-BL firmware (do not worry, it's still under 2 Kbytes ) and I did it!

1. The CLI calculates the bin file size.
2. It requests the free flash memory from the bootloader.
3. The Bootloader responds.
4. The CLI checks if the available flash memory is larger than the BIN file size
5. If it is, it continues with the bin upload
6. If it is not, it terminates the upload process and prints a message

[BennehBoy]

So how are different sized flash memory devices going to be tackled?

Will there be 32kb, 64kb, & 128kb versions shipped to start with? Or will this be left to the user? I think either way introduces confusion for those not so familiar with what they are doing, which seems to be against the philosophy behind this change - afterall, at least the low mem BL hides all of this behind the IDE menus, the most the user has to do is pick which LED pin firmware to use (and even then it will still work if (s)he gets it wrong).

[RogerClark]

Sound like a nightmare for people to install, because they first have to work out how much flash they have.

Also, Regarding Vassilis point about no need to change the linker if there’s is a change in the BL.
It’s been years since I had to change the core because of changes to the DFU bootloader.

Currently the latest HID BL is only 2k, but with very little free space of changes.

It would be far easier to manage in the long term if the HID was assumed to be 4K, as anyone who is so concerned about needing another 2k of application space should probably use a different MCU.


And we still have no solution for the EEPROM library, except to move it, and potentially waste 2k for people not using this bootloader, unless we add some method for the EEPROM library to know when the high memory HID bootloader has been selected.


And all these changes for a unverified need for binary compatibility which I don’t think anyone will care about.

[Squonk42]

Currently the latest HID BL is only 2k, but with very little free space of changes.

In my PR that can be used either in low or high-mem, the BL size is currently 1924 bytes, far (124 bytes, ~6%) away from the 2K (2048 bytes) page limit. Right now, besides this low/high mem dilemma, there is not much expected changes in the pipe.

And we still have no solution for the EEPROM library, except to move it, and potentially waste 2k for people not using this bootloader, unless we add some method for the EEPROM library to know when the high memory HID bootloader has been selected.

The way the EEPROM library is handling its Flash space is ugly: there is even an "HACK ALERT" in the comment just above its definition in the sources. Basically, the user has to define a preprocessor symbol to match the correct CPU

The right way to handle this is to export linker symbols in the linker script to provide the correct Flash size (possibly including a BL) and use these to reserve the correct location and size for the EEPROM space, without requiring the user to modify anything by hand.

[Squonk42]

[So, there is a danger to compile a large sketch and the produced file will be larger from the available free flash memory. As a result, a part of the bootloader will be overwritten.

No, in the PR I sent, the bootloader refuses to overwrite itself. The user sketch will not be completely written, but there is no chance to brick the device.

The only way to destroy the BL is to use either ST-Link or Serial BL, but this is also the case for all low-mem bootloaders (including DFU and Maple) if you flash a firmware that is not aware of the bootloader location and size.

[Vassilis]

Disadvantages

• Too many bootloader binaries

On low-memory HID bootloader we have to choose the F103 according to the led pin due to the board manufacturer.

On high-memory HID bootloader, except the LED pin, we have to choose the flash size because there are F103 with 16, 32, 64, 128, 256, 512, 768, 1024 Kbytes of flash memory. The bootloader is loaded at the last 2 Kbytes. So it could be a small problem, not to the final user but to those who are going to rewrite the bootloader makefile and ld files .

It needs a lot of work to make a high-mem bootloader full functional because there are too many variables to be examined.

[Vassilis]

I am thinking to change the F103 HID-BL compiling procces to something like that:

Code: Select all

make FLASH_SIZE=64 RAM_SIZE=20 LED_PIN=PC13 LED_ON=HIGH
make FLASH_SIZE=64 RAM_SIZE=20 LED_PIN=PC13 LED_ON=HIGH DISC=PB9 DISC_EN=HIGH

The FLASH_SIZE, FLASH_SIZE, LED and the DISC (Disconnect pin) values are defined by the user.

It will be only one generic F103 linker script (STM32F103.ld). That will cover the most board and mcu variants.
The Arduino_STM32 files will not be affected. Only the HID-BL source code files.
I have done some compiling tests and it seems that the above syntax it is long but it works good enough

What is your opinion about that ?

[RogerClark]

It’s definitely easier to manage than having multiple targets in the Make file

Why is flash size needed, is this for the HIgh Memory version only ?

Do you need both DISC_EN and DISC_PIN ?
E.g. can you have Disc pin without Disc enable ?


Also. I think @squonk42 was working on a system to configure most if this by changing the compiler binary.
He was going to store config information in an unused location in the vector table, and the bootloader would read this data and use it to control which pin is the LED etc

So you would not need to have build configs for LED pin , and perhaps not for some other settings either

[Squonk42]

Yes, managing all these targets is just a pain.

If we remove the unused LED2, the HID BL deals with 3 pins at most:

• the BOOT pin, used to force entering the bootloader. Default is PB2/BOOT1 on the BP that has a jumper for it as well as for BOOT0, and the combination of BOOT0=HIGH + BOOT1=LOW was actually unused. We could hard-code the pin number for this function, but this may be useful to let it configurable for custom boards.
• the LED pin. Default is PC13 on the BP
• the DISC pin. Not needed on the BP, but used for the Maple board which has an external USB disconnect circuit. Default for the Maple is PB9

If we define a "pin" as a port name or number (ranging in theory from "A" to "G", but more realistically from "A" to "C" or 0 to 3) plus a bit number (from 0 to 15), this can be packed into 7 bits, with a reserved port (4) for unused pins.

There is often the need to define a pin as either ACTIVE_LOW or ACTIVE_HIGH, so the 8th bit could be used for this purpose.

Then, some pins may require to be set into a specific MODE/CNF in the sense of the Port bit configuration table. For the STM32F103, it is here:
https://www.st.com/content/ccc/resource ... f#page=161

For example, the BOOT pin may require an internal pull-up for the Maple board, whereas it is in floating input mode for the other boards having an external pull-up resistor. The LED pin is usually a push-pull output, but the DISC pin is an open-drain output.

This pin mode configuration requires 4 bits per pin, and the minimum required size to store the pin configuration is thus 3 * (8 + 4) = 36 bits = 4.5 bytes.

RAM_SIZE is not needed, and FLASH_SIZE is only required for the high-memory BL.

Most STM32s have "Reserved" vectors in their Vector Table, and addresses 0x0000001C - 0x0000002B (16 bytes) and 0x00000034 - 0x00000037 ( 4 bytes) seem to be the same for most chips.

In my high-memory PR, I used addresses 0x0000001C - 0x0000001F (4 bytes) to store the original user reset vector to jump to after running the BL, but I could have used the second area as well.

We could use the 0x0000001C - 0x0000002B to store the pin configuration and configure them at run time rather than fix them during compilation like today, this won't require much code in the BL.

But then, there is a need for a tool to flash this configuration: it can be by ST-Link or Serial BL, or using a specific sketch, but I don't know how practical this is for a user, any though on this subject is welcome!

EDIT: the Flash size can be determined at run time using the F_SIZE register (address 0x1FFFF7E0 on the STM32F103C8T6).

[RogerClark]

@Squonk42

I presumed the "configurator" would need to be some form of web application.

Initially I though perhaps PHP. But thinking again, I think Javascript can probably do this.

I know there are JS libraries to generate PDF's dynamically and then make the browser think the user has downloaded a file, so I presume something similar can be done for this.

UI could be very simple, just some drop down menus and radio buttons etc