Menu ▾ ▴

#34 dfu-util requires -a X when flashing .dfu (DfuSE) files

0.12
closed
nobody
doc (13)
2021-09-19
2016-11-17
Roger Wolff
No

I thought that the DFU file format was meant to tell the flashing utility stuff like "the ALT port to use" and "The address to flash to".

So I created a DFU file, told it to use address 0x08000000, and when I dump it it says; "ALT 0". But when I try to program it, it sees the ALT0 (flash) and ALT1 (option bytes) as separate devices and requires me to sepecify -a 0 on the commandline. IMHO that is defeating the purpose of the DFU file format.

Discussion

1 2 > >> (Page 1 of 2)

  • Tormod Volden

    Tormod Volden - 2016-11-26

    Defeating the purpose is maybe taking it too far, because the current behaviour still prevents users from mistakenly writing an image to the wrong memory device/partition, which is the most important about this feature. However from the view of convenience it could be nice if it automatically sets the correct alternate interface by itself. The reason it is like it currently is, is that dfu-util was not originally written for .dfu files (or DfuSe devices) and such support was added later. Allowing the DFU file to control which alternate interface to use will require more invasive changes. It can still be done, though!

     

  • Roger Wolff

    Roger Wolff - 2016-12-23

    Yes... I've been a dfu-util user since "before" the .dfu file format. I had a script that called dfu-util with the proper option to simply flash a binary. The current situation is a lot better as I can generate and test the .dfu file on Linux before passing it on the people using windows.

     

  • Tormod Volden

    Tormod Volden - 2017-06-03
    • labels: --> doc
     

  • Roger Wolff

    Roger Wolff - 2019-04-14

    FYI: I don't agree with the label "doc" and I still think that an effort should be made to work towards the situation where something like dfu-util somefile.dfu simply does the right thing which is to flash the chip. I have less technical clients that may need to field-upgrade a board/module/system, so the less they have to type, the less that can go wrong.

    But I don't have time to work on this, and probably you don't either....

     

  • Tormod Volden

    Tormod Volden - 2019-04-14

    The label "doc" means that this should be documented better, or pointed out clearer.

    Yes, it is not a feature I intend to work on, but as they say, "patches are welcome" :)

    I can only suggest a quick workaround by having a wrapper script extract the alternative interface values from the file (dfuse-pack.py can be of use) and run dfu-use with the values.

     

  • Tormod Volden

    Tormod Volden - 2020-10-10
    • summary: dfu-util requires -a x on the commandline when flashing .dfu files. --> dfu-util requires -a X when flashing .dfu (DfuSE) files
    • Description has changed:

    Diff:

    --- old
+++ new
@@ -1,3 +1,3 @@
-I thought that the DFU fileformat was meant to tell the flahsing utility stuff like "the ALT port to use" and "The address to flash to". 
+I thought that the DFU file format was meant to tell the flashing utility stuff like "the ALT port to use" and "The address to flash to". 

-So I created a DFU file, told it to use address 0x8000000, and when I dump it it says; "ALT 0". But when I try to program it, it sees the ALT0  (flash) and ALT1 (option bytes) as separate devices and requires me to sepecify -a 0 on the commandline. IMHO that is defeating the purpose of the DFU file format. 
+So I created a DFU file, told it to use address 0x08000000, and when I dump it it says; "ALT 0". But when I try to program it, it sees the ALT0  (flash) and ALT1 (option bytes) as separate devices and requires me to sepecify -a 0 on the commandline. IMHO that is defeating the purpose of the DFU file format.
     

  • Tormod Volden

    Tormod Volden - 2020-10-10

    Can someone test this patch? It should automatically set alternate interfaces and flash them for all images in the DfuSe file. It is of course important that the patch doesn't cause issues for other workflows.

     
    0001-DfuSe-Set-alternate-interface-from-DfuSe-file.patch

  • Hendry Kaak

    Hendry Kaak - 2020-10-12

    Hello,
    @tormod I build dfu-util with your patch applied. But it doesn't seem to do the trick. Dfu-util shows the message:
    "More than one DFU capable USB device found! Try --list' and specify the serial number or disconnect all but one device",
    even though the command was as following (with serial number specified): dfu-util -s :leave -S "307637963237" -D file.DfuSe :-)

     

    • Tormod Volden

      Tormod Volden - 2020-10-12

      Thanks for testing! Can you please check what entries you have on the dfu_root chain, e.g. by debugger or printf's inside the while loop in dfuse_multiple_alt()?

       

      • Hendry Kaak

        Hendry Kaak - 2020-10-14

        Hello again, I was quite busy yesterday so I couldn't really check what was going on (and I'm still not really sure where the problem is). I'm able to download a Dfuse file to my device with the '-a 0' option in the commandline, but when I remove the alternate setting from commandline it seems to breaks. I think the last piece of the verbose log shows the most important information, like so:

        Determining device status: libusb: debug [libusb_alloc_transfer] transfer 0x8000650f0
libusb: debug [libusb_submit_transfer] transfer 0x8000650f0
libusb: debug [winusbx_submit_control_transfer] will use interface 0
libusb: debug [usbi_add_pollfd] add fd 1 events 1
libusb: debug [libusb_get_next_timeout] next timeout in 5.000000s
libusb: debug [libusb_handle_events_timeout_completed] doing our own event handling
libusb: debug [handle_events] poll fds modified, reallocating
libusb: debug [handle_events] poll() 2 fds with timeout in 5000ms
libusb: debug [handle_events] poll() returned 1
libusb: debug [windows_handle_events] checking fd 1 with revents = 0001
libusb: debug [usbi_remove_pollfd] remove fd 1
libusb: debug [windows_transfer_callback] handling I/O completion with errcode 0, size 6
libusb: debug [usbi_handle_transfer_completion] transfer 0x8000650f0 has callback 0x3d8ac7060
libusb: debug [sync_transfer_cb] actual_length=6
libusb: debug [libusb_free_transfer] transfer 0x8000650f0
state = dfuIDLE, status = 0
dfuIDLE, continuing
DFU mode device DFU version 011a
Device returned transfer size 2048
DfuSe interface name: "Device Feature"
dfu-util: Non-valid multiplier ' ', assuming bytes
Memory segment at 0xffff0000   1 x    4 =     4 (rw)
file contains 1 DFU images
parsing DFU image 1
No target name
image for alternate setting 0, (1 elements, total size = 128824)
Setting Alternate Setting #0 ...
libusb: debug [libusb_set_interface_alt_setting] interface 0 altsetting 0
libusb: debug [windows_assign_endpoints] no endpoints found for interface 0
parsing element 1, address = 0x08000000, size = 128816
dfu-util: Last page at 0x0801f72f is not writeable

        As you can see the alternate seem to be set correct to alternate setting 0 ("Internal Flash"), shown as:

        image for alternate setting 0, (1 elements, total size = 128824)
        Setting Alternate Setting #0 ...
        libusb: debug [libusb_set_interface_alt_setting] interface 0 altsetting 0

        What seems to be out of order is "DfuSe interface name: "Device Feature", which is altSetting 3 in my case. This is supposed to be labeled as "Internal Flash" AFAICT.
        I suspect that the parser (parse_memory_layout) is obtaining the wrong alternate settings from the interface description.
        Perhaps because the interface description is in reverse order? The interface list from dfu-util looks like this:

        Found DFU: [0483:df11] ver=2200, devnum=2, cfg=1, intf=0, path="7-1", alt=3, name="@Device Feature/0xFFFF0000/01*004 e", serial="307637963237"
Found DFU: [0483:df11] ver=2200, devnum=2, cfg=1, intf=0, path="7-1", alt=2, name="@OTP Memory /0x1FFF7800/01*512 e,01*016 e", serial="307637963237"
Found DFU: [0483:df11] ver=2200, devnum=2, cfg=1, intf=0, path="7-1", alt=1, name="@Option Bytes  /0x1FFFC000/01*016 e", serial="307637963237"
Found DFU: [0483:df11] ver=2200, devnum=2, cfg=1, intf=0, path="7-1", alt=0, name="@Internal Flash  /0x08000000/04*016Kg,01*064Kg,03*128Kg", serial="307637963237"

        When manually setting the alternate Setting, the output looks like this:

        Multiple alternate interfaces for DfuSe file
Opening DFU capable USB device...
libusb: debug [libusb_open] open 7.2
ID 0483:df11
Run-time device DFU version 011a
Claiming USB DFU Interface...
libusb: debug [libusb_claim_interface] interface 0
libusb: debug [winusbx_claim_interface] claimed interface 0
libusb: debug [windows_assign_endpoints] no endpoints found for interface 0
Setting Alternate Setting #0 ...
libusb: debug [libusb_set_interface_alt_setting] interface 0 altsetting 0
libusb: debug [windows_assign_endpoints] no endpoints found for interface 0
Determining device status: libusb: debug [libusb_alloc_transfer] transfer 0x8000650b0
libusb: debug [libusb_submit_transfer] transfer 0x8000650b0
libusb: debug [winusbx_submit_control_transfer] will use interface 0
libusb: debug [usbi_add_pollfd] add fd 1 events 1
libusb: debug [libusb_get_next_timeout] next timeout in 5.000000s
libusb: debug [libusb_handle_events_timeout_completed] doing our own event handling
libusb: debug [handle_events] poll fds modified, reallocating
libusb: debug [handle_events] poll() 2 fds with timeout in 5000ms
libusb: debug [handle_events] poll() returned 1
libusb: debug [windows_handle_events] checking fd 1 with revents = 0001
libusb: debug [usbi_remove_pollfd] remove fd 1
libusb: debug [windows_transfer_callback] handling I/O completion with errcode 0, size 6
libusb: debug [usbi_handle_transfer_completion] transfer 0x8000650b0 has callback 0x3d8ac7060
libusb: debug [sync_transfer_cb] actual_length=6
libusb: debug [libusb_free_transfer] transfer 0x8000650b0
state = dfuIDLE, status = 0
dfuIDLE, continuing
DFU mode device DFU version 011a
Device returned transfer size 2048
DfuSe interface name: "Internal Flash  "
Memory segment at 0x08000000   4 x 16384 = 65536 (rew)
Memory segment at 0x08010000   1 x 65536 = 65536 (rew)
Memory segment at 0x08020000   3 x 131072 = 393216 (rew)
file contains 1 DFU images
parsing DFU image 1
No target name
image for alternate setting 0, (1 elements, total size = 128824)
parsing element 1, address = 0x08000000, size = 128816

        Hope this helps in finding the culprit. Maybe I can print some data later on (I have to go for today though).

         

        Last edit: Tormod Volden 2020-10-14

        • Tormod Volden

          Tormod Volden - 2020-10-14

          It would have been good to see the dfu-util -l output for reference. No, scratch that, I see it now. What is this device?

          It should not have printed "Multiple alternate interfaces for DfuSe file" when you specified alternate interface manually.

           

          Last edit: Tormod Volden 2020-10-14

          • Hendry Kaak

            Hendry Kaak - 2020-10-15

            It's a custom PCB with a stm32F4 MCU and native USB where I'm working on :-)
            People should be able to flash the firmware over USB and those files should match the board. DfuSe files have the VID/PID information (and more to verify) in the file to match the device with, which is the reason for using DfuSe files.

            Here I printed the information from on top in the while loop in 'dfuse_multiple_alt()' as requested (the method returns 1/true when finished):

            dif=0x800065010
dev=0x8000658b0; dif->dev=0x8000658b0
configuration=1; dif->configuration=1
interface=0; dif->interface=0

dif=0x8000651e0
dev=0x8000658b0; dif->dev=0x8000658b0
configuration=1; dif->configuration=1
interface=0; dif->interface=0

dif=0x800064fa0
dev=0x8000658b0; dif->dev=0x8000658b0
configuration=1; dif->configuration=1
interface=0; dif->interface=0
Multiple alternate interfaces for DfuSe file
Opening DFU capable USB device...
libusb: debug [libusb_open] open 7.2
ID 0483:df11
Run-time device DFU version 011a
Claiming USB DFU Interface...
libusb: debug [libusb_claim_interface] interface 0
libusb: debug [winusbx_claim_interface] claimed interface 0
libusb: debug [windows_assign_endpoints] no endpoints found for interface 0
Setting Alternate Setting #3 ...
libusb: debug [libusb_set_interface_alt_setting] interface 0 altsetting 3
libusb: debug [windows_assign_endpoints] no endpoints found for interface 0
Determining device status: libusb: debug [libusb_alloc_transfer] transfer 0x8000650f0
libusb: debug [libusb_submit_transfer] transfer 0x8000650f0
libusb: debug [winusbx_submit_control_transfer] will use interface 0
libusb: debug [usbi_add_pollfd] add fd 1 events 1
libusb: debug [libusb_get_next_timeout] next timeout in 5.000000s
libusb: debug [libusb_handle_events_timeout_completed] doing our own event handling
libusb: debug [handle_events] poll fds modified, reallocating
libusb: debug [handle_events] poll() 2 fds with timeout in 5000ms
libusb: debug [handle_events] poll() returned 1
libusb: debug [windows_handle_events] checking fd 1 with revents = 0001
libusb: debug [usbi_remove_pollfd] remove fd 1
libusb: debug [windows_transfer_callback] handling I/O completion with errcode 0, size 6
libusb: debug [usbi_handle_transfer_completion] transfer 0x8000650f0 has callback 0x3d8ac7060
libusb: debug [sync_transfer_cb] actual_length=6
libusb: debug [libusb_free_transfer] transfer 0x8000650f0
state = dfuIDLE, status = 0
dfuIDLE, continuing
DFU mode device DFU version 011a
Device returned transfer size 2048
DfuSe interface name: "Device Feature"
dfu-util: Non-valid multiplier ' ', assuming bytes
Memory segment at 0xffff0000   1 x    4 =     4 (rw)
file contains 1 DFU images
parsing DFU image 1
No target name
image for alternate setting 0, (1 elements, total size = 128824)
Setting Alternate Setting #0 ...
libusb: debug [libusb_set_interface_alt_setting] interface 0 altsetting 0
libusb: debug [windows_assign_endpoints] no endpoints found for interface 0
parsing element 1, address = 0x08000000, size = 128816
dfu-util: Last page at 0x0801f72f is not writeable
             

            • Tormod Volden

              Tormod Volden - 2020-10-15

              Good, thanks a lot. The logic seems to work correctly here, it detects a 1.1a file and that there is /one/ device with multiple alternate interfaces. However there seems to be another bug surfacing. The dfuse code was up to now always run having only one entry on the dfu_root list, now it has several, maybe that triggers a bug.

               

    • Tormod Volden

      Tormod Volden - 2020-10-14

      In this example (above), where you (presumably) have a DfuSe 1.1a file, and you have several alternate interfaces available (because the device has several and you are not filtering with -a) it should not complain about "More than one", unless there are other devices. I need to see what goes wrong in dfuse_multiple_alt(), possibly a typo or thinko that I am blind for.

       

      Last edit: Tormod Volden 2020-10-14

      • Hendry Kaak

        Hendry Kaak - 2020-10-15

        Yes, that was my mistake, sorry for the misconception. Here I used the dfu-util in my environment path by accident (which is the default dfu-util v0.9) without the patch applied. That one returned the "More than one DFU capable USB device found! Try `--list`' and specify the serial number or disconnect all but one device" message. But in this case it actually should've given a message saying that I should specify an altSetting with '-a', because more than one alternate setting options are found in the same device interface, right?

         

        • Tormod Volden

          Tormod Volden - 2020-10-15

          Yes, you're right the wording is not precise in these cases, technically it is "More than one available alternate interfaces, possibly from multiple devices".

          My "dfuse_multiple_alt()" could maybe be generalized and return "multipled devices" or "multiple configurations" or "multiple alternate interfaces" and so on, and then the message can be chosen based on this.

           

  • Tormod Volden

    Tormod Volden - 2020-10-15

    I had a new look at the code, seeing where the memory layout is parsed and used. It is all based on the assumption only one alternate interface is available. This would need a lot more rework, my patch is unfortunately far from complete. It would maybe do the right thing if you use the "force" modifier to ignore memory layout mismatch, but then without all the sanity checks.

     

  • Tormod Volden

    Tormod Volden - 2020-10-15

    Here is a new patch, which I only have tested with DfuSe files that have one image (thus one alternate interface) - this is what dfuse-pack.py currently is able to create.

     
    0001-DfuSe-Set-alternate-interface-from-DfuSe-file_v2.patch

    • Hendry Kaak

      Hendry Kaak - 2020-10-16

      Nice, that seems to be working well already!
      The only log output to mention is probably this memory leakage at the end of the upload log, but the upload process seems to be working correct with a single image indeed.

      libusb: debug [libusb_close]
libusb: debug [libusb_exit]
libusb: debug [libusb_exit] destroying default context
libusb: warning [libusb_exit] some libusb_devices were leaked
       

      Last edit: Hendry Kaak 2020-10-16

      • Tormod Volden

        Tormod Volden - 2020-10-16

        Thanks. Can you please also try a download with multiple images in one file?

         

        • Hendry Kaak

          Hendry Kaak - 2020-10-19

          I'll see what I can do. Perhaps I can write a few optional bytes for now to verify if dfu-util works with multiple images.

           

          Last edit: Hendry Kaak 2020-10-19

          • Hendry Kaak

            Hendry Kaak - 2020-10-21

            @tormod
            I did a couple of tests with the dfuse-pack.py script and it seems to work well if the alternateSetting is set with '@x'. The only con I saw when using this option is that people can still unknowingly enter 2 or more different '@x' parameters and depending on the sequence of the build parameters it will always pick the last alternateSetting in the command line as the alternateSetting for the whole target suffix (without warning the user). If this is the case, it should probably create a new target with different alternateSetting (but I know this is not implemented yet).

            Further on I also tried to test downloading 2 images to a single alternate setting (flash memory). But for some reason I can't seem to download the second image to the specified address when the first build parameter of the packer script needs to be written at higher address than the other FSR, the first one seems to download just fine.
            So let's say I create a DfuSe file with the following command:

            python dfuse-pack.py --build=0x0801F730@0:test2.bin
--build=0x08000000@0:Firmware.bin --device=0x16D0:0x0F44 Firmware-test-fails.DfuSe

            This one ^^^ only downloads the firmware image to flash and not the "test2.bin", but the one below downloads both images as expected:

            python dfuse-pack.py --build=0x08000000@0:Firmware.bin --build=0x0801F730@0:test2.bin --device=0x16D0:0x0F44 Firmware-test-ok.DfuSe

            I made sure that the second image downloads some 0xDEADBEAF after the address index of the first complete firmware image, so they wouldn't overwrite one or another. Between each test I also mass-erased and checked the flash memory to make sure that nothing else is written.
            Both the Dfuse files seem to be correct as well, when swapping the build parameters in the script both the images are swapped in the file, which is as expected (with correct addresses and image sizes).
            I'll put the console output in text file in the attachments to prevent clogging this ticket too much :")
            If I need to print some variables again, it's ok let me know.

            Also a few remarks on the dfu-util output, I saw the following 2 lines in the output:

            file contains 1 DFU images
parsing DFU image 1

            Looking at this can be a bit confusing, one could interpret both lines as the file image elements, but AFAICS the first output line is all about the amount of targets in the file read from the Dfuse Prefix right?
            Maybe it's better to reword "file contains 1 DFU images" as "File contains x target(s)". And "parsing DFU image x" as "Parsing image element x" or even better would be to display the current target index with the image element that is being parsed at the time as in "Parsing image element x from target x".

            Thanks :-)

             
            download failed.txt

            • Hendry Kaak

              Hendry Kaak - 2020-10-21

              Console output of the download that went well.

               
              download ok.txt

            • Tormod Volden

              Tormod Volden - 2020-10-21

              depending on the sequence of the build parameters it will always pick the last alternateSetting in the command line as the alternateSetting for the whole target suffix (without warning the user). If this is the case, it should probably create a new target with different alternateSetting

              This should be implemented already. Maybe I don't understand the description. It is confusing because the terminology in dfuse-pack.py does not match ST's document 100%.

              If your image elements are listed with alternate interface numbers in e.g. the order @1 @1 @2 it will make a DfuSe file with two DFU images, the first having two image elements. If the order is @1 @2 @1 you will get three DFU images.

               

            • Tormod Volden

              Tormod Volden - 2020-10-21

              Maybe it's better to reword "file contains 1 DFU images" as "File contains x target(s)". And "parsing DFU image x" as "Parsing image element x" or even better would be to display the current target index with the image element that is being parsed at the time as in "Parsing image element x from target x".

              The word "target" is only used internally in the script code. I think formally none of the parts of a DfuSe file is a "target", but each "DFU image" part has a target attribute, specifying for which alternate interface it is targeted.

              EDIT: Yes, displaying the target (alternate interface) number for each DFU image would make sense.

               

              Last edit: Tormod Volden 2020-10-21
1 2 > >> (Page 1 of 2)

Anonymous
Anonymous

Add attachments
Cancel