Upgrade Failing EM7565, Linux, 01.05.03.00 to 01.07.02.00


#1

Relatively new EM7565, purchased through Future (authorized dealer), operational on as-delivered

Revision: SWI9X50C_01.05.03.00 23e1da jenkins 2018/03/28 22:42:08

As I was having problems getting a qmicli script to be able to retrieve both the IPv4 and IPv6 addresses when in raw-ip mode (I can get one or the other, but not both, subject of a later post, perhaps), I tried upgrading the firmware.

TL;DR

In short, neither the “full” or the “lite” SDK’s tool seem able to add/upgrade firmware, only ending up with the original firmware showing in memory. I’ve tried the full SDK, with and without the qmiserial and qmi_wwan kernel modules loaded. I have moved the modem, attached through a USB housing, to a USB 2.0 port on the machine. Machine is running Debian 9.5 “stretch”, fully patched. Hardware is an AMD Ryzen 5 2600X on a Asus ROG Strix B450-F Gaming motherboard, with no history of any USB issues.

Suggestions or requests for additional information, debug output, all welcome!


Firmware and SDK were obtained directly from the Sierra Wireless site, https://source.sierrawireless.com/resources/airprime/minicard/75xx/airprime-em_mc75xx-approved-fw-packages/ and https://source.sierrawireless.com/resources/airprime/software/linux-qmi-sdk-software-latest/

I did not see any MD5/SHA checksums available to confirm the integrity of the downloads.

-rw-r--r-- 1 jeff jeff 246692693 Sep 28 21:05 SLQS04.00.16.bin.tar.gz
-rw-r--r-- 1 jeff jeff  57351701 Sep 28 20:55 SLQS04.00.16-lite.bin.tar.gz
-rw-r--r-- 1 jeff jeff  74070922 Jul  1 16:34 swi9x50c_01.07.02.00_generic_002.004_000.zip
jeff@deb-devel:~/Downloads/sierra-wireless$ sha256sum SLQS*.gz swi9x50c_01.07.02.00_generic_002.004_000.zip 
0d09fc3788610315d4d8c828ac8f0b43c5aaef6d5e6403d2ec0b980e748afeab  SLQS04.00.16.bin.tar.gz
969c9330a8071a30bb727a5086f8d1c0074084b4fddc16002eb9e2d9baaaa607  SLQS04.00.16-lite.bin.tar.gz
e71f8aaaddd20bc00f3739b15223d5b1dffa794c932027db8f2296a5d94ed079  swi9x50c_01.07.02.00_generic_002.004_000.zip

I have re-downloaded the firmware and the SHA256 is the same.

jeff@deb-devel:~/Downloads/sierra-wireless$ ls -l SWI9X50C_01.07.02.00/
total 72677
-rwxr-xr-x 1 jeff jeff 74850426 Jun 25 18:00 SWI9X50C_01.07.02.00.cwe
-rwxr-xr-x 1 jeff jeff    10829 Jun 14 12:52 SWI9X50C_01.07.02.00_GENERIC_002.004_000.nvu

In some cases, the update appeared to “take” very quickly (including the first time), indicating success, contrary to the Appendix A: Firmware Upgrade Instructions from Release 9 and older information. In other cases, after the modem switched to bulk mode, it would spit out periods for minutes or hours, with no apparent progress.

First attempt

jeff@deb-devel:~/Downloads/sierra-wireless/SLQS/SampleApps/Firmware_Download/bin$ ./fwdldhostx86_64 -d 9x50 -p ~/Downloads/sierra-wireless -s /home/jeff/Downloads/sierra-wireless/SLQS/build/bin/hostx86_64/slqssdk
Detecting USB of the target
DONE
Communicating with the target
DONE
Switching to firmware download mode
......DONE
Downloading the firmware
........................................................DONE
Rebooting the module
........DONE
Firmware Download SUCCESS

(no, still running old version and new version not present)

Later attempt:

jeff@deb-devel:~/Downloads/sierra-wireless/SLQS/SampleApps/Firmware_Download/bin$ ./fwdldhostx86_64 -d 9x50 -p ~/Downloads/sierra-wireless -s /home/jeff/Downloads/sierra-wireless/SLQS/build/bin/hostx86_64/slqssdk
Detecting USB of the target
DONE
Communicating with the target
DONE
Downloading the firmware
...........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................^C

(No, I wasn’t sitting there waiting for it! :wink:)

I do have a set of full logs, including output of fwdldhostx86_64 interspersed with the syslog entries. A section that caught my eye because of the two errors around malformed XML, including ERROR: XML not formed correctly. Expected a < character at loc 0 similar to that found in the output of the “lite” run:

[...]
Sep 29 07:42:27 deb-devel SWI0 SDK Process: fmsExtractParameter: File Name: ./SWI9X50C_01.07.02.00//SWI9X50C_01.07.02.00_GENERIC_002.004_000.nvu
Sep 29 07:42:27 deb-devel SWI0 SDK Process: fmsExtractParameter: File Name: ./SWI9X50C_01.07.02.00//SWI9X50C_01.07.02.00.cwe
[...]
Sep 29 07:42:39 deb-devel SWI0 SDK Process: FHIM:Firehose Resp: st:6, ev:6
Sep 29 07:42:39 deb-devel SWI0 SDK Process: FHIM:Writing...
Sep 29 07:42:39 deb-devel SWI0 SDK Process: Last Frame
Sep 29 07:42:39 deb-devel SWI0 SDK Process: FHIM:pad one extra sector only to end transfer
Sep 29 07:42:39 deb-devel SWI0 SDK Process: FHIM:Write Completes
Sep 29 07:42:39 deb-devel SWI0 SDK Process: pad one extra sector only to end transfer
Sep 29 07:42:39 deb-devel SWI0 SDK Process: payload [95] <?xml version="1.0" encoding="UTF-8" ?>#012<data>#012<log value="Finished sector address 0"/>#0
12</data>
Sep 29 07:42:39 deb-devel SWI0 SDK Process: payload [92] <?xml version="1.0" encoding="UTF-8" ?>#012<data>#012<log value="SWI CWE image received"/>#012<
/data>
Sep 29 07:42:39 deb-devel SWI0 SDK Process: payload [98] <?xml version="1.0" encoding="UTF-8" ?>#012<data>#012<log value="SWI CWE image processing ..."/>#012</data>
Sep 29 07:42:40 deb-devel SWI0 SDK Process: payload [191] <?xml version="1.0" encoding="UTF-8" ?>#012<data>#012<log value="SWI CWE image processed OK"/>#012</data><?xml version="1.0" encoding="UTF-8" ?>#012<data>#012<response value="ACK" rawmode="false" />#012</data>
Sep 29 07:42:40 deb-devel SWI0 SDK Process: FHIM:Firehose Resp: st:7, ev:7
Sep 29 07:42:40 deb-devel SWI0 SDK Process: FHIM:Send power XML, st:8
Sep 29 07:42:40 deb-devel SWI0 SDK Process: payload [137] <?xml version="1.0" encoding="UTF-8" ?>#012<data>#012<log value="ERROR: XML not formed correctly. Expected a &lt; character at loc 0"/>#012</data>
Sep 29 07:42:40 deb-devel SWI0 SDK Process: FHIM:Send power XML, st:8
Sep 29 07:42:40 deb-devel SWI0 SDK Process: XML error recieved, tick state machine
Sep 29 07:42:40 deb-devel SWI0 SDK Process: payload [79] <?xml version="1.0" encoding="UTF-8" ?>#012<data>#012<response value="NAK" />#012</data>
Sep 29 07:42:40 deb-devel SWI0 SDK Process: Wait 50 Sec if no message received from modem before resend reset
Sep 29 07:42:40 deb-devel SWI0 SDK Process: payload [137] <?xml version="1.0" encoding="UTF-8" ?>#012<data>#012<log value="ERROR: XML not formed correctly. Expected a &lt; character at loc 0"/>#012</data>
Sep 29 07:42:40 deb-devel SWI0 SDK Process: payload [79] <?xml version="1.0" encoding="UTF-8" ?>#012<data>#012<response value="NAK" />#012</data>
Sep 29 07:42:40 deb-devel SWI0 SDK Process: payload [114] <?xml version="1.0" encoding="UTF-8" ?>#012<data>#012<log value="Inside handlePower() - Requested POWER_RESET"/>#012</data>
Sep 29 07:42:40 deb-devel SWI0 SDK Process: payload [79] <?xml version="1.0" encoding="UTF-8" ?>#012<data>#012<response value="ACK" />#012</data>
Sep 29 07:42:41 deb-devel SWI0 SDK Process: sleep(1)
Sep 29 07:42:41 deb-devel SWI0 SDK Process: FHIM:Send power XML, st:8
Sep 29 07:42:41 deb-devel SWI0 SDK Process: XML error recieved, tick state machine
Sep 29 07:42:41 deb-devel SWI0 SDK Process: Wait 50 Sec if no message received from modem before resend reset
Sep 29 07:42:41 deb-devel kernel: [ 3283.977450] usb 1-5: USB disconnect, device number 13
Sep 29 07:42:41 deb-devel SWI0 SDK Process: USB read unexpected EOF error
Sep 29 07:42:41 deb-devel SWI0 SDK Process: swi_sdkusbioerrorunlocked
Sep 29 07:42:41 deb-devel SWI0 SDK Process: 2624/swi_ossdkusbopen: osusbstate: 0
DONE
Rebooting the module

I found a post here suggesting https://github.com/danielewood/sierra-wireless-modems/blob/master/Sierra-Linux-QMI-SDK.md#flash-using-sierra-wireless-linux-flashing-tool-fwdwl-lite as a suitable reference for flashing. From that description I created the “lite script”, adding --dmreset as suggested to resolve the initial Error in modem crash state checking!error code = 103 condition.

The “lite script” based on the referenced github instructions

#!/bin/sh

devpath=`ls /dev | grep -i -E 'cdc-wdm|qcqmi'`
devtype=`expr "$devpath" : '\(cdc-wdm\|qcqmi\)[0-9]$'`
case $devtype in
    cdc-wdm) devtype="MBIM" ;;
    qcqmi) devtype="QMI" ;;
    *) printf "Unknown Device Type = $devtype\r\nDevice Path = /dev/$devpath\r\n"; exit
esac
printf "Device Type = $devtype\r\nDevice Path = /dev/$devpath\r\n"

./fwdwl-litehostx86_64 \
    --devmode $devtype  \
    --devpath /dev/$devpath \
    --modelfamily 4 \
    --logfile "fwdwl-lite-$devpath.log" \
    -e 1 \
    --fwpath "./swi_fwlatest/" \
    --dmreset

Output of the “lite script”

jeff@deb-devel:~/Downloads/sierra-wireless$ sh upgrade.full.sh 
Device Type = MBIM
Device Path = /dev/cdc-wdm0
litefw version :lite-fwSLQS04.00.16 
INFO: QDL Port  : /dev/ttyUSB0
INFO: Device Path: /dev/cdc-wdm0
INFO: FW  Path  : ./swi_fwlatest/
Package Info:
Carrier    :GENERIC
FW Version :01.07.02.00
Model ID   :SWI9X50C
Package ID :000
PRI Version:002.004
SKU        :9999999
file_type : 6
Switch Modem to QDL mode.
Waiting for modem to come up in BOOT and HOLD mode ...
BOOT and HOLD Mode. Downloading firmware ...
iFWImageType:6
eFW_TYPE_CWE_NVU
Tx:
Tx:
Tx:
Tx:<?xml version="1.0" encoding="UTF-8" ?>
<data>
<configure MemoryName="eMMC" Verbose="0" AlwaysValidate="0" MaxDigestTableSizeInBytes="8192" MaxPayloadSizeToTargetInBytes="16384" ZlpAwareHost="1" SkipStorageInit="0" TargetName="8960" />
</data>

Tx:<?xml version="1.0" encoding="UTF-8" ?>
<data>
<configure MemoryName="eMMC" Verbose="0" AlwaysValidate="0" MaxDigestTableSizeInBytes="8192" MaxPayloadSizeToTargetInBytes="16384" ZlpAwareHost="1" SkipStorageInit="0" TargetName="8960" />
</data>

<?xml version="1.0" encoding="UTF-8" ?>
<data>
<program PAGES_PER_BLOCK="64" SECTOR_SIZE_IN_BYTES="4096" filename="spkg.cwe" num_partition_sectors="3" physical_partition_number="0" start_sector="-1"  />
</data>

<?xml version="1.0" encoding="UTF-8" ?>
<data>
<program PAGES_PER_BLOCK="64" SECTOR_SIZE_IN_BYTES="4096" filename="spkg.cwe" num_partition_sectors="3" physical_partition_number="0" start_sector="-1"  />
</data>

Write:./swi_fwlatest//SWI9X50C_01.07.02.00_GENERIC_002.004_000.nvu
RESET resp:<?xml version="1.0" encoding="UTF-8" ?>
<data>
<response value="NAK" />
</data>d correctly. Expected a &lt; character at loc 0"/>
</data>
litefw_DownloadFW succeeded.
Waiting for modem to come up in ONLINE mode ...
FWVersionAfterDownload_MBIM: mbim fd<0
Firmware Download Process completed
Exiting Application!!!

Current status after running the “lite script”:

at
OK
ati
Manufacturer: Sierra Wireless, Incorporated
Model: EM7565
Revision: SWI9X50C_01.05.03.00 23e1da jenkins 2018/03/28 22:42:08
IMEI: <redacted>
IMEI SV:  5
FSN: UF818592720106
+GCAP: +CGSM


OK
at!gstatus?
!GSTATUS: 
Current Time:  80               Temperature: 30
Reset Counter: 3                Mode:        ONLINE         
System mode:   LTE              PS state:    Attached     
LTE band:      B4               LTE bw:      20 MHz  
LTE Rx chan:   2300             LTE Tx chan: 20300
LTE SSC1 state:NOT ASSIGNED
LTE SSC2 state:NOT ASSIGNED
LTE SSC3 state:NOT ASSIGNED
LTE SSC4 state:NOT ASSIGNED
EMM state:     Registered       Normal Service 
RRC state:     RRC Idle       
IMS reg state: No Srv   

PCC RxM RSSI:  -77              PCC RxM RSRP:  -112
PCC RxD RSSI:  -77              PCC RxD RSRP:  -112
Tx Power:      --               TAC:         3875 (14453)
RSRQ (dB):     -14.9            Cell ID:     01626b01 (23227137)
SINR (dB):     -7.2


OK
at!entercnd="A710"
OK
at!pcinfo?
State: Online
LPM voters - Temp:0, Volt:0, User:0, W_DISABLE:0, IMSWITCH:0, BIOS:0, LWM2M:0, OMADM:0, FOTA:0, RFCAL:0
LPM persistence - None

OK
at!priid?
PRI Part Number: 9907344
Revision: 001.004
Customer: Generic-M2M

Carrier PRI: 9999999_9907259_SWI9X50C_01.05.03.00_00_GENERIC_001.032_000

OK
at!image?
TYPE SLOT STATUS LRU FAILURES UNIQUE_ID   BUILD_ID
FW   1    GOOD   1   0 0      ?_?         01.05.03.00_?
FW   2    EMPTY  0   0 0                  
FW   3    EMPTY  0   0 0                  
FW   4    EMPTY  0   0 0                  
Max FW images: 4
Active FW image is at slot 1

TYPE SLOT STATUS LRU FAILURES UNIQUE_ID   BUILD_ID
PRI  FF   GOOD   0   0 0      001.032_000 01.05.03.00_GENERIC
Max PRI images: 50


OK

#2

See this bit of documentation:
(You basically need to flash it twice.)

image


#3

Thanks @josephmarsden, those were the instructions I was working from and the Appendix of the release notes that I referred to. I did again try back-to-back firmware updates, but the results are the same – the firmware version reported as running and the only one present on the device is still SWI9X50C_01.05.03.00

I did not downgrade to SWI9X50C_01.05.01.00_00_GENERIC_001.028_000.zip as the device is already on Release 9, and that 01.05.01 is Release 8, both according to SWI9X50C-CRN-R01.10.pdf. I am always hesitant to blindly downgrade a device, especially as the Sierra Wireless documentation does not indicate that a downgrade from Release 9 to Release 8 is required to flash one of the “signed” versions, such as Release 11.

The errors variously described as

still occur, with the “lite” firmware tool indicating

[...]
Write:./swi_fwlatest//SWI9X50C_01.07.02.00_GENERIC_002.004_000.nvu
RESET resp:<?xml version="1.0" encoding="UTF-8" ?>
<data>
<response value="NAK" />
</data>d correctly. Expected a &lt; character at loc 0"/>
</data>
litefw_DownloadFW succeeded.
Waiting for modem to come up in ONLINE mode ...
mbim open respons type error  -2147480831
Firmware Download Process completed
Exiting Application!!!

for both runs.

Prior to the runs, I again confirmed that both ModemManager are NetworkManager are not activated and not running.

A third run has the same output and the same result – still on the older firmware and no other images in the device’s storage.


#4

If you have access to a Windows machine, you could try flashing the firmware with the Windows tool - this is what ultimately worked for me when I was upgrading my EM7565 from this old firmware version. I remember I ran into some weird errors while trying to upgrade with litefw.

Try running the Windows upgrade tool twice, and if this fails, you could try flashing the Windows 01.05.03.00 and then the Windows 01.07.02.00 twice.


#5

No Windows here, macOS, FreeBSD, various Linux flavors only.

I saw use of Windows in


and a few other threads, but that is not an option.

That thread is why I moved to a USB 2 port, as suggested by