MC7421 QmiDevice: Transaction timed out

IoT Modules MC/EM Series
1

HI
We have a unit of 100 MC7421 devices and we suffer from their bad behavior, most of the time when the board restarts or turns on/off the module will disappear or the qmi command fails to complete the transaction and we have to restart the board a lot of time to make it work
we tried the MBPL_DRIVERS_R32 driver and the generic driver and both gave the same result
so our question did you have any idea how to resolve this the issue

below the module didn’t show up after rebootand
root@test201:/dev# ls
bus loop-control loop7 mtd3 mtd7 mtdblock4 ptmx ttyMSM0 ubi0_2 zero
console loop0 mesh_detect mtd3ro mtd7ro mtdblock5 pts ttyMSM1 ubi0_3
cpu_dma_latency loop1 mtd0 mtd4 mtd8 mtdblock6 random ttyS0 ubi0_4
full loop2 mtd0ro mtd4ro mtd8ro mtdblock7 shm ttyS1 ubi_ctrl
gpiochip0 loop3 mtd1 mtd5 mtdblock0 mtdblock8 stderr ttyUSB1 ubiblock0_1
hwrng loop4 mtd1ro mtd5ro mtdblock1 net stdin ubi0 urandom
kmsg loop5 mtd2 mtd6 mtdblock2 null stdout ubi0_0 watchdog
log loop6 mtd2ro mtd6ro mtdblock3 port tty ubi0_1 watchdog0

here the log of crash dump data
root@test201:~# atexecute ttyUSB2 “AT!GCDUMP”
AT!GCDUMP

Src: FatalError
File: ds_rmnet_meta_sm.c
Line: 5285
Str: Assertion 0 failed
00000000 00000000 00000000 00000000
Prc: MPSS
Task: AMSS
Time: 00014055
R0: 00000000 R1: C0046130 R2: EB102000 R3: CEE818E0 R4: 000000FE
R5: 00000000 R6: 000000E9 R7: 00000001 R8: A55A6BB6 R9: 0000001F
R10: 00000000 R11: 00000002 R12: C0053E20 R13: 00000000 R14: C1564DE0
R15: 00000013 R16: C52488EA R17: 00000000 R18: 00000000 R19: 00000000
R20: C004601C R21: 00000000 R22: C3F8C794 R23: C3F8C798 R24: 5D4B0EA5
R25: 00000000 R26: D00DFEED R27: DEADBEEF R28: CEE7DFE8 SP: CEE81430
FP: CEE81438 LR: C009F27C
PC: 2FD61E56
CPSR: 00000000
Mod: Unknown
TOS
CEE81458 2FD61F62 00000000 00000000 C4A59A40 01000000 C52488EA
00000000 CEE81468 2FD61E56 C12D4A38 00000000 CEE81470 2EF2A74E
CEE814B0 2F5B7A4A CEE814B0 C3F8C790 CEE818E0 C15687DC 00000000
00000000 00000000 00000000 0000005A 0000007F C49F9E28 C0849684
C213DA1C 00000000 CEE81550 2EEE956E
BOS
App ver: SWI9X50C_01.14.03.00

OK

2

It seems there is crash problem in your board
Have you tried some newer firmware?

3

you mean MC7421 firmware
no I use
root@test201:/dev# atexecute ttyUSB2 “AT!PRIID?”
AT!PRIID?
PRI Part Number: 9909877
Revision: 002.004
Customer: Generic-M2M

Carrier PRI: 9999999_9907259_SWI9X50C_01.14.03.00_00_GENERIC_002.036_010

OK

it’s recommanded to upgrade it to fix the isue ???

4

I don’t know
Just have a trial and see if there is any luck

5

according to this link
https://source.sierrawireless.com/resources/airprime/minicard/74xx/em_mc74x1%20approved%20fw%20packages/#sthash.WOSfE3pz.dpbs

Can I upgrade it to GCF Approved— Release 20 ??

6

you can have a try on that

7

Do you have a doc that describe the upgrade procedures
and can I use openwrt to upgrade it or only linux /windows os
because we have many routers already installed and we can’t unplug the module again

8

I have never used openwrt, but you can have a look on here:

9

I tried to use airvantage fota to upgrade the firmware but I can’t see the MC7421 in drop list devices
you know how to add it or it’s not yet supported ???

10

how about " Register AirPrime EM75 series"

11

I didn’t understand you mean I should register MC7421 as EM75 model???

on the other hand I used the command line to upgrade the firmware to
Generic / SWI9X50C_01.14.13.00 /002.048_000

root@test201:~# atexecute ttyUSB2 “AT+GMR”
AT+GMR
SWI9X50C_01.14.13.00 883709 jenkins 2022/01/05 07:08:35

OK
root@test201:~# atexecute ttyUSB2 “AT!PRIID?”
AT!PRIID?
PRI Part Number: 9909877
Revision: 002.004
Customer: Generic-M2M

Carrier PRI: None

OK
root@test201:~#

But I met a big problem the module can’t connect to the LTE service and shows that the RSSI=-128dbm

root@test201:~# qmicli --device=/dev/cdc-wdm0 --device-open-proxy --nas-get-signal-strength
[/dev/cdc-wdm0] Successfully got signal strength
Current:
Network ‘none’: ‘-128 dBm’
RSSI:
Network ‘none’: ‘-128 dBm’
ECIO:
Network ‘none’: ‘-2.5 dBm’
IO: ‘-106 dBm’
SINR (8): ‘9.0 dB’
root@test201:~#
root@test201:~# qmicli --device=/dev/cdc-wdm0 --device-open-proxy --nas-get-cell-location-info
error: Unknown option --nas-get-cell-location-info
root@test201:~# qmicli --device=/dev/cdc-wdm0 --device-open-proxy --nas-get-operator-name
error: Unknown option --nas-get-operator-name
root@test201:~# qmicli --device=/dev/cdc-wdm0 --device-open-proxy --wds-get-autoconnect-settings
Autoconnect settings retrieved:
Status: ‘enabled’
Roaming: ‘allowed’
root@test201:~# qmicli --device=/dev/cdc-wdm0 --device-open-proxy --wds-get-current-settings
error: couldn’t get current settings: QMI protocol error (15): ‘OutOfCall’
root@test201:~#

what’s the wrong now how to resolve this

12

Maybe there is antenna problem

13

I got the issue I used generic instead of GENERIC
now it works and below list of command for who want to upgrade with openwrt

qmicli -d /dev/cdc-wdm0 -p --dms-set-firmware-preference=“firmware-version=01.14.13.00,config-version=002.048_000,carrier=GENERIC”
AT!BOOTHOLD (wait for /dev/cdc-wdm0 to disappear)
qmi-firmware-update -t “/dev/ttyUSB0” -L upgrade.log --ignore-mm-runtime-check -U /tmp/SWI9X50C_01.14.13.00.cwe /tmp/SWI9X50C_01.14.13.00_GENERIC_002.048_000.nvu

15

hello jyijyi,

I upgraded the firmware as you said

root@test201:/dev# atexecute ttyUSB2 “ATI”
ATI
Manufacturer: Sierra Wireless, Incorporated
Model: MC7421
Revision: SWI9X50C_01.14.13.00 883709 jenkins 2022/01/05 07:08:35
IMEI: 357339340309005
IMEI SV: 14
FSN: 912154221404B1
+GCAP: +CGSM

root@test201:/dev# atexecute ttyUSB2 “at!priid?”
at!priid?
PRI Part Number: 9909877
Revision: 002.048
Customer: GENERIC

Carrier PRI: 9999999_9907259_SWI9X50C_01.14.13.00_00_GENERIC_002.048_000

OK

but module crashes still here and after 100 reboots I collected the following results

1- module interfaces not created (13/100)

only ttyUSB1 appears (not functional) and hence Os couldn’t communicate with MC7421 either by qmi or serial (17/100 )

root@test201:/dev# ls
bus loop-control loop7 mtd3 mtd7 mtdblock4 ptmx ttyMSM0 ubi0_2 zero
console loop0 mesh_detect mtd3ro mtd7ro mtdblock5 pts ttyMSM1 ubi0_3
cpu_dma_latency loop1 mtd0 mtd4 mtd8 mtdblock6 random ttyS0 ubi0_4
full loop2 mtd0ro mtd4ro mtd8ro mtdblock7 shm ttyS1 ubi_ctrl
gpiochip0 loop3 mtd1 mtd5 mtdblock0 mtdblock8 stderr ttyUSB1 ubiblock0_1
hwrng loop4 mtd1ro mtd5ro mtdblock1 net stdin ubi0 urandom
kmsg loop5 mtd2 mtd6 mtdblock2 null stdout ubi0_0 watchdog
log loop6 mtd2ro mtd6ro mtdblock3 port tty ubi0_1 watchdog0

after reboot

root@test201:/dev# atexecute ttyUSB2 “AT!GCDUMP”
AT!GCDUMP

Src: FatalError
File: ds_rmnet_meta_sm.c
Line: 5285
Str: Assertion 0 failed
00000000 00000000 00000000 00000000
Prc: MPSS
Task: AMSS
Time: 000137C3
R0: 00000000 R1: C0046130 R2: EB102000 R3: CE8828E0 R4: 000000FE
R5: 00000000 R6: 00000025 R7: 00000001 R8: A55A6BB6 R9: 0000001F
R10: 00000000 R11: 00000004 R12: C0053E20 R13: 00000000 R14: C1565DC0
R15: 00000013 R16: C52488EA R17: 00000000 R18: 00000000 R19: 00000000
R20: C004601C R21: 00000000 R22: C3F8C794 R23: C3F8C798 R24: 5AC88526
R25: 00000000 R26: D00DFEED R27: DEADBEEF R28: CE87EFE8 SP: CE882430
FP: CE882438 LR: C009F27C
PC: 31A2AA87
CPSR: 00000000
Mod: Unknown
TOS
CE882458 31A2ABB3 00000000 00000000 C4A59F48 01000000 C52488EA
00000000 CE882468 31A2AA87 C12D5978 00000000 CE882470 3086005F
CE8824B0 312FCE9B CE8824B0 C3F8C790 CE8828E0 C15697BC 00000000
00000000 00000000 00000000 0000005A 0000007F C49F9E28 C0849684
C213DA8C 00000000 CE882550 309ADE7F
BOS
App ver: SWI9X50C_01.14.13.00

OK
root@test201:/dev#
root@test201:/dev# ls
bus log loop6 mtd2ro mtd6ro mtdblock3 port tty ubi0 urandom
cdc-wdm0 loop-control loop7 mtd3 mtd7 mtdblock4 ptmx ttyMSM0 ubi0_0 watchdog
console loop0 mesh_detect mtd3ro mtd7ro mtdblock5 pts ttyMSM1 ubi0_1 watchdog0
cpu_dma_latency loop1 mtd0 mtd4 mtd8 mtdblock6 random ttyS0 ubi0_2 zero
full loop2 mtd0ro mtd4ro mtd8ro mtdblock7 shm ttyS1 ubi0_3
gpiochip0 loop3 mtd1 mtd5 mtdblock0 mtdblock8 stderr ttyUSB0 ubi0_4
hwrng loop4 mtd1ro mtd5ro mtdblock1 net stdin ttyUSB1 ubi_ctrl
kmsg loop5 mtd2 mtd6 mtdblock2 null stdout ttyUSB2 ubiblock0_1

2- module hangs during operation (15/100)
the Os logged the following error
NETDEV WATCHDOG: wwan0 (qmi_wwan): transmit queue 0 timed out

3-QMI time out (31/100)

root@test201:/tmp# cat qmicli_error
error: operation failed: Transaction timed out
root@test201:/tmp# atexecute ttyUSB2 “AT!GCDUMP”
AT!GCDUMP

Src: FatalError
File: sys_m_smsm_mpss.c
Line: 285
Str: APPS force stop
00000000 00000000 00000000 00000000
Prc: MPSS
Task: AMSS
Time: 00085792
R0: 00000000 R1: C0046130 R2: EB102000 R3: 4FDE91A0 R4: 000000FE
R5: C3772C74 R6: 00000001 R7: 00000001 R8: C00A8770 R9: 018F0045
R10: 00000000 R11: C0110960 R12: C51B5000 R13: 4FDE91A0 R14: 719EDB61
R15: 00000186 R16: C51C61B2 R17: 00000000 R18: 00000000 R19: 00000000
R20: C004601C R21: 0000005A R22: 0000007F R23: 00000000 R24: 00000000
R25: 00000000 R26: 4FDE91A0 R27: C15697BC R28: 4FDA3430 SP: 4FDE8C18
FP: 4FDE8C20 LR: C009F27C
PC: 3900E2B1
CPSR: 00000000
Mod: Unknown
TOS
4FDE8C40 3900E385 C0791580 0000005A 00000002 C49F9E28 C51C61B2
C51BC2B8 4FDE8C50 3900E2B1 C51C6194 C51BC2B8 4FDE8C70 397006C9
00000000 00000001 00000000 C49F9E28 C213CF84 FFFFFFFF 4FDE8D18
38386DB5 00000000 0631340D C213CF68 00000000 C213CF84 FFFFFFFF
00000000 C49F9E28 C0791580 0000005A
BOS
App ver: SWI9X50C_01.14.13.00

Src: FatalError
Str: Recursive fault!

00000000 00000000 00000000 00000000
Prc: APSS
Task: C50F8A80
Time: 00000000
R0: 00000000 R1: 00000000 R2: 00000000 R3: 00000000 R4: 00000000
R5: 00000000 R6: 00000000 R7: 00000000 R8: 00000000 R9: 00000000
R10: 00000000 R11: C50FFEBC R12: 00000000 R13: C50FFE90 R14: C0011DC4
PC: C001483C
CPSR: 00000000
Mod: Unknown
TOS
C0011DC4 88BC8594 C001F1E8 00000023 C07BF984 C0E124F8 C0A1D204
00000000 001190D8 C50FFF88 C001F298 00000000 00000023 C0A18DB4
C07BF984 00000001 C1067DF8 00000000 C07DD134 C50FFF88 00000001
C0E9508C C65FB100 00000001 C013A994 C50FFF88 001190D8 00000002
00000002 00000002 C5B246C0 001190D8
BOS

OK

→ according to to those results The module failed to operate except for 41 cases

do you have any suggestion to overcome this issue

16

how to reproduce the crash?
BTW, you need to type AT!GCCLR to clear the AT!GCDUMP information before the stress test.
Do you see problem if no SIM card is inserted?

17

I wrote a script that reboots the board every 5 minutes
yes the cache was cleared before reboot

Do you see problem if no SIM card is inserted?
I will check that

FYI :
I did this test with NL668 from Fibocom and the module works well all the time
the mother board is DR40x9 from wallystech
openwrt 22.03
kernel 5.10.168

18

just want to see if the crash is related to the network

BTW, how do you reboot the module?
using AT!RESET?

19

I use Hardware reset pin 22, because with serial interface doesn’t work in some cases

20

You might need to check the power down sequence in PTS

21

I repeated the test without SIM card and over 60 iteration I never saw neither a qmi error or crash data
root@test201:~# qmicli --device=/dev/cdc-wdm0 --device-open-proxy --uim-get-card-status
[/dev/cdc-wdm0] Successfully got card status
Provisioning applications:
Primary GW: session doesn’t exist
Primary 1X: session doesn’t exist
Secondary GW: session doesn’t exist
Secondary 1X: session doesn’t exist
Slot [1]:
Card state: ‘error: no-atr-received (3)’
UPIN state: ‘not-initialized’
UPIN retries: ‘0’
UPUK retries: ‘0’
Slot [2]:
Card state: ‘absent’
UPIN state: ‘not-initialized’
UPIN retries: ‘0’
UPUK retries: ‘0’
root@test201:~# atexecute ttyUSB2 “AT!GCDUMP”
AT!GCDUMP
No crash data available

OK
root@test201:~# cat /tmp/qmicli_error
root@test201:~#

what’s the next step