MC7455 random module reset

IoT Modules MC/EM Series
1

I am developing an embedded system using the MC7455 mounted to an i.MX 6 board (the Digi ConnectCore 6UL SBC Pro, https://www.digi.com/products/embedded-systems/single-board-computers/connectcore-for-i-mx6ul-sbc-pro) running Digi Embedded Yocto 4.14, and I am having a stability problem. I am interested in narrowing down where the issue might be, and I am hoping someone here can point me in the correct direction.

The problem seems to happen more often when sending/receiving data. When pinging google.com, for example, I will see many pings succeed and then suddenly, pinging stops. Immediately after this, if I run
‘mmcli -L’ it says “No modems were found”. Shortly thereafter, the modem appears once again but has an ID number that has incremented. When I view the kernel/dmesg log, I see a disconnect followed by a re-connect. I have supplied the kernel log below. This happens repeatedly: if it disconnects, I wait for it to re-appear and re-connect to the cell network before disappearing again. Each time, I see the same errors in the dmesg log.

Things I have checked already:

  • I am using a lab bench power supply that is capable of supplying 30V/20A with the current limit set to 2A (at 5V) and the current limit has so far not been reached. The most I’ve seen is approximately 400mA. From the documentation, it appears that the max expected current draw for this device is 1.1A.
  • Does this happen with one modem only? No: there are at least 2 other identical test systems that exhibit the same behaviour.
  • Is it a firmware problem? I have tried this with both the default factory firmware from 2017 as well as the newest (SWI9X30C_02.33.03.00) and I see the same behaviour.
  • I originally saw this happening with QMI and tried switching to MBIM with no change. I have since switched back to QMI.
  • This forum post at LTEhacks (http://ltehacks.com/viewtopic.php?t=306) seems to be very similar to my issue and so far there are no obvious solutions as of today (February 2020.)

I hope this is enough to go on - otherwise I am happy to provide more information as needed.

dmesg:

usb 1-1.3: USB disconnect, device number 4
qcserial ttyUSB0: Qualcomm USB modem converter now disconnected from ttyUSB0
qcserial 1-1.3:1.0: device disconnected
qcserial ttyUSB1: Qualcomm USB modem converter now disconnected from ttyUSB1
qcserial 1-1.3:1.2: device disconnected
qcserial ttyUSB2: Qualcomm USB modem converter now disconnected from ttyUSB2
qcserial 1-1.3:1.3: device disconnected
qmi_wwan 1-1.3:1.8 wwan0: unregister 'qmi_wwan' usb-ci_hdrc.1-1.3, WWAN/QMI device
usb 1-1.3: new high-speed USB device number 5 using ci_hdrc
usb 1-1.3: config 1 has an invalid interface number: 8 but max is 3
usb 1-1.3: config 1 has no interface number 1
qcserial 1-1.3:1.0: Qualcomm USB modem converter detected
usb 1-1.3: Qualcomm USB modem converter now attached to ttyUSB0
qcserial 1-1.3:1.2: Qualcomm USB modem converter detected
usb 1-1.3: Qualcomm USB modem converter now attached to ttyUSB1
qcserial 1-1.3:1.3: Qualcomm USB modem converter detected
usb 1-1.3: Qualcomm USB modem converter now attached to ttyUSB2
qmi_wwan 1-1.3:1.8: cdc-wdm0: USB WDM device
qmi_wwan 1-1.3:1.8 wwan0: register 'qmi_wwan' at usb-ci_hdrc.1-1.3, WWAN/QMI device, 2a:59:a1:ad:4d:db

uname -r

4.14.141-dey+g0f5a740ab5b8

ati

Manufacturer: Sierra Wireless, Incorporated
Model: MC7455
Revision: SWI9X30C_02.33.03.00 r8209 CARMD-EV-FRMWR2 2019/08/28 20:59:30
MEID: <removed>
IMEI: <removed>
IMEI SV: 20
FSN: LQ930446730310
+GCAP: +CGSM

at!gstatus?

!GSTATUS:
Current Time:  252              Temperature: 35
Reset Counter: 8                Mode:        ONLINE
System mode:   WCDMA            PS state:    Attached
WCDMA band:    WCDMA 1900
WCDMA channel: 662
GMM (PS) state:REGISTERED       NORMAL SERVICE
MM (CS) state: IDLE             NORMAL SERVICE

WCDMA L1 state:L1M_PCH_SLEEP    LAC:           AFCB (45003)
RRC state:   DISCONNECTED       UTRAN Cell ID: 0DC1F390 (230814608)
RxM RSSI C0:   -101             RxD RSSI C0:  ---
RxM RSSI C1:    ---             RxD RSSI C1:  ---

mmcli -m 1

  --------------------------------
  General  |            dbus path: /org/freedesktop/ModemManager1/Modem/1
           |            device id: 09dd5c461ce9b9fbacce9d8819210a50a67de993
  --------------------------------
  Hardware |         manufacturer: Sierra Wireless, Incorporated
           |                model: MC7455
           |    firmware revision: SWI9X30C_02.33.03.00 r8209 CARMD-EV-FRMWR2 2019/08/28 20:59:30
           |         h/w revision: 1.0
           |            supported: gsm-umts, lte
           |              current: gsm-umts, lte
           |         equipment id: <removed>
  --------------------------------
  System   |               device: /sys/devices/soc0/soc/2100000.aips-bus/2184200.usb/ci_hdrc.1/usb1/1-1/1-1.3
           |              drivers: qmi_wwan, qcserial
           |               plugin: Sierra
           |         primary port: cdc-wdm0
           |                ports: ttyUSB1 (gps), ttyUSB2 (at), cdc-wdm0 (qmi), wwan0 (net)
  --------------------------------
  Status   |                 lock: sim-pin2
           |       unlock retries: sim-pin (3), sim-pin2 (10), sim-puk (10), sim-puk2 (10)
           |                state: connected
           |          power state: on
           |          access tech: umts
           |       signal quality: 21% (recent)
  --------------------------------
  Modes    |            supported: allowed: 3g; preferred: none
           |                       allowed: 4g; preferred: none
           |                       allowed: 3g, 4g; preferred: 3g
           |                       allowed: 3g, 4g; preferred: 4g
           |              current: allowed: 3g, 4g; preferred: 4g
  --------------------------------
  Bands    |            supported: utran-1, utran-3, utran-4, utran-5, utran-8, utran-2,
           |                       eutran-1, eutran-2, eutran-3, eutran-4, eutran-5, eutran-7, eutran-8,
           |                       eutran-12, eutran-13, eutran-20, eutran-25, eutran-41
           |              current: utran-1, utran-3, utran-4, utran-5, utran-8, utran-2,
           |                       eutran-1, eutran-2, eutran-3, eutran-4, eutran-5, eutran-7, eutran-8,
           |                       eutran-12, eutran-13, eutran-20, eutran-25, eutran-41
  --------------------------------
  IP       |            supported: ipv4, ipv6, ipv4v6
  --------------------------------
  3GPP     |                 imei: <removed>
           |          operator id: 302220
           |        operator name: TELUS
           |         registration: home
  --------------------------------
  3GPP EPS | ue mode of operation: csps-2
  --------------------------------
  SIM      |            dbus path: /org/freedesktop/ModemManager1/SIM/1
  --------------------------------
  Bearer   |            dbus path: /org/freedesktop/ModemManager1/Bearer/1
1 Like
2

Hi @csalts

Please use the latest SDK and driver from Sierra at here:

https://source.sierrawireless.com/resources/airprime/software/linux-qmi-sdk-software-latest/#sthash.ln6QitWe.dpbs
https://source.sierrawireless.com/resources/airprime/software/usb-drivers-linux-qmi-software-s2,-d-,39n2,-d-,60/

If you still see issue after using the latest SDK and driver, Please contact your reseller or distributor directly for technical support.

It is a known issue. Sierra is aware of this issue.

3

You have done your homework wrt power supply issues :slight_smile: But this might still be the root cause here. I assume you have connected the MC7455 to the mini-PCIe slot on the i.MX 6 board? This means that you are using the 5V to 3.3V DC-DC converter on the board. This might be the limiting factor. Are you sure this converter is capable of delivering the 2.5 A maximum inrush current as specified in table 5.3 in the PTS?

An interesting test would be using the same modem and host board, but connected to one of the USB ports with an adapter instead. This would use the DC-DC converter on the adapter board instead.

4

Thank you, Donald, this looks like it’s working.

Swapping out qmi_wwan and qcserial for the GobiNet and GobiSerial drivers dramatically improves stability. While the in-tree kernel drivers were convenient and “worked” out-of-the-box, GobiNet and GobiSerial do appear to work more consistently and don’t suffer from periodic module resets. To be honest, I was hoping that the in-tree kernel drivers would be sufficient but it looks like in this case, they aren’t.

5

We appreciate the suggestion - thank you. We tried this out and even with a USB-PCIe adapter, we saw the same periodic re-setting behaviour. For the record, the USB adapter we had in stock was this one:

Worth noting: when the modem reset and disappeared from mmcli, the blue light on the USB adapter went off at the same time and came on when the modem re-appeared.

Taking Donald’s advice we tried the Gobi drivers and we no longer see the module resetting and the stability is as we would expect.

We share your concern about the 3.3V regulator being able to supply sufficient current. We will check the 3.3V rail with an oscilloscope to make sure there are no issues. Even though it looks like it’s working, it would be nice to know that this will not be an issue when we deploy at scale.

Thank you again!

6

Glad to hear that you made it work !

I am a little puzzled why this would make a difference, since I don’t see any way the drivers or userspace applications can make the module disconnect from the USB bus except by setting the modem operating mode to an offline or reset state. And I don’t think we do that. The only other alternative I see is that we do something weird an so unexpected that the firmware crashes. But I have a hard time imagining what that could be. The current firmwares are very stable, and the MC/EM7455 modules are very well tested with MM + qmi_wwan. So what makes your setup different?

7

Yes, we’re a bit perturbed also as we were looking forward to using what we also understand to be a stable driver. That being said, Donald did say:

I looked through the recent Sierra driver changelogs and didn’t see anything that resembled my specific issue. I would be curious to know what Sierra knows that you and I don’t.

8

Revisiting this issue, I had the opportunity to upgrade Linux on the target device to 4.14.170, up from 4.14.141. With qmi_wwan, we see things working as expected.

While we did have some improvements with the Gobi drivers, we were still seeing an issue where the USB port provided by the Gobi driver would disappear and re-appear, which looked very similar the the initial issue with qmi_wwan. It was at this time we realized that Digi had updated their Yocto package from 2.6-r2 to 2.6-r3 which included the update to the kernel, of which we took advantage.

9

Hi CSalts,

This is an over year old thread. Thought I would share an Qualcomm Sierra Wirelss experience very simliar to yours.
purchased a cheap Qualcomm/Sierra Wireless USB dongle just to get acllimated with lte modems. Think this like 9600 series chipset FYI
The lte modem worked fine intially plugged in,light went green as expected.
Plugged in sim card was still green. As soon as i activated ModemManger.service in Debian Bullseye the light started continually cycling off,red,green
Long story short pretty sure just wasnt being supplied steady voltage,similar to what you experienced. Even put it on new USB 3.0 hub ,same scenario.
AS you said,almost certain the actual native linux drivers was eating too much resources or not supplying enough?
Also once ModemManager.service was activated when running mmcli-m ## it was as though the sim card could not be read correctly, Never did say ‘sim missing’ but shown sim failed—reason ‘unknown’.
This same USB dongle worked fine rebooting same server into Windows 10 so the dongle itself,dont think is defective.
Hope this may add a little to your theory.