EM7455: LTE connection drops after a while

Hello.

The problem is on a specific box, not sure what is to blame. Other one works correctly.
After system starts, we connect to lte network, get an IP,and it works. After some time, after attempt to restart network service, we notice that LTE can not take IP address, and no rx/tx via it, and modem reports disconnected from network:
at!scact ?
!SCACT: 1,0

If at that moment we again issue a connect command, it starts working again:
at!scact=1.1
OK
at!scact?
!SCACT: 1,1

Using GobiNet drivers, manually set up connection to LTE using AT-commands.
We can’t find obvious reason why network disconnects us. Please advice how to find cause, or to fix it.

Here is status when there is NO traffic:
at!err
QDSP6 ARM9 (not saved)
00 01 seccrypt 00209
01 01 gpstask 04120
02 01 stm 00515
03 01 hdrscmdb 05334
04 01 gsnvif 00855
05 01 rdevmap 01604
06 01 hdridle 00819
07 01 sys_eplm 00691

OK
AT!GSTATUS?
!GSTATUS:
Current Time: 1666 Temperature: 39
Reset Counter: 4 Mode: ONLINE
System mode: LTE PS state: Attached
LTE band: B3 LTE bw: 15 MHz
LTE Rx chan: 1275 LTE Tx chan: 19275
LTE CA state: NOT ASSIGNED
EMM state: Registered Normal Service
RRC state: RRC Idle
IMS reg state: No Srv

PCC RxM RSSI: -61 RSRP (dBm): -91
PCC RxD RSSI: -65 RSRP (dBm): -94
Tx Power: 0 TAC: 08FD (2301)
RSRQ (dB): -10.3 Cell ID: 007D6401 (8217601)
SINR (dB): 3.6

OK

AT!SCACT?
!SCACT: 1,0

OK
ATPI
Manufacturer: Sierra Wireless, Incorporated
Model: EM7455
Revision: SWI9X30C_02.24.05.06 r7040 CARMD-EV-FRMWR2 2017/05/19 06:23:09
MEID: 35907306083970
IMEI: 359073060839705
IMEI SV: 12
FSN: LF815573790210
+GCAP: +CGSM

OK
AT+CGDCONT?
+CGDCONT: 1,“IP”,“wap-gprs.mtel.bg”,“0.0.0.0”,0,0,0,0

Thank you!
martin.

Hi Martin,

The question is whether the connection is dropped between the host & the modem or between the modem and NTWK.

When the connection is dropped check the IP address with:

at+cgcontrdp

+CGCONTRDP: 1,5,default,192.168.2.2,32.1.4.104.32.0.0.1.32.1.4.104.32.0.0.1, 254.128.0.0.0.0.0.0.112.81.31.67.49.189.124.206,8.8.8.8 32.1.72.96.72.96.0.0.0.0.0.0.0.0.136.136,

If the IP is that same as before then you lost the connection between the host & the modem.

Assuming your account doesn’t provide a static IP.

Thanks,

James

Hi James,

So seems problem between modem and network.

Last known IP that lte0 took
lease {
interface “lte0”;
fixed-address 10.68.108.193;
option subnet-mask 255.255.255.252;
option dhcp-lease-time 7200;
option routers 10.68.108.194;
option dhcp-message-type 5;
option dhcp-server-identifier 10.68.108.194;
option domain-name-servers 10.118.50.120,10.118.50.100;
option interface-mtu 1500;
option host-name “fwa-1012vcc400ada14c34”;
renew 3 2018/09/19 22:15:52;
rebind 3 2018/09/19 23:15:13;
expire 3 2018/09/19 23:30:13;
}

at+cgcontrdp
+CGCONTRDP: 1,5,wap-gprs.mtel.bg,10.161.223.13,10.118.50.120,10.118.50.100

What does this mean ? Network gave us new IP and disconnected modem ?

Thanks
Martin.

Hi Martin,

Yes, in a reverse order. modem got disconnected and re-attached to TE, this is when the NTWK assigned a new IP to the modem.
“The problem is on a specific box”
Are all these boxes on the same location?
Could the issue be signal related? although -91,-94dBm are not too bad.
How long does it take for the modem to disconnect?
Could this be an inactivity issue?
All carriers have an inactivity timer. on ATT it is 4hrs for example.
If no data is sent in 4hrs, ATT will send a detach REQ with reattach required flag. so the modem will the call and get a new IP.
maybe the other boxes send some traffic

Hi James,

The two boxes are one next to another. During boot , network service restarts few times, and lte interface gets few times ifup/ifdown. We see the problem only on one of the box, shortly after it boots.
It seems that somehow ifdown/ifup on the lte0 interface causing the problem.
How could we detect that modem is detached, and need to startt another call ?

Thanks.
martin.

Hi Martin,

Try to swap the modem (if you can) and the SIM to see if the detach follows the modem/SIM.
“It seems that somehow ifdown/ifup on the lte0 interface causing the problem”
I don’t think this is the case.
To test this theory, do not establish the data connection between the modem and host. (don’t send AT!SCACT=1)
keep monitoring at+cgcontrdp to see if the IP changes.

“How could we detect that modem is detached, and need to start another call ?”
with the QMI SDK.

Thanks,
James

Hi James.

I did the experiment you advised - without establishing data connection between host and modem.
at+cgcontrdp keeps showing the same, the IP address is constant.

The problem is reproducible on all three devices we tried from the same type - Advantech FWA-1012VC, each one with different operator, and in different location.

After data is established (AT!SCACT=1), after some time it gets disconnected, AT!SCACT? returns “1,0”.
We work with few types of Sierra modems, and only here we see such behavior. Can it be HW or FW issue?

Can you please advice for APIs in SDK , or any example that can be used for reference to design the solution?

Thanks,
martin.

Hi,

When run GobiNet driver with debug…

[root@fwa-1012vcc400ada14c34 ~]# dmesg -w | grep Gobi | grep QMIWDSCallback
[ 35.282531] GobiNet::QMIWDSCallback(2111) Net device link is disconnected
[ 35.346478] GobiNet::QMIWDSCallback(2111) Net device link is disconnected
[ 131.187220] GobiNet::QMIWDSCallback(2111) Net device link is connected <<< here we set up connection
[ 135.315225] GobiNet::QMIWDSCallback(2111) Net device link is disconnected << and here is lost
[ 934.201224] GobiNet::QMIWDSCallback(9835) Net device link is connected << And here we do SCACT=1,1 again, and restoring it.
[ 952.281376] GobiNet::QMIWDSCallback(9835) Net device link is connected
[ 955.289383] GobiNet::QMIWDSCallback(9835) Net device link is connected
[ 957.273397] GobiNet::QMIWDSCallback(9835) Net device link is connected
[ 958.297373] GobiNet::QMIWDSCallback(9835) Net device link is connected
[ 960.281415] GobiNet::QMIWDSCallback(9835) Net device link is connected
[ 962.297430] GobiNet::QMIWDSCallback(9835) Net device link is connected
[ 964.281453] GobiNet::QMIWDSCallback(9835) Net device link is connected
[ 965.273446] GobiNet::QMIWDSCallback(9835) Net device link is connected
[ 970.297493] GobiNet::QMIWDSCallback(9835) Net device link is connected
[ 976.281517] GobiNet::QMIWDSCallback(9835) Net device link is connected
…

Hi Martin,

It is hard to tell why your connection is dropping. Can you post the full log?
Let’s try it with the SDK.

We offer two SDKs. SLQS and Lite
If it is a new design we are recommending SDK lite which is basically a collection of APIs in a library.
SLQS is always running a process in the background.

https://source.sierrawireless.com/resources/airprime/software/linux-qmi-sdk-software-latest/

sudo su
mkdir SLQS04.00.16_customer
cd SLQS04.00.16_customer
tar xvf SLQS04.00.16-lite.bin.tar.gz
cd SampleApps/lite-connection-manager
make
Compiling src/displaysettings.c
obj/hostx86_64/ exists - good.
Compiling src/connectmgr.c
cd bin/

./lite-connectmgrhostx86_64

Please select one of the following options or press to exit:

  1. Start UMTS Data Session
  2. Start LTE Data Session
  3. Start CDMA Data Session
  4. Start RUIM data session
  5. Stop the currently active Data Session
  6. Display all the profiles stored on the device
  7. Display the settings for a particular profile stored on the device
  8. Create a Profile on the device
  9. Modify the settings of an existing profile stored on the device
  10. Delete a profile stored on the device
    Option : 2

ID PDPType IPAddress PrimaryDNS SecondaryDNS Auth ProfileName APNName UserName
1 3 0.0.0.0 0.0.0.0 0.0.0.0 0 default

Please provide a profile id(1-16), or press to exit: 1

  1. IPV4 (default)
  2. IPV6
  3. IPV4V6
    Please select IP family preference for the call, or press to exit: 1
    LTE/CDMA data session started successfully for Profile ID: 1

Please select one of the following options or press to exit:

  1. Start UMTS Data Session
  2. Start LTE Data Session
  3. Start CDMA Data Session
  4. Start RUIM data session
  5. Stop the currently active Data Session
  6. Display all the profiles stored on the device
  7. Display the settings for a particular profile stored on the device
  8. Create a Profile on the device
  9. Modify the settings of an existing profile stored on the device
  10. Delete a profile stored on the device
    Option :

=========== console #2

##find your NTWK device
dmesg |grep GobiNet
…
[406984.181534] GobiNet 3-6:1.8 enp0s20u6i8: renamed from eth0
…

root@sierra-Lenovo-G510:/home/sierra# dhclient enp0s20u6i8
root@sierra-Lenovo-G510:/home/sierra# ifconfig enp0s20u6i8
enp0s20u6i8 Link encap:Ethernet HWaddr 3e:bd:20:44:e5:cc
inet addr:192.168.2.2 Bcast:192.168.2.3 Mask:255.255.255.252
inet6 addr: fe80::3cbd:20ff:fe44:e5cc/64 Scope:Link
UP BROADCAST RUNNING NOARP MULTICAST MTU:2000 Metric:1
RX packets:2 errors:0 dropped:0 overruns:0 frame:0
TX packets:5 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:652 (652.0 B) TX bytes:824 (824.0 B)

root@sierra-Lenovo-G510:/home/sierra# ping 192.168.2.1
PING 192.168.2.1 (192.168.2.1) 56(84) bytes of data.
64 bytes from 192.168.2.1: icmp_seq=1 ttl=64 time=153 ms
64 bytes from 192.168.2.1: icmp_seq=2 ttl=64 time=33.1 ms
64 bytes from 192.168.2.1: icmp_seq=3 ttl=64 time=30.4 ms
^C
— 192.168.2.1 ping statistics —
3 packets transmitted, 3 received, 0% packet loss, time 2001ms
rtt min/avg/max/mdev = 30.493/72.310/153.309/57.285 ms
root@sierra-Lenovo-G510:/home/sierra#

Assuming the modem is already LTE attached to the NTWK an acquired an IP address before the datacall is established.
to check
at+cgcontrdp
+CGCONTRDP: 1,5,default,192.168.2.2,32.1.4.104.32.0.0.1.32.1.4.104.32.0.0.1, 25,

OK

Thanks,
James

1 Like

Hi James.

When using SDK connection manager , I can’t see such problem. LTE connection is fine.
I think I have reached to the root cause of the problem.
See following sequence of commands:
AT!SCACT=1,1
OK
AT!SCACT?
!SCACT: 1,0
If at this point I give again AT!SCACT=1,1 the connection is established.
IS it normal above ?

Thanks.

Hi Martin,

This confirms that the issue is between the host and the modem.
It could be that your PDN1 IP family is configured with “ipv4v6” and when you issue at!scact=1 the modem is expecting an IPv4 and an IPv6 session.
ipv4 IP address is brought up with DHCP
IPv6 uses SLAAC if your host doesn’t support SLAAC then the modem will time out on ipv6 and the whole session will terminate.

check your PDN1 with at+cgdcont?
if it says “ipv4v6”, then reconfigure it to ipv4 “IP”

at+cgdcont=1,“IP”,“apn”
OK
at+cgdcont?
+CGDCONT: 1,“IP”,“apn”,“0.0.0.0”,0,0,0,0

OK

With the SDK you can specify ipv4, ipv6 or both.

about at!scact
at!scact=1 -> at!scact=1 brings up the default data profile , on ATT and Generic it is on PDN1 , on Verizon it is on PDN3)
at!scact=1,1 -> forces the modem to use PDN1 , which is OK on most carriers, but will not work on Verizon.

Regards,
James

Hi James.

No, IPV6 is not involved here.
AT+CGDCONT?
+CGDCONT: 1,“IP”,“globul”,“0.0.0.0”,0,0,0,0
OK

Here is small script that does not connect:
[root@fwa-1012vcc400ada14c34 scripts]# cat nonworking.script
atz
AT+cfun=1
at+cgdcont=1,“IP”,“globul”
AT!scact=1,1
at!scact?

Here is modified script that connects:
[root@fwa-1012vcc400ada14c34 scripts]# cat working.script
atz
AT+cfun=1
at+cgdcont=1,“IP”,“globul”
AT!scact=1,1
at!scact?
at!scact=1,1
at!scact?
at!scact=1,1
at!scact?
at!scact=1,1
at!scact?

Result after executing first script:
at!scact?
!SCACT: 1,0

After second:
at!scact?
!SCACT: 1,1

I wonder how this is possible… Same non-working script works on another Vendor box (again with Sierra modem) .

Thanks
martin.

I think I know what’s going on.

don’t issue atz - it is unnecessary.
it is not a good practice to reprogram the APN every time the modem connects. this forces the modem to detach and reattach to the NTWK. The modem is not even attached to the NTWK when you try to bring up the connection with the first !scact.

at+cfun=1 is turning on the radio, you don’t need it if the radio is already on.

to setup the datacall just issue at!scact=1 once

Thanks,
James

Hi James,

So, to summarize, issue is that connection activation fails (at!scact) because we try to do it too early, when modem did not yet succeeded to attach to network (at+cgdcont).

We can easily workaround it , by adding delay and some checks and retries, but isn’t this a bug of the modem? - connection attempt (at!scact) returns OK, but it doesn’t do it eventually.

Many thanks.
Martin.

Hi Martin,

It was only a theory, does it work when you do not reprogram the profile before activating it?

What you should do is to read the APN and if it needs to be updated then reprogram it. But don’t write it every time.
at+cgdcont?
if the APN is correct send at!scact=1
if not, reprogram it. Wait a few seconds and then send at!scact=1

at!scact normally returns an error if the session cannot be brought up.
This could be a corner case because the modem is already attaching to the NTWK.

Thanks,
James