HL7800 stops responding to AT

IoT Modules HL78
1

Hi,

We have a product built with a HL7800 modem for 4G/LTE connectivity. Our Linux platform communicates with the modem over USB and uses /dev/ttyACM1 for ppp communication and /dev/ttyACM0 for AT communication. Our system is continuously monitoring the modem and signal, typically with AT+KCELL=0, AT+CEREG? and AT+CSQ at some interval.

Occasionally we see that the modem completely stops responding on the AT port while the ppp connection is running fine. Once this state occurs it stays this way forever, or until a hard reset of the modem.

This failure occurs seldom enough to be hard to catch on our local test devices. But often enough over the hundreds+ devices running at customer sites to be disturbing (but where debugging possibilities are limited).

The failure has been present on any firmware we have tested so far. We mainly run firmware 4.6.9 nowadays.

We are wondering if this is a failure anyone else has seen and if there is some solution to it. Or if you have any ideas how to debug it further.

Thanks in advance!

/Olle

2

Can we swap /dev/ACM interface for the ppp and AT command?

3

Do you mean we should try using ACM1 for AT when the failure occurs or swap from the start?

I was under the impression that ACM0 is for AT and ACM1 is for ppp.

5

yup, you are correct

When problem happens, can you use +++ to quite the PPP data mode?

6

Hm, I’ve tested this on my local device but can’t seem to exit the data mode with the “+++” sequence. Is this suppose to work on the USB TTY’s?

7

so even problem not happening, you still cannot exit with +++, right?

8

No I can’t exit data mode.

My local test case is just using minicom on the ttyACM1 and start data mode with ATD*99***1#. After this sending +++ does nothing.

9

Here i have quick trial in UART, +++ can help to exit data mode:


ati3
HL7800.4.6.9.10

OK
atd*99***1#
CONNECT
~ÿ}#À!}!}!} }4}"}&} } } } }%}&©Éîb}'}"}(}"Ì~
OK
atd*99***1
OK
ato
CONNECT

OK

atd*99***1#
CONNECT

OK
atd*99***1#
CONNECT

OK
10

Thanks, yes it does work to exit data mode with +++ on my local device (over USB too), I was just doing it wrong.

However I’m not sure how to use this on a failing device. Since I can’t easily repeat the original failure on my local device and running it on a remote device at customer site would obviously shut down the remote connection which I would use for debug.

Do you know if this a known bug that the USB AT port can stop listening to requests?

We have the UART1 connected to our MCU too but it seems like that one is disabled once you enable USB on HL7800 (AT+KUSBCOMP=1). Can UART1 be enabled for debugging purposes somehow while still using USB for AT+ppp?

11

Have you isolated if this is your MCU problem or not?
For example, do you see such problem in ubuntu PC?

BTW, do you have UART0 connected to MCU?
I remember you can also enter AT command in UART0

Lastly you can also try to enable only AT port in USB and let UART1 still have the function of AT_PPP so that you can try to dial up by UART1

12

Have you isolated if this is your MCU problem or not?
For example, do you see such problem in ubuntu PC?

No, since we can’t repeat the error easily we haven’t been able to try it with any other host.

To me it looks like a modem bug since a hard reset of the modem solves it. No reboot of the host is needed.
If it was the host then a restart of our application would probably solve it, but it doesn’t. Or if it was a failure where our host is not able to receive responses then the modem should still act on commands (like sending a reset AT+CFUN=1,1), but it doesn’t. Also the ppp port is still working during this. But no, we can’t be sure it’s the modem. Guess the failure could be in the Linux cdc_acm driver too.

BTW, do you have UART0 connected to MCU?
I remember you can also enter AT command in UART0

We do not have UART0 connected unfortunately, only UART1.

Lastly you can also try to enable only AT port in USB and let UART1 still have the function of AT_PPP so that you can try to dial up by UART1

How would we do that?

13

Not sure if the PPP too busy will make your MCU USB driver go wrong on other ACM port, just a guess

See if AT+KUSBCOMP=1,1,0,0 can help
I never tested this, but the idea is to have AT_PPP in UART1 port

14

See if AT+KUSBCOMP=1,1,0,0 can help

Seems the UART1 is still disabled after this, at least I see nothing is being written and no responses to AT commands neither, even after ATD*99***1#.
I tried AT+KUSBCOMP=1,2,0,0 too, to make ACM0 the ppp interface but seems UART1 is still disabled unfortunately.

15

How about using AT+KUSBCOMP=1,2,1,0 but ppp dial up with /dev/ACM0?

Is it same result?

16

Do you think there’s any chance that it could give any other result? Seems like quite a long shot to me. On the other hand we have no other ideas at this time so I guess we could try on a bunch of devices.

To bad UART1 can’t be used for anything while USB is enabled.

If we did have access to UART0 during one of theses failures… Do you have any ideas what we could try on the debug port to understand this failure?

17

Did you contact distributor to report this?

I remember the uart0 debug port can send AT command
You can try whether it still responds when problem happens

Btw, i am still wondering whether +++ can quit data mode and back to AT mode when problem happens

18

Did you contact distributor to report this?

Did a long time ago and if I remember correctly they wanted us to do some logging on UART0 while the failure occurred. Which would be really hard to do given how infrequent it occurs. We’re reaching out here in hope someone might have had a similar error.

Btw, i am still wondering whether +++ can quit data mode and back to AT mode when problem happens

I will investigate if we can use +++ to reach AT mode again when the error occurs. It doesn’t seem like a solution but I guess it could provide some context to the error if possible.

19

I just tested with AT+KUSBCOMP=1,1,0,0.
i can have AT command in UART1 (COM45) together with AT command in USB (COM92)
And COM30 is the UART0 which can also have AT command communication:

image
image1103×698 121 KB

I am not quite sure why you cannot make this work in your board…

20

Ok, I think I understand this now… When going from USB off to AT+KUSBCOMP=1 UART1 will be disabled:

AT+SWITRACEMODE?                                                                                                                                          
+SWITRACEMODE: CUSTOMER,0,0,0

Changing to AT+KUSBCOMP=1,1,0,0 after this will not re-enable UART1 but that has to be either manually done with SWITRACEMODE=CUSTOMER,0,2,0 or by turning off USB (AT+KUSBCOMP=0) and then setting AT+KUSBCOMP=1,1,0,0 again.

Tried to get AT_PPP mode on USB while keeping only AT on UART1 but couldn’t get that combination running.

Anyway I think this is a good start and we can probably evaluate and see if this makes any difference on our devices.

21

hmm… can baud rate of UART1 be increased or will 115kbaud now limit our bandwidth if that is to be used for ppp?