Hello,
I have tricky problem with Modem crashing after about 7 days (locks up and network status light stays on).
Difficult to know what’s going on, and I thought I’d try to rule out memory leaks first. But I can’t find a way to do this. There have been similar threads in the past but they seem to be broken.
I’ve tried to use static memory allocation where I can, but I do have some memget functions.
I’ve had a little play around with memGetInfo but these figures stay the same, even if I deliberately introduce massive memory leaks.
People have mentioned putting a wrapper around the memget function with your own up/down counter in it.
Also check out the adl_errSubscribe(ErrEventHandler); it can tell you if there was a watchdog reset for instance. The values read back via the TMT interface are obscure but it can sometimes give you a clue. Check for any while loops or tasks that might take along time triggering a watchdog reset.
Hi,
Memory management is broken in OpenAT. Memory get de-fragmented over time and can not be recovered.
At some stage I did a small app to prove the point and send it to Sierra Wireless. They acknowledge the problem and say:
“We confirm your observation. We can not re-use the freed memory allocated by adl_memGet()”
They suggested workaround is to implement on memory management by creating large memory pool and allocate memory from it in application. And manage it in application. No plans to fix it in the near future as far as I know
Since then , I also discovered that internally openAT is also using dynamic memory allocation. For example when one sends an AT command from software (Adl_atCmdCreate()). Result is same – eentually system runs out of memory.
We have installed error handler and, when problem is detected, we reset the modem ourselves.
This is really alarming! I’ve been carefully trying to manage maintaining connections in various network conditions (spending alot of time, i.e. cash, trying to work through various Open AT problems), but it looks like I might as well just reset once a day as its going to crash anyway!!!
Hmm. I can’t reproduce memory allocation errors. If memory fragmented so no solid block of requested size - yes, error (but what we can expect? relocation mechanism? wonderful memory defragmenter?)
But whole available memory (close to) can be allocated again and again, big, small or random size blocks. 100000+ allocations tested.
Hi,
I can e-mail sample app to demonstrate the problem, so you can check yourself.
Algorithm is pretty simple – allocate number of 1k blocks up to the maximum you can get (in my case about 234). System eventually runs out of memory.
Then try to deallocate every fifth one. So, I allocated 234 and freed up 47 1k blocks. One would expect that about 47k can be allocated again, but, in reality, I can only allocate up to about 1k in a single chunk.
I tried the code with R74a, but, as far as I know, problem still exists in later versions.
You should us dynamic memory allocation when you need memory for short period of time. If not, use static buffers instead. If there is garbage collector then it would consume processor time, which is obviously not good.
If dynamic memory exists, it should perform “garbage collection”. Even if user application has to do it manually. Same problem exists when using flash memory and there is a way to “recompact” it so that one can use large consecutive block.
I agree that dynamic memory is not the best way in embedded system, but, sometimes, there is no way around it. We are working on an embedded modem that sends data through it and this generally involve creating/destroying buffers.
OpenAT uses dynamic memory internally as well, when processing AT commands, for example.
What rudolfl describes as a problem is actually the expected behavior of dynamic memory allocations and will not change. The problem can only be “workarounded” on systems that support the types of hardware exceptions, like the protected mode of the x86 and x64 cpus, what applications see as complete memory block can be scattered in parts in memory and the disk.
I have just experienced this using Q64s. No error message, nothing just a reset out of the blue. I have reduced the problem by cutting back the use atcmdcreate/atCmdSend. But as a failsafe I have used some persistent variables to store important accumulating statics. (ADL_MEM_UNINIT() etc). Tested with an AT+CFUN=1 values are retained but not with AT+CPOF=1.
Can anyone please tell me for sure that these variables are retained in this sort of memory related crash?
