Backtracing Problems [SOLVED]


#1

Hi All,

We are currently struggling to receive backtraces from our firmware. With the sample program (bug.c) can successfully retrieve all backtraces. We initially thought it might have something to do with c++ but after converting the sample program to a small c++ program and running that all backtraces were successfully retrieved. Dont know if it’s due to the complexity/size of our program (305 KB with ± 50.cpp and header files). When trying to retrieve backtraces with our firmware retrieval doesn’t move past 0% (Loading unwinding data) even when left overnight and the only way to kill it is to restart the DS the cancel button does nothing. No errors are logged in the .log file either. I know why the program crashes and fixed this bug but if an unknown one appears will not know where to start looking.

Other methods tried with some success:
[u]https://forum.sierrawireless.com/t/function-names-in-backtraces/4862/17]
Results:

ADS Bckt	-> Rtk11_Schedule : 0x00001383
ADS Bckt	-> rtk_TaskStackOverflow : 0x000644D3
ADS Bckt	-> RtkExceptionRoutine : 0x0006518B
ADS Bckt	-> KER_ERROR_DIAGNOSE : 0x000342A9
ADS Bckt	-> KER_VERROR_DIAGNOSE : 0x000341F3
ADS Bckt	-> sys_BackTrace : 0x00066B10
ADS Bckt	-------> ADS BACK TRACE <-------
ADS Bckt	The function corresponding to this address : 0x3c8c, is not in the .axf file
Trace	SYS	1	Current OAT Task index : 0
Trace	CUS4	1	ARM Data Abort caught at 0026B214, Current Task 0x1F by CP15 
ADS Bckt	-> Unresolved symbol : 0x0026B214
ADS Bckt	-------> ADS BACK TRACE <-------
ADS Bckt	The function corresponding to this address : 0x180c6da8, is not in the .axf file

But dont know if im on the right track or if im going nowhere.

See other people have had the same problems was hoping someone found out how to solve it:
[u]https://forum.sierrawireless.com/t/function-names-in-backtraces/4862/8]

Any alternative methods I can try or advice would be appreciated.

~C

Info:
Embedded Module: SL Series (SL6087)
Firmware: R7.46.0.201108091301
Open AT OS Package: 6.36.0.201108111228
Open AT Embedded Software Suite Package: 2.36.0.201108306638
Developer Studio: 2.0.0.201103101643-R6604
OS: Windows 7 64-bit


#2

You should upgrade your Developer Studio to 2.1.1 --> backtraces decoding has been strongly enhanced in this version.


#3

Thank you daav will give it a try and let you know. Right now close to the end of the development cycle and didn’t want to risk upgrading on a system that’s overall pretty stable but will give it a shot.


#4

Ok managed to get DS 2.1.1 running but this did not seem to solve our problem. Backtraces also gets stuck with the only difference being the message

Also tried to use TMT and managed to get some results but nothing useful. Went through all options and permutations of selecting from archive or from compiled directory with no success.


The code im using to create an error is by setting unallocated memory in a part of the program where i can consistently reproduce the error. So I know where the error is and would like the backtraces even to just point to the general area if thats the best that could happen. Once again any tips or things I could try will be much appreciated.

char *buffer;
buffer[5]= 'a';

#5

We need to reproduce the issue to understand what’s going wrong…
Please can you make a record of your backtrace decoding request?

  1. Connect to the device
  2. Start recording (using the action from the Devices view)
  3. Request backtraces decoding (ensuring that the bug still reproduces…)
  4. Stop recording
    We’ll need the .record file + your app AXF file. You can send it to me in PM if you prefer.

#6

Thanks again daav, sent files via PM.


#7

Got them. We’re reproducing the issue here… Maybe something related to the size of the app (AXF > 10MB…)
We’re investigating now and I’ll let you when we have found the issue.


#8

We’ve fixed the AXF loading issue, which is due to the fact that the application uses C++
The fix will be integrated in a new DS release (to be delivered in the coming days)

However, this doesn’t fix completely backtrace decoding for C++: stack trace decoding is for the moment still not working correctly in this case.
We’re working on finding a solution for next releases.


#9

Thanx for the feedback daav! We thought it might have had something to do with c++ but was holding thumbs that it was something else. Looking forward to the following releases.


#10

Just an update.

Our backtrace retrieval have been fixed in DS2.2.0.
Currently running Build Version 2.2.0.201205041104-R9457