Decoding backtraces does not work

Hi,

We have some bug in our application that causes the device to reset and it seems to appear when trying to close the UART2 port (using FCM).
We tried to read the backtraces, but when loading the AXF file, the Developer Studio memory consumption increases to about 6GB and the traces cannot be decoded. Here are the backtraces without the AXF file:

Except RTX: task V24, heap memory corruption of 0x1806C7E0, caller = 0x192f
	Unknown function (42e78)
	Unknown function (24bd9)
	Unknown function (24c75)
	Unknown function (27031)
	Unknown function (26f35)
	Unknown function (27005)
	Unknown function (192f)
	Unknown function (dc9)
	Unknown function (690ad)
	Unknown function (68f71)
	Unknown function (69d25)
	Unknown function (45cc3)
	Unknown function (445b1)
	Unknown function (203a0)

And from the Traces view:

2015/02/03;17:22:15:373;001;DEV;1;s_ss error id:1,state:0
2015/02/03;17:22:15:374;001;DEV;1;hub_set_sig IOC_UART_SS io_control failed
2015/02/03;17:22:15:375;001;DEV;1;BSP_UART::write there is no pending operation
2015/02/03;17:22:15:377;001;DEV;1;data_driver_release_uart_sp: Port = 0x00000002 / Status = 0
2015/02/03;17:22:15:379;001;DEV;1;s_ss error id:2,state:0
2015/02/03;17:22:15:380;001;DEV;1;hub_set_sig IOC_UART_SS io_control failed
2015/02/03;17:22:15:385;001;ADL;25;Switch V24 0 0 : 0
2015/02/03;17:22:15:386;001;DEV;1;FCM_CLOSE_FLOW_IND
2015/02/03;17:22:15:388;001;DEV;1;s_ss error id:1,state:0
2015/02/03;17:22:15:389;001;DEV;1;hub_set_sig IOC_UART_SS io_control failed
2015/02/03;17:22:15:391;001;FCM;1;Close- [0x0,0x2] unknown flow
2015/02/03;17:22:15:395;001;ADL;25;FCM Unsubs 0 : 0
2015/02/03;17:22:15:403;001;RTK;1;Except RTX: task V24, heap memory corruption of 0x1806C7E0, caller = 0x192f

Our environment:
SiWi module: WMP100
Developer Studio: 3.1.0 / 3.2.0
Open AT Framework package: 2.51.0.201206190958
Open AT OS Package: 6.51.0.201206010944
Firmware Package: 7.51.0.201205311751

Has someone experienced similar problems? Is there a way to find out what functions are behind the “unknown function” traces?
Is there a chance a SiWi FAE can look at the traces (we could provide a .bin file with the backtraces and the .axf file)?

Thanks!

Hi,
Can you share your application and steps to reproduce the issue so that I can check at my end…

Thanks,
Rex

Hi,

the application is too big and I don’t think it’s needed. If really necessary, I could try to create a sample project…

I tried to get the backtraces and store them into the flash memory so I can download them with Developer Studio. The problem is that the file contains mostly 0x00 (see attachment), whereas with Developer Studio I can at least see the addresses (see previous post). Here is the code snippet to retrieve the backtraces:

s8 AnalysisHandle = adl_errStartBacktraceAnalysis();

    if ( AnalysisHandle >= OK ) {
        SAL::File file; // this is a wrapper for the ADL Filesystem functions
        file.open("backtraces.bin", "w");

        u8 Buffer[1024];
        bool stop = false;

        for(uint8_t index = 0; !stop; ++index) {
            memset(Buffer, 0, sizeof(Buffer));
            s32 retVal = adl_errRetrieveNextBacktrace ( AnalysisHandle, NULL, 0);

            if(retVal > 0) {
                s32 retVal2 = adl_errRetrieveNextBacktrace ( AnalysisHandle, Buffer, retVal);
                if (file.isOpened()) file.write(Buffer, retVal);
                TRC_ERROR("BACKTRACE(%d):", index);
                TRC_BUF( (char*)Buffer, retVal);
                retVal = retVal2;
            } else {
                switch(retVal) {
                case ADL_RET_ERR_PARAM:
                    TRC_INFO("BACKTRACE(%d): %s", index, "ADL_RET_ERR_PARAM");
                    break;
                case ADL_RET_ERR_NOT_SUBSCRIBED:
                    TRC_INFO("BACKTRACE(%d): %s", index, "ADL_RET_ERR_NOT_SUBSCRIBED");
                    break;
                case ADL_RET_ERR_UNKNOWN_HDL:
                    TRC_INFO("BACKTRACE(%d): %s", index, "ADL_RET_ERR_UNKNOWN_HDL");
                    break;
                case ADL_RET_ERR_DONE:
                    TRC_INFO("BACKTRACE(%d): %s", index, "ADL_RET_ERR_DONE");
                    stop = true;
                    break;
                case ADL_RET_ERR_SERVICE_LOCKED:
                    TRC_INFO("BACKTRACE(%d): %s", index, "ADL_RET_ERR_SERVICE_LOCKED");
                    break;
                }
            }
        }
        file.close();
        // Erase all backtraces
        adl_errEraseAllBacktraces();
    }

Do the ADL backtraces functions have to be called in a specific task? We call these in the immediate task (0).
Any ideas?

Thanks!

Small update: reading the backtraces with Developer Studio 3.1 (with the AXF file loaded) seems to work. Some other exception could be decoded (so we could see the mangled function name), but the trace from the first post, Except RTX: task V24, heap memory corruption, still could not be decoded. So we assume it’s some ADL call. Is there a way to decode the calls?

As a side note, the memory consumption of Developer Studio is about 6-7 GB RAM when decoding traces with the AXF file, so we presume there is some memory leak there. The DWL file size is abou 700k and the AXF file size about 25MB. With smaller samples, the decoding seems to work.

Thanks!