# DS 1.2.0 Trace CSV Format not compatible with Excel

**URL:** <https://forum.sierrawireless.com/t/ds-1-2-0-trace-csv-format-not-compatible-with-excel/4893>\
**Category:** Development tools\
**Created:** [February 14, 2011, 4:27pm UTC](https://forum.sierrawireless.com/t/ds-1-2-0-trace-csv-format-not-compatible-with-excel/4893 "2011-02-14T16:27:33Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![awneil](https://avatars.discourse-cdn.com/v4/letter/a/49beb7/32.png) [@awneil](https://forum.sierrawireless.com/u/awneil)\
**Post date:** [February 14, 2011, 4:27pm UTC](https://forum.sierrawireless.com/t/ds-1-2-0-trace-csv-format-not-compatible-with-excel/4893/1 "2011-02-14T16:27:33Z")

</div>

Ref: [viewtopic.php?f=108&t=5123&start=0](https://forum.sierrawireless.com/viewtopic.php?f=108&t=5123&start=0)

There’s a few issues with the DS 1.2.0 Traces CSV format that make it incompatible with MS Excel (and, I suspect, other tools also):

1. Date format

> [@](#):
>
> 11/02/04,15:32:55:984 - 136;ADL;7;WM\_L3\_INFO\_SYNC\_CELL\_INFO

Because all fields are 2 digits, Excel can’t tell which is the year (human readers may also get it wrong due to US/RoW confusion about mm/dd or dd/mm)

A 4-digit year needs to be used.

2. Time format

> [@](#):
>
> 11/02/04,15:32:55:984 - 136;ADL;7;WM\_L3\_INFO\_SYNC\_CELL\_INFO

Excel doesn’t recognise this as a time, because of the colon between seconds & decimal seconds.  
It needs to be a decimal point here.

3. Time width  
The width of the decimal seconds varies between 1 & 3 digits - which breaks the tabulation, and makes the plain text file hard to read.; eg,

```auto
11/02/04,15:32:55:984 - 137;ADL;7;[L3] same PLMN
11/02/04,15:32:55:98 - 138;ADL;7;02 03 04 01 00 FF 
11/02/04,15:32:56:0 - 001;ADL;7;[L3] plmn_change_timer_cb
11/02/04,15:32:56:0 - 002;L3RR;3;==>C RRC_TIM_PERIODIC_SI
11/02/04,15:32:56:15 - 003;L3RR;3;<==C PH_BCCH_REQ on 27
11/02/04,15:32:56:150 - 004;L3RR;3;C: S4 P0 N1 M10
```

This also means that the above problems can’t be worked around by importing as fixed-width fields.

4. Quoting  
The Trace strings themselves need to be “quoted” in some way - so that semicolons within the text are not takn as delimiters!  
Or, again, the preceding fields need to be fixed width.

---

<div class="post-metadata">

**Author:** ![daav](https://sea1.discourse-cdn.com/flex025/user_avatar/forum.sierrawireless.com/daav/32/2767_2.png) [@daav](https://forum.sierrawireless.com/u/daav)\
**Post date:** [February 15, 2011, 9:29am UTC](https://forum.sierrawireless.com/t/ds-1-2-0-trace-csv-format-not-compatible-with-excel/4893/2 "2011-02-15T09:29:20Z")

</div>

Correct me if I’m wrong, but all of this can be configured in the DS preferences: Window \> Preferences \> Developer Studio \> Target Management \> Traces

---

<div class="post-metadata">

**Author:** ![awneil](https://avatars.discourse-cdn.com/v4/letter/a/49beb7/32.png) [@awneil](https://forum.sierrawireless.com/u/awneil)\
**Post date:** [February 15, 2011, 10:09am UTC](https://forum.sierrawireless.com/t/ds-1-2-0-trace-csv-format-not-compatible-with-excel/4893/3 "2011-02-15T10:09:19Z")

</div>

Oh yes - so it is! 😊

---

<div class="post-metadata">

**Author:** ![awneil](https://avatars.discourse-cdn.com/v4/letter/a/49beb7/32.png) [@awneil](https://forum.sierrawireless.com/u/awneil)\
**Post date:** [February 16, 2011, 8:03am UTC](https://forum.sierrawireless.com/t/ds-1-2-0-trace-csv-format-not-compatible-with-excel/4893/4 "2011-02-16T08:03:28Z")

</div>

When the traces are displayed in the Studio, it manages to get each one on a single row.

But AT responses in the recorded file get broken across lines; eg,

```auto
11/02/04,15:47:58:93 - 007;ADL;7;l3_info_rsm_cb
11/02/04,15:47:58:109 - 008;ADL;7;Chan Lev BS MCC MNC LAC CI
11/02/04,15:47:58:109 - 009;ADL;7; 75 15 60 234 15F 00BF 1588
11/02/04,15:47:58:109 - 010;ADL;7;RSM end.
11/02/04,15:47:58:125 - 011;ATI;1;ati_SndUnsolicitedAtRsp: Len=12
11/02/04,15:47:58:125 - 012;ATI;1;
+CREG: 2

11/02/04,15:47:58:125 - 013;ATI;1;ati_SndUnsolicitedAtRsp: Len=13
11/02/04,15:47:58:125 - 014;ATI;1;
+CGREG: 0

11/02/04,15:47:58:140 - 015;ADL;1;+CREG: 2 state received 
11/02/04,15:47:58:140 - 016;ADL;1;Force RSSI=99
11/02/04,15:47:58:140 - 017;ADL;1;+CREG: 2 Change registration state from 5
```

---

<div class="post-metadata">

**Author:** ![daav](https://sea1.discourse-cdn.com/flex025/user_avatar/forum.sierrawireless.com/daav/32/2767_2.png) [@daav](https://forum.sierrawireless.com/u/daav)\
**Post date:** [February 16, 2011, 8:11am UTC](https://forum.sierrawireless.com/t/ds-1-2-0-trace-csv-format-not-compatible-with-excel/4893/5 "2011-02-16T08:11:54Z")

</div>

> [@awneil](#):
>
> AT responses in the recorded file get broken across lines

We already have logged an evolution request to handle “special characters” in trace strings.  
E.g. print “” or “” explicit keywords, in order to:

1. recognize them in the Traces view (instead of displaying weird squares characters 😉)
2. avoid this line break issue in the logged files

---

<div class="post-metadata">

**Author:** ![awneil](https://avatars.discourse-cdn.com/v4/letter/a/49beb7/32.png) [@awneil](https://forum.sierrawireless.com/u/awneil)\
**Post date:** [February 16, 2011, 8:48am UTC](https://forum.sierrawireless.com/t/ds-1-2-0-trace-csv-format-not-compatible-with-excel/4893/6 "2011-02-16T08:48:42Z")

</div>

> [@daav](#):
>
> E.g. print “” or “” explicit keywords

It would be really useful if you could find a font that can represent these codes in a single character space - as the VT100 et al used to:  
 ![](https://us1.discourse-cdn.com/flex025/uploads/sierrawireless/original/2X/e/edce0fc32b0be8d8479e69e63d952f402d79bdfa.png)  
[vt100.net/dec/vt220/glyphs](http://vt100.net/dec/vt220/glyphs)  
[vt100.net/docs/vt220-rm/table2-16a.html](http://vt100.net/docs/vt220-rm/table2-16a.html)

> [@](#):
>
> 1. recognize them in the Traces view (instead of displaying weird squares characters 😉 )

Yes, that would be good - I currently tend to use both TRACE and DUMP where I suspect there may be a mix of prointable and non-printable characters…

---

<div class="post-metadata">

**Author:** ![awneil](https://avatars.discourse-cdn.com/v4/letter/a/49beb7/32.png) [@awneil](https://forum.sierrawireless.com/u/awneil)\
**Post date:** [July 21, 2011, 1:24pm UTC](https://forum.sierrawireless.com/t/ds-1-2-0-trace-csv-format-not-compatible-with-excel/4893/7 "2011-07-21T13:24:22Z")

</div>

With quoting enabled, I get this kind of thing:

> [@](#):
>
> “2011/07/07,14:59:55.546 - 002”;“ADL”;“31”  
> “2011/07/07,14:59:56.531 - 001”;“ADL”;“11”  
> “2011/07/07,14:59:56.546 - 001”;“ADL”;“31”  
> “2011/07/07,14:59:57.531 - 001”;“ADL”;“11”

This means that Excel still can’t use the time - because of those suffixes.

---

<div class="post-metadata">

**Author:** ![daav](https://sea1.discourse-cdn.com/flex025/user_avatar/forum.sierrawireless.com/daav/32/2767_2.png) [@daav](https://forum.sierrawireless.com/u/daav)\
**Post date:** [July 21, 2011, 2:01pm UTC](https://forum.sierrawireless.com/t/ds-1-2-0-trace-csv-format-not-compatible-with-excel/4893/8 "2011-07-21T14:01:40Z")

</div>

> [@awneil](#):
>
> With quoting enabled, I get this kind of thing:
> 
> > [@](#):
> >
> > “2011/07/07,14:59:55.546 - 002”;“ADL”;“31”  
> > “2011/07/07,14:59:56.531 - 001”;“ADL”;“11”  
> > “2011/07/07,14:59:56.546 - 001”;“ADL”;“31”  
> > “2011/07/07,14:59:57.531 - 001”;“ADL”;“11”
> 
> This means that Excel still can’t use the time - because of those suffixes.

We’ll see to add an option to remove the counter suffix when persisting traces in a file.

---

<div class="post-metadata">

**Author:** ![awneil](https://avatars.discourse-cdn.com/v4/letter/a/49beb7/32.png) [@awneil](https://forum.sierrawireless.com/u/awneil)\
**Post date:** [July 21, 2011, 2:04pm UTC](https://forum.sierrawireless.com/t/ds-1-2-0-trace-csv-format-not-compatible-with-excel/4893/9 "2011-07-21T14:04:48Z")

</div>

Might also be better to have the date separate from the time?

---

<div class="post-metadata">

**Author:** ![daav](https://sea1.discourse-cdn.com/flex025/user_avatar/forum.sierrawireless.com/daav/32/2767_2.png) [@daav](https://forum.sierrawireless.com/u/daav)\
**Post date:** [July 22, 2011, 6:18am UTC](https://forum.sierrawireless.com/t/ds-1-2-0-trace-csv-format-not-compatible-with-excel/4893/10 "2011-07-22T06:18:14Z")

</div>

> [@awneil](#):
>
> Might also be better to have the date separate from the time?

Sounds difficult to make this working along with the capability to configure the timestamp format… If you don’t need the date, just remove it from the timestamp…

---

<div class="post-metadata">

**Author:** ![awneil](https://avatars.discourse-cdn.com/v4/letter/a/49beb7/32.png) [@awneil](https://forum.sierrawireless.com/u/awneil)\
**Post date:** [July 22, 2011, 6:50am UTC](https://forum.sierrawireless.com/t/ds-1-2-0-trace-csv-format-not-compatible-with-excel/4893/11 "2011-07-22T06:50:43Z")

</div>

See: [https://forum.sierrawireless.com/t/getting-ds-1-2-0-to-remember-my-layout/4842/7](https://forum.sierrawireless.com/t/getting-ds-1-2-0-to-remember-my-layout/4842/7)

I generally find that the date just wastes space in the screen display, but I do want it recorded in the log file.

Having separate date & time fields for both screen & file seems good to me

---

<div class="post-metadata">

**Author:** ![daav](https://sea1.discourse-cdn.com/flex025/user_avatar/forum.sierrawireless.com/daav/32/2767_2.png) [@daav](https://forum.sierrawireless.com/u/daav)\
**Post date:** [July 22, 2011, 11:40am UTC](https://forum.sierrawireless.com/t/ds-1-2-0-trace-csv-format-not-compatible-with-excel/4893/12 "2011-07-22T11:40:53Z")

</div>

Ok, this is what we’ve decided to implement in 2.1.0:

- Use two fields in preferences for timestamp configuration:
  - One for time (default “HH:mm:ss:SSS”; prints UNIX raw millisecond timestamp if empty)
  - One for date (default “yyyy/MM/dd”; doesn’t print anything if empty)

- UI behavior: use “time” format field + " - XXX" suffix for traces counter (date is not displayed; seems clearly not useful in the UI)
- Log behavior: persist date, time and counter on three different columns
