OpenAT SDK on Linux porting HOWTO

IoT Modules Development tools
21

wizmo_coder, a bit later and I don’t know if you still have the problem, but what do you mean by an error message saying “that the .dwl file cannot be installed”? Is an error message generated by the module or by the software you are using to upload the dwl?

I’m asking since I have a problem after updating from a 32bits fedora core 8 installation to a 64-bits fc10, in which the generated dwl images are rejected by the module (transfer is aborted immediately, I’m using an expect script that uses picocom to upload the image). If I use the same code base but compile it under a 32bits fc8 or centos5, the image uploads without issues. Of course I recompiled the gcc arm compiler for each case. Probably not related… but who knows :wink:.

BTW, and a little off-topic, anyone knows if it’s possible to run in RTE mode under linux? I need a way to test my code automatically without having to upload a new binary and run it on the module, so I thought I might use the RTE mode to run unit tests against it… The only other option I found at this time was to build a wrapper library over some wavecom specific calls and provide an implementation for linux also, this way I can write unit test with check, and run them and other test code under valgrind… I have a nice part covered (and I will not test some of the code this way, like for example processing of AT commands), but It’s definitely too much work…

22

I assume that the root problem is related to the fact that you went from 32-bits to 64-bits.
To determine which of the tools is causing the problem, compare each of the intermediary files between your old installation and your new one. (e.g. If the .wpb file is identical but the .wpb.dwl file isn’t, then genbin is at fault.) There will be some files, like .elf, that probably won’t match, but the resulting .bin will probably strip out all of the extra stuff (such as paths) that don’t matter. (Disclaimer: I’m not using the OpenAT build scripts any more (see viewtopic.php?f=19&t=2388), so my file extensions may not match your environment.)

There’s so much that can’t be done in RTE, I’ve given up on it a long time ago. Sorry I can’t help.

23

I found the solutions rather quickly, but I forgot to post it here. Sorry for that.

You are quite right in your assumption: it is a problem with python and 64bit environments. Python has some bizarre datatypes that vary depending on the environment. Therefore a Python program written for a 32bit environment doesn’t work correctly in a 64bit environment without certain adjustments. I attached the corrected version of the gendwl.py script. Be careful not to mix it up with the 32bit version, because they are identical except for a few bytes. It is meant to replace $(OAT_ROOT)/IDE/IDE/$(IDE_VERSION)/sgt/tools/scripts/gendwl.py.

Now, the RTE_Kernel.exe certainly is a windows program, but you could always try to run it with wine. If you manage to explain to wine where your com ports are you might even have a chance…

Best regards,

wismo_coder

24

Excellent! I had to use python code using struct on 64 and 32 and faced the same issues before, but I didn’t know where to look in this case! :slight_smile:

Here I attach a version of the script that is explicit on the size, endianess and align of the elements of the struct, so it should work both on 32 and 64 bits (by using the ‘standard’ int types,according to python :wink:).

25

Well, the comment helps since it gives me some more reasons to not go down that path. I will create a new thread asking how do people usually test their applications.

26

Did anyone tried to port the current M2M Studio (well, not the whole Studio, rather the OpenAT SDK part) on Linux? I am just trying to do this based on what I found here, but I don’t know if I have any chances. Some things just chaged as far as I can see after installing the M2M Studio (for one, I can’t find any command line utilities anymore). Also, would it work using binutils 2.18 and gcc 4.2? Apparently Wavecom distributes the M2M Studio still with gcc 4.0.1.

Thanks for any input. I will keep you informed on any possible progress (note that I am trying to install it on MacOS X, but this should not make any difference to Linux).

27

Well, I made some progress, but I have an issue with the linker generated files. I suspect that there are some linker options or maybe the linker script file that must be adjusted, because the .elf file is something like 400 MBytes (yes, you read it right!).

What I have done to date (as a matter of principle I largely followed the path indicated by Matt Otto):

  • Copied the M2M Studio on MacOS X
  • Installed arm-elf-gcc from MacPorts (version 4.3.2 with arm-elf-binutils 1.18.50, however, the portfile need some minor tweaking before installing)
  • Adapted the makefile to the new paths of the M2M Studio (lots of locations changed!)
  • Tweaked the makefile after running the project (“Hello World”) on M2M Studio on Windows, so that the makefiles’ output (Windows/MacOS X) is identical

Obviously, the large .elf file results because of the linker padding the “distance” between the .text and the .data sections. I am at a loss why on Windows this does not happen, and how could this be managed. Maybe a gcc-toolchain guru has an idea…

To clarify things, the toolchain is the only difference I can think of between the Windows version (gcc-4.0.1/binutils-1.16) and the MacOS X version (gcc-4.3.2/binutils-1.18.50). However, compilation and linking goes without a warning! I am testing on the “Hello World” sample, as soon as this will work I will try some other, more complex samples.

Any hint would be greatly appreciated.

28

First, take a look here to see how to build code without M2M: viewtopic.php?f=19&t=2388

The Makefile is based on the older SDK (pre-M2M), but it still has some useful information. For example, to trim the fat from your .elf file, try this (assuming hello.elf):

arm-elf-objcopy --strip-all --strip-debug --strip-unneeded --discard-all --weaken hello.elf hello.tmp
arm-elf-objcopy --output-target binary hello.tmp hello.bin

If hello.bin is >= 1MByte, then there’s something wrong. If it shrinks to a normal size, then the extra 400MBytes is harmless debugging information.

29

Hello Matt,

Thanks for the input. As a matter of fact, I based my makefiles on your work, and yes, indeed the objcopy is also used. However, the input file goes out almost the same length as the input (that is, about 400MB).

This is what I finally did and worked:

$(OBJCOPY) --strip-all --strip-debug --strip-unneeded --discard-all --weaken  -O binary -j .data $*.elf $*.data || (RVAL=$$?; rm -f $*.data; exit $$RVAL)
	$(OBJCOPY) --strip-all --strip-debug --strip-unneeded --discard-all --weaken  -O binary -j .oat_header -j .text $*.elf $*.text || (RVAL=$$?; rm -f $*.text; exit $$RVAL)
	cat $*.text $*.data > $*.tmp

Thus, I am saving the text and data segments in two separate files that are then concatenated (as you see, I shamelessly copied from your makefiles). Using this trick, the resulting file becomes usable, the “hello_world.tmp” becomes about 70K long.

There is something else that bothers me: the wzpacker utility. I suspect that this is compacting the binary before being wdl-ized (as a matter of fact, I don’t really understand why did Wavecom used such a convoluted way for preparing a flash-able file: what does wzpacker, what does genwdl do?). So coming back to the wzpacker, I cannot understand what do you use under Linux? The M2M Studio comes with a wzpacker.exe, so how did you port it on Linux? Or maybe by that time the wzpacker was written in Python? Any ideas how to get over this? I tried wdl-izing a standard .bin file, the Fastrack supreme I am using (with FW 7.3) does not accept it when trying to x-modem download it.

You probably noticed by now that I am a newbee with OpenAT, I am evaluating the product to see if it can be used for a new project. For the time being I am trying to see if it can be consistently used on the MacOS X platform (most of our developers are working on MacOS X). If I manage to solve all the problems, I will post a HOWTO for others that might be interested (as a matter of fact, the procedure applies largely to Linux too, for those starting from the M2M Studio).

30

As you’ll learn, Wavecom does everything in a convoluted way. The pre-M2M build is worse. It’s obviously the same tools they use for internal builds, which makes our builds more complicated and slower than it needs to be. (This is the reason we created the simple Makefiles in the other thread.)

wzpacker obviously compresses the file (sorry, I don’t know more), but genwdl just adds a header to the file (run genwdl on an empty file to see - most of the header is the current path).

My porting to Linux was made much easier by the fact that internally at Wavecom they obviously use Linux, so all the tools already exist in the SDK directories. Sorry, there’s no MacOS versions.

I think I’ve heard that older firmware didn’t used to support compression (wzpacker), so I’d try just running only genbin on the file and see if you can download it. I wouldn’t be surprised if the newer firmware still supports the “old”, uncompressed format.

Now, back to the original problem:
Concatenating the two sections won’t work, because the OS won’t know know where to load the two sections. The linker thinks the two sections should be 400MB apart in the memory map, which seems to be the root problem. What linker file are you using? The linker.opt that I include in the Makefile thread wouldn’t work with FW7.3 (it may not work on the Fasttrack supreme at all - I’m using the Q2687.)

Hmm… Looking at my linker.opt, RAM_MAP has a value of 0x180C0000 which is ~400MB.

Other than the linker file, I suppose there could be some new behavior in gcc-4.3.2 or binutils-1.18.50 that would cause this. (Based on past experience, I don’t think this is the case, but it’s worth mentioning.)

31

Thanks Matt for your comments.

I advanced substantially since my last post, to the point I now can generate OpenAT applications 100% natively on MacOS X. I will post during the next days a HOWTO (it is obviously in large part valid for Linux too).

There are still two minor issues: the wzpacker utility was never written in Python (I checked older OpenAT SDKs) therefore I can’t generate compressed binaries; and secondly, there is the issue with the large .elf files generated by the linker. The linker issue is most probably due to a possbile bug (or new feature?) in the gcc-4.3.2 I am using, but can be overcome with the method described by me (extract the .text and .data and concatenate them back). It’s only that the linking phase takes longer than it should (e.g. the hello-world application takes about 10 seconds to link on my MacBook Pro - 2,53 GHz Core 2 Duo, while compiling is just one or two seconds).

Otherwise I compiled several sample applications without any problems (the occasional warnings are the same as those I get when compilling with the M2M Studio on Windows) and all of them run correctly when downloaded on the Fastrack supreme (by the way, the Fastrack is based on the 2686). I compiled and run both ADL and WIP samples with no issues at all, as an added bonus the .bin files generated with gcc-4.3.2 are 3-5% smaller than the original ones (with gcc-4.0.1).

32

I have to make a correction, under the same conditions (-Os -O3) the files generated with gcc-4.3.2 are only about 1 to 2% smaller than those generated by gcc-4.0.1.