I have a multitasking application and I have some data, which should be available from several tasks. In Windows I have to use critical sections or mutexes to synchronize access to such data from multiple threads. Should I worry about this in Open AT OS? Or does the OS cater for such synchronization? If not, what can I use?
As far as I understand, nobody ever had any multitasking problems? 
You must take care for task synchronization. Overall the RAM memory is globally available (global variables), but if you access the same area from two different tasks - you must take care of concurrency.
See OpenAT docs for Semaphore, message and event services.
regards