Understanding Flash Limitation

[Jono]

Just a few queries that I hope someone here has a better understanding than I do.

I currently have a Fastrack Supreme 10, running on firmware 6.63g and it’s got an IESM GPS chipset.

It’s programmed to collect GPS data every 30 seconds, and in the interval, there is a GPIO that might be triggered to cause GPS data to be sent earlier as well. Due to the nature of the solution, there are several cases where GPRS is out of range, and the data will not be sent back to the server (TCP/IP), this data’s are stored in flash.

I currently have 3 flash handles

FlashRetVal = adl_flhSubscribe("StoreForward", 2000);


FlashRetVal = adl_flhSubscribe("StoredCount",1);


FlashRetVal = adl_flhSubscribe("SentCount",1); 

StoreForward will be the flash that handles the GPS data.

StoredCount is the flash that will handle the object ID that it has been stored till. So let’s say we have 200 data stored in the flash, the StoreCount will keep track of the value 200, and SentCount is the flash handle that indicates till which index the GPS data has been sent from flash.

Here my predicament truly lies. Every time a GPS data is written in the flash, the StoredCount will have to be updated as well, and it is constantly written on the same object.

FlashRetVal = adl_flhWrite((ascii*)currentStoredCount, 0, 4, storedCount);

I’ve re-read the ADL documentation but I am still unsure whether it is considered as one erasure cycle. I’ve put a constant size on the object just to be sure it doesn’t fire up the garbage collector. So, does this one operation constitutes to the erasure cycle? Because if it does, I have to re-implement another method to do this, as it will dry off that 100,000 in 100 days or so. (Two flash handlers btw).

My other question will be this. It says in the documentation that one handler can subscribe to 2000 objects, but how big of an object can it store?

Like at the moment, im subscribing to 2000 objects, and each time the GPS data is stored, it’s about 80-100 bytes at most. So will this be feasible or will a problem persist?

I’ve tried ADL with the previous implementation but the re-compact sometimes messes things up

[fer.caballero]

Why do you use flh for this? Why don’t you store data in RAM using a “circular” queue? If you want the data stored between restarts, I find using A&D a better soution, and much simplier.

Anyway, how often would you have to store GPS data: once a week, once a day, twice a day…? Be careful with the number of writing cycles of the flash (±100.000 times) if you do it more often.

It says in the documentation that one handler can subscribe to 2000 objects, but how big of an object can it store?

As written on adl_flhWrite: A single flash object can use up to 30 Kbytes of memory.

So, does this one operation constitutes to the erasure cycle?

Which one?

So will this be feasible or will a problem persist?

Which problem?

[fer.caballero]

Why do you use flh for this? Why don’t you store data in RAM using a “circular” queue? If you want the data stored between restarts, I find using A&D a better soution, and much simplier.

Anyway, how often would you have to store GPS data: once a week, once a day, twice a day…? Be careful with the number of writing cycles of the flash (±100.000 times) if you do it more often.

It says in the documentation that one handler can subscribe to 2000 objects, but how big of an object can it store?

As written on adl_flhWrite: A single flash object can use up to 30 Kbytes of memory.

So, does this one operation constitutes to the erasure cycle?

Which one?

So will this be feasible or will a problem persist?

Which problem?

[blackyblack]

I’ve re-read the ADL documentation but I am still unsure whether it is considered as one erasure cycle. I’ve put a constant size on the object just to be sure it doesn’t fire up the garbage collector. So, does this one operation constitutes to the erasure cycle? Because if it does, I have to re-implement another method to do this, as it will dry off that 100,000 in 100 days or so. (Two flash handlers btw).

Don’t worry about erasure cycles. OpenAT flash driver provides wear leveling and writes data in different cells to save memory resource. Just do it simple and OpenAT will do everything needed by itself.
Also in my project I use one flash object for all data. It is not necessary to have an object for every used structure.

[fer.caballero]

OpenAT flash driver provides wear leveling and writes data in different cells to save memory resource.

The problem for -Jono- is they can use the complete flash objects section. 2000 objects * 80 B = 160000 B, more than the 128 kB of flash objects section. Does OpenAT moves the flash objects section over the total size of the flash? Does anybody knows?
Wavecom guys: Could you answer to this question? Please.

[Jono]

@fer.caballero

Why do you use flh for this? Why don’t you store data in RAM using a “circular” queue? If you want the data stored between restarts, I find using A&D a better soution, and much simplier.

Anyway, how often would you have to store GPS data: once a week, once a day, twice a day…? Be careful with the number of writing cycles of the flash (±100.000 times) if you do it more often.

Because in 6.63g, there is no multitasking available. Everything is sequential. The project, consists of polling GPS data every 1 second, and on the 30 seconds mark, it will send GPS information through the TCP socket. How often i store GPS data depends on how flaky the connection is going to be. If the GPRS does not exist, the system will automatically store the information in flash. Writing cycles is not 100,000, erasure cycle is the one with the limit. I have used ADL before and the only issue is doing the recompact. It takes up to 10 seconds sometime, and because it has no threading system, it pushes the system to the limit because there’s still that much operation going on behind. The watchdog reset will kick in sometimes…
I have to poll GPS every 1 second, so that is a non-question issue.

It says in the documentation that one handler can subscribe to 2000 objects, but how big of an object can it store?
As written on adl_flhWrite: A single flash object can use up to 30 Kbytes of memory.

So, does this one operation constitutes to the erasure cycle?
Which one?

See, the problem is this, everytime i write into the flash, i have two other objects, one to keep track of which index it has sent to, and one to track which index it has been stored to. This is essential to know and ensure no repeated data’s are sent, and all data’s are sent. Each time a data is stored in the main object (GPS Data), the index that keeps track of the index value is written on the same object.

adl_flhWrite((ascii*)currentStoredCount, 0, 4, storedCount);

So my question is this, with the size being constant (4 bytes), if i keep writing on the same object, it does flh_erase and flh_write (according to documentation), but does this constitute to an erasure cycle and the 100,000 limit, or the chipset has a smart way of knowing not to do Garbage collector just yet etc etc etc.

So will this be feasible or will a problem persist?
Which problem?

So each flash object has only a maximum of 128kb? That means even if i subscribe to 2000 object, it will have ADL_FLH_RET_ERR_MEM_FULL error.

[blackyblack]

if i keep writing on the same object, it does flh_erase and flh_write (according to documentation), but does this constitute to an erasure cycle and the 100,000 limit, or the chipset has a smart way of knowing not to do Garbage collector just yet etc etc etc.

Even if you are writing the same object it will be moved over the free flash space. And it will not be actually erased but marked as erased. And when you will have low free space the GC will trigger and actually erase your “erased” data.

So each flash object has only a maximum of 128kb?

No. The total flash availaible is 128K. And each flash object has a maximum of 30K.

That means even if i subscribe to 2000 object, it will have ADL_FLH_RET_ERR_MEM_FULL error.

I guess you will have this error much earlier than you expected. And I think you do not actually need 2000 objects.

[Jono]

Right. I might have misused the terms right there.

adl_flhSubscribe(“StoreForward”, 1400);

So the StoreForward is the handle, and each index in the handle is the object (1400)

So i can store up to 128kb on the handle, and each object at most 30kb. Not an issue as each object will be 90 bytes, so, 1400 is still a foreseeable mark.

I would love to use ADL, seeing there is more space available, but the recompaction is just a memory consuming and time consuming process on the program i have at the moment.

[blackyblack]

Actually FLASH facility is much easier than A&D (not ADL ). You do not care about garbage collection, recompaction. You have normal read/write/erase API. Yes the 128K is a big disadvantage, but you do not need to write a complex driver for A&D proper work.

[Jono]

And also, i just needed to nail down whether my scenario will cause a + count for the physical erasure limit of 100,000.