Occasionally when my (Q26 Extreme) module does an FTP upload (using WIP) it detects that it completed successfully, however the data isn’t actually received by the FTP server.
I’ve implemented two workarounds that have made this issue almost non-existent, but it can still fail.
My workarounds were:
A sleep delay of 4s in WIP_CEV_WRITE after bytes_written == bytes_total before closing the FTP data channel.
A sleep delay of 4s between closing the FTP data channel and closing the FTP control channel. Without this the file isn’t properly closed from the FTP server’s perspective.
Has anyone else encountered this and found a better solution to it? I used to have it check the file size before and after an upload, but this isn’t possible for me anymore.
It seems like I can only add a Finalizer callback to the FTP control channel?
I gave it a go. It seems to fix issue #2 (if I remove the delay workaround), however I still need the #1 delay before closing the data channel. Is there some sort of event or callback that I should be waiting for after WIP_CEV_WRITE has called wip_write() enough times to upload all data?
Edit: No, correction - using a Finalizer doesn’t fix either issue after all. =(
I’ve just dug up my FTP code for another look and noted the following things in the WIP_CEV_WRITE handler for the DATA channel:
I’m checking the return value from wip_write() function. If this is 0, then the WIP TX buffer is full/nothing was going up the stick, and I break out of the event without updating my internal write buffer. I get another WIP_CEV_WRITE event when the stack is ready for more data. If the return is non(and greater than)-zero, then this many bytes were added to the WIP TX buffer (note that this may NOT be the number specified in the call to wip_write()!) and can be removed from my internal buffer.
When I’ve emptied my internal write buffer into the WIP TX buffer using wip_write(), I call wip_shutdown(), rather than wip_close(). I’ve but my code (and comments) below:
// use wip_shutdown() here rather than wip_close() as close terminates the connection NOW,
// even though all data may not have been uploaded!
// wip_shutdown() will force a WIP_CLOSE event which will call the finalizer() when all is done
wip_shutdown( myDataChannel, FALSE, TRUE );
break;
Now, this use of wip_shutdown() instead of wip_close() IS NOT explicitly documented in the API docs - but I remember finding it after plowing through some sample code. wip_shutdown() appears to mark as closed only HALF of the channel (i.e. in this case the UPLOAD side), but doesn’t tear down the connection as wip_close() does. I’ve had a similar issue when using HTTP to upload large quantities of data, and then wait for a reply.
Note that you should handle the cleanup in your finalizer, not the WIP_CEV_PEER_CLOSE event for your DATA handler. In fact, I can’t remember if closing the DATA channel calls WIP_CEV_PEER_CLOSE at all unless there is an issue with the remote server.
Again, note that the use of the wip_shutdown() function is not well documented in the API.
Using wip_shutdown() on the data channel looks promising. Do you call wip_close() on the control channel immediately after, or do you use wip_shutdown on that too? I’m finding that if I try to close the control channel I get -989 (no route to host), which I believe is why I avoided using wip_shutdown in the first place.
I’ve been through my code some more, and it appears that I only call wip_close() if there’s been an error and it’s time to clean up the mess. Otherwise, both my data and control handlers simply ignore (well, output a trace message) the WIP_CEV_CLOSE events. It appears from my code that both channels are closed by the time the finalizer is called - as all that is done in the finalizer is that memory associated with the transfer is released.
It’s been a while since I did this, but there’s something in the back of my mind that’s saying with a FTP wip_putFile(), only the write channel is opened, and calling wip_shutdown() marks the uploaded file as complete. There’s a one-line note in the API (section 7.4.2) for wip_putFile() that indicates that only possible event is WIP_CEV_WRITE. However, the wip_shutdown() doco (6.5.6) indicates that a WIP_CEV_PEER_CLOSE event is sent to the peer (server??) socket when the output comms is closed.
In my code, it appears that the WIP stack calls wip_close() internally after the contents of the wip write buffer have been uploaded, and then calls the finalizer after the internal wip_close() has completed.
As I said, it’s been a while since I’ve done this - all my own applications use HTTP post to a server rather than FTP - as I was having issues with proxies and firewalls etc when using FTP. Note that it’s important to use wip_shutdown() after finishing the write when doing a HTTP POST - otherwise you won’t get the return from the web server!!!
And it doesn’t help that the API docs are fairly sparse and the examples provided are simplistic at best.
Hmm… calling wip_shutdown() does seem to close both the data and control channels (WIP_CEV_PEER_CLOSE, WIP_CEV_DONE are both called, in that order).
What do you do about the bearer, though? In my code I open and close it for every transfer. I now remember that I had awful trouble getting it to close properly and it was only after a lengthy process of trial and error that I ultimately settled on wip_close(data_channel); wip_close(control_channel); wip_bearerClose(bearer);.
Edit: I think I’ve got something stable working, without the use of the workaround sleep delays.
I never close the bearer unless the GPRS link fails (and detecting and dealing with that is a whole different ball game). But I don’t have the power requirements that I’m supposing that you do (from your 32k posts).
My gut feeling is that I would stop the bearer in the FTP finalizer rather than in the WIP_CEV_DONE event - that way you know absolutely that the transmission is complete and tidied up. Then close the bearer in the BEARER_STOPPED event handler as you are doing now.
Anyway, good to hear that you’ve managed to get it all together.