FTP: Is there a way to know a file has been written

Hi,

I’m developing an application that put a file on an ftp and then rename the file when it has been completely written.

The problem I am facing is making sure the writing of the file is done on the server side.

I saw a wip_shutdown() function that is supposed to be called to when the writing is done from modem side and then a WIP_CEV_PEER_CLOSE event arrives via a callback on ftp_control_channel.

1/ I’m lead to think that when I receive WIP_CEV_PEER_CLOSE event the control channel is not usable any more, am I correct ?

2/ I’m no even sure that when I receive this event the file is fully written on server side ?

3/ Is there any method no involving timers that can ensure that the file has been fully written on the ftp server without closing the ftp_control_channel ?

Thanks in advance

Hey even m facing the same problem…as a temp solution m usin a timer of 1 minute … within this time m able to write the file succesfully but i feel this time is too long and can slow the process. u found any solution reg the same?

Using wip_shutdown() should be the right way . You can check the size of the file using wip_getFileSize().

This works fine in my application if I check the pEvent->content.done.result for 213 after the WIP_CEV_DONE event was received.

But you must check the result because it is possible to receive ‘transfer complete’ (result = 226) as the first answer. In this case you must repeat the call of wip_getFileSize().

hi Ralf,
I haven’t been able to solve the same problem, would you mind giving me a hand?,

I wrote a function that just gives you the size of a file, I mean, using:

k=wip_getFileSize(src_ftp_cx,SRC_FILENAME);

I can see this during the execution:

[FTP=>] “220 Xlight FTP Server 3.1 ready…”
[FTP<=] USER javier
[FTP=>] “331 Password required for javier”
[FTP<=] PASS javier
[FTP=>] “230 Login OK”
[FTP<=] TYPE I
[FTP=>] “200 Type set to I.”
WIP_CEV_OPEN
FTP connectionOK**************

[FTP<=] SIZE 12.html
ERRLOG ftp.c:155: Assertion Failure:

STATE_READY == cfg->internal_state

[FTP<=] SIZE 12.html
[FTP=>] “213 28”
size of the file : 213 bytes //this is the response I’m getting
[FTP=>] “213 28”
ERRLOG ftp_evh.c:126: Unexpected reply arriving from server

As you can see, there is a couple of errors which I don’t know what they mean,
on the other hand I’m not receiving WIP_CEV_PEER_CLOSE (actually, nothing hapens
after this point)

So I still don’t know how to close the channel, my server program is showing a new user each time I
execute the function I described above,

would you post a piece of code for supporting your comments?

any advice or help on how to acomplish this would be appreciated
:slight_smile:
Javier


case WIP_CEV_DONE:
if(!k){
k=ev->content.done.result;
wm_sprintf(buf,“size of the file : %d bytes \r\n”,k);
adl_atSendResponse(ADL_AT_RSP,buf)
}


I read the ftp file size in the following way, it works file.

wip_getFileSize(dst_ftp_cx,DST_FILENAME);




/* get the file size to kwow where to append data */
case WIP_CEV_DONE:
k=ev->ev->content.done.aux;
wm_sprintf(buf,“size of the file : %d bytes \r\n”,k);

according the wip manual:
the wip_getFileSize function is used to get the file size in bytes. on success, a wip_cev_done event is sent to ftpCtx with event->content.done.aux set to file’s size. on failure, a WIP_CEV_ERROR event is triggered.

Hi,
since 2009, is there a strong response in order to develop a simple application that put a file on an ftp server and then rename the file when it has been completely written?

(I am using a fastrack FXT009)

Thanks for your response.

Hi,

There are 3 sample applications provided within the Internet Library which can help you to understand the FTP APIs:

  • ftp_list (gets a listing of the files available at the FTP server’s root, and displays that list in WIP traces).
  • ftp_put (sends a dummy text file to a FTP server).
  • ftp_ftp (gets a file from one FTP server, converts its content to uppercase, and puts the result on another FTP server).

To rename a file on a FTP server, have a look to the wip_renameFile function.

Best Regards,

JayM2M

Hi, Thanks for your response.
Separately, the ftp_put, wip_list, wip_renamefile are working.

The question is: on a same program, how is it possible to know that a file has been correctly transfered (ftp) in order to be renammed?

After sending the file (last WIP_CEV_WRITE), the wip_getState returns WIP_CSTATE_BUSY. How long to wait in order to execute the wip_renamefile correctly?

Regards.
H. Puech

Hi,

In order to know if the transfer is terminated, you have to wait for WIP_CEV_DONE event.
Then you can initiate a new “FILE” command (new transfer, rename, change directory, …):

Best Regards,

JayM2M

Hi,

After sending a file through ftp, I never receive the WIP_CEV_DONE. In order to make a test, at the last WIP_CEV_WRITE, i have put a loop of wip_getSate (with a timer) and the return is always WIP_CSTATE_BUSY …

Thanks.
H. Puech

Hi,

On my side, I’m able to see the WIP_CEV_DONE event as soon as the file has been completly received by the FTP server :confused:

Could you confirm that you are waiting for the WIP_CEV_DONE event in the Control handler (the callback function provided in wip_FTPCreateOpts) and not in the Data handler (the callback function provided in wip_putFile)?

Also, do not close the Control channel when you have finished to send all the data of your file (close only the Data channel).
Then you can reuse the same Control channel to execute the wip_renameFile function (do not wait too long if you don’t want to have Control channel closed by the server due to timeout).

Best Regards,

JayM2M

Hi,
I confirm that i never receive the WIP_CEV_DONE event as soon as the file has been completly received by the FTP server…

I have made many tests and this is the best result: after sending the file through ftp, I close all channel (data and control). I re open a ftp session in order to renamme to file over ftp.

It is not the best solution but is works (see the traces below)…

Regards.
H. Puech

open ftp Opening FTP connection…
(evh_cx)WIP_CEV_OPEN received
ftp state -> WIP_CSTATE_READY
(evh_data): WIP_CEV_OPEN received
(evh_data) Ftp write: 5840 bytes sent

(evh_data) Ftp write: 775 bytes sent
(evh_data): Ftp data transfer done (by calculating the number of written bytes)
(evh_data): wip shutdown dst_data 0
(evh_data): WIP_CEV_PEER_CLOSE received
(evh_data): wip close dst_data
(evh_data): wip shutdown dst_ftp_cx
(evh_cx) WIP_CEV_PEER_CLOSE received
(evh_cx): wip close dst_ftp_cx 0
(evh_cx2)WIP_CEV_OPEN received
(evh_cx2): start renaming file
(evh_cx2) WIP_CEV_DONE received
(evh_cx2) file renamed successfully
(evh_cx2): wip shutdown dst_ftp_cx2
(evh_cx2) WIP_CEV_PEER_CLOSE received
(evh_cx2): wip close dst_ftp_cx2

Hi,

Could you activate the WIP traces in order to see if the FTP server is sending an acknowledgement when the file is received (“226 Transfer Complete”).
To activate WIP traces, you must add an option in the WIP init function. Example:

wip_netInitOpts( WIP_NET_OPT_DEBUG_PORT, WIP_NET_DEBUG_PORT_TRACE,
                 WIP_NET_OPT_END);

I’m using the latest software package (FW R7.51, OAT 2.51, Internet Lib 5.54).
Here is the log of my FTP session (send a file and then rename it):

The “FTP CTRL → DONE” trace is dumped by my FTP Control Event Handler function (ftp_ctrl_evh) when the WIP_CEV_DONE event is received.
As you can see in my log, I’m able to see this event when the FTP sever has acknowledged the file transfer and then the file rename.

Here is my “quick & dirty” code (incomplete, as the GPRS related functions are in a separate file):

#include "adl_global.h"
#include "generated.h"
#include "wip.h"
#include "gprs.h"

#define FTP_SERVER    "ftpperso.free.fr"     /* Address of the FTP server */
#define FTP_USER      "XXXXX"                /* FTP username */
#define FTP_PASSWORD  "YYYYY"                /* FTP password */
#define FTP_FILENAME  "test_1.txt"           /* Name of the file to put */
#define FTP_FILERENAME  "test_rename.txt"    /* Rename of the file */
#define FTP_ISPASSIVE FALSE                  /* Active or passive mode? */

/* buffer of data to send in [FTP_FILENAME]. */
static u8 ftp_file_buffer [200];

/* How many bytes of [ftp_file_buffer] have already been sent. */
static int ftp_file_offset = 0;

static wip_channel_t
  ftp_ctrl   = NULL, /* channel controlling the FTP connection. */
  ftp_data   = NULL; /* channel used for file data transfer. */

static void ftp_ctrl_evh( wip_event_t *ev, void *ctx);
static void ftp_data_evh( wip_event_t *ev, void *ctx);

bool ftp_wait_for_rename = FALSE;

/***************************************************************************/
/*  Function   : ftp_start                                                 */
/*-------------------------------------------------------------------------*/
/*  Object     : Start the FTP session.                                    */
/*-------------------------------------------------------------------------*/
/***************************************************************************/
void ftp_send(ascii *FileContent)
{
	if ( wm_strlen(FileContent) > 199 )
	{
		wip_debug( "Too much data!\n");
		return;
	}
	else if ( wm_strlen(FileContent) == 0 )
	{
		wip_debug( "No data!\n");
		return;
	}

	wm_strcpy( (ascii *)ftp_file_buffer, FileContent );

	wip_debug( "Opening FTP connection...\n");

  ftp_file_offset = 0;

  ftp_ctrl = wip_FTPCreateOpts (
             FTP_SERVER,  ftp_ctrl_evh, NULL,
             WIP_COPT_USER,     FTP_USER,
             WIP_COPT_PASSWORD, FTP_PASSWORD,
             WIP_COPT_PASSIVE,  FTP_ISPASSIVE,
             WIP_COPT_END);

  if( ! ftp_ctrl) { wip_debug( "Can't open destination FTP connection\n"); return; }
}


/***************************************************************************/
/*  Function   : ftp_data_evh                                              */
/*-------------------------------------------------------------------------*/
/*  Object     : Handling events happenning on the FTP connection          */
/*               control channel.                                          */
/*-------------------------------------------------------------------------*/
/*  Variable Name     |IN |OUT|GLB|  Utilisation                           */
/*--------------------+---+---+---+----------------------------------------*/
/*  ev                | X |   |   |  WIP event                             */
/*--------------------+---+---+---+----------------------------------------*/
/*  ctx               |   |   |   |  unused                                */
/*--------------------+---+---+---+----------------------------------------*/
/***************************************************************************/
static void ftp_ctrl_evh( wip_event_t *ev, void *ctx) {

  switch( ev->kind) {

  case WIP_CEV_OPEN:
    /* FTP control initialized successfully -> open data channel */
    ftp_data = wip_putFile( ftp_ctrl, FTP_FILENAME, ftp_data_evh, NULL);
    if( ! ftp_data) { wip_debug( "Can't create data channel"); return; }
    break;

  case WIP_CEV_DONE:
	wip_debug( "FTP CTRL -> DONE\n");
	if (ftp_wait_for_rename)
	{
		ftp_wait_for_rename = FALSE;
		wip_renameFile( ftp_ctrl, FTP_FILENAME, FTP_FILERENAME );
	}

	break;

  case WIP_CEV_ERROR:
    wip_debug( "Error %i on cx channel\n", ev->content.error.errnum);
    /* fall through */

  case WIP_CEV_PEER_CLOSE:
    wip_debug( "Closing everything\n");
    if( ftp_data) { wip_close( ftp_data); ftp_data = NULL; }
    wip_close( ftp_ctrl); ftp_ctrl = NULL;
    break;

  default:
	  wip_debug( "Event %d\n", ev->kind);
	  break;
  }
}

/***************************************************************************/
/*  Function   : ftp_data_evh                                              */
/*-------------------------------------------------------------------------*/
/*  Object     : Handling events happenning on the data transfer channel   */
/*-------------------------------------------------------------------------*/
/*  Variable Name     |IN |OUT|GLB|  Utilisation                           */
/*--------------------+---+---+---+----------------------------------------*/
/*  ev                | X |   |   |  WIP event                             */
/*--------------------+---+---+---+----------------------------------------*/
/*  ctx               |   |   |   |  unused                                */
/*--------------------+---+---+---+----------------------------------------*/
/***************************************************************************/
static void ftp_data_evh( wip_event_t *ev, void *ctx) {

  switch( ev->kind) {

  case WIP_CEV_OPEN:
    /* Here, we could optionnally change the threshold which triggers
     * [WIP_CEV_WRITE] events with the following setting:

     wip_setOpts( ev->channel, WIP_COPT_SND_LOWAT, 3000, WIP_COPT_END);

     * (This way, we wouldn't receive WRITE events until at least 3KB
     *  of TCP send ftp_file_buffer are available).
     */
    break;

  case WIP_CEV_DONE:
	wip_debug( "FTP DATA -> DONE\n");
	break;

  /* There is some space available in TCP send ftp_file_buffer, which allows me
   * to write more data from [ftp_file_buffer]; let's write as much data as
   * possible, up to the end of [ftp_file_buffer]. [nwrite] is the number of bytes
   * I actually managed to put in the TCP ftp_file_buffer. */
  case WIP_CEV_WRITE: {
    int nwrite = wip_write( ftp_data, (void *)ftp_file_buffer, wm_strlen( (ascii *)ftp_file_buffer) - ftp_file_offset);
    if( nwrite < 0) { TRACE (( 1, "ftp_data_evh: Failed to write data (%d)", nwrite )); return; }
    TRACE (( 1, "ftp_data_evh: %d bytes have been written", nwrite ));
    ftp_file_offset += nwrite; /* Update the number of bytes sent. */
    if( ftp_file_offset == wm_strlen( (ascii *)ftp_file_buffer)) { /* All of [ftp_file_buffer] has been sent */
      wip_close( ftp_data);   ftp_data   = NULL;
      //Warning: Don't close the Control channel here as we need to do other stuff (e.g. rename the file)
      ftp_wait_for_rename = TRUE;
    }
    break;
  }

  case WIP_CEV_ERROR:
    wip_debug( "Error %i on channel 0x%x\n", ev->content.error.errnum,
               ev->channel);
    wip_close( ftp_data); ftp_data = NULL;
    if( ftp_ctrl) { wip_close( ftp_ctrl); ftp_ctrl = NULL; }
    break;
  }
}

void atFTP_Handler( adl_atCmdPreParser_t *paras)
{
	if (gprs_isConnected())
	{
		// Send the file with the following content...
    ftp_send("This is just a test\r\n\r\nBye.");
		adl_atSendStdResponse(ADL_AT_PORT_TYPE(paras->Port,ADL_AT_RSP), ADL_STR_OK);
	}
	else
	{
		adl_atSendStdResponse(ADL_AT_PORT_TYPE(paras->Port,ADL_AT_RSP), ADL_STR_ERROR);
	}
}

void main_task ( void )
{
	int r;

	r = wip_netInitOpts(
			//WIP_NET_OPT_DEBUG_PORT, WIP_NET_DEBUG_PORT_UART1,
			//WIP_NET_OPT_DEBUG_PORT, WIP_NET_DEBUG_PORT_UART2,
			WIP_NET_OPT_DEBUG_PORT, WIP_NET_DEBUG_PORT_TRACE,
			WIP_NET_OPT_END);

	CfgGprs("websfr", "" , "");

	adl_atCmdSubscribe("AT+FTP", atFTP_Handler, ADL_CMD_TYPE_ACT);
}

Best Regards,

JayM2M

Ok thanks a lot.
Now I receive the DONE event. (I don’t know what was wrong in my last program…)
I can now rename the file in the same ftp session.

Regards.
H. Puech