Got to use node.js for a work project recently. We needed an FTP server with special user authentication that would run custom code after a file was uploaded. There was one node.js FTP server implementation on github, so I forked it and started rounding out the basic functionality. My fork is here.
The first significant change I made was to encapsulate the data connection logic. File lists and file contents are transferred over the data connection (FTP commands and responses over the control connection). I quickly found that some clients are super eager to send you data and will do so once a passive data connection is made, even before the FTP server tells them it’s ok to do so. This was especially problematic for file uploads over passive data connections. Without a workaround for these aggressive clients, the flow looked something like this:
- Receive PASV command from client
- Start listening on a port and tell client which port to connect to
- Receive STOR command from client, stating it’s going to upload a file
- Attach data listener to data connection that saves incoming data to file
- On data connection end event, make sure file data has been written
- Close the file
Due to the asynchronous nature of node.js, data often arrived between steps 3 and 4, before a data event listener had been attached. Some of the file chunks were falling through the cracks and the saved file was incomplete.
Also regarding the above flow, ensuring that all data had been written to disk before we attempted to close file was … well … convoluted. Maybe a solution could be found using fs.createWriteStream, but I’m still happier with what I’ll outline next.
Once I found out that data was falling through the cracks I did some searching and found that setting up a buffering data handler was a common solution to a common problem. Good to know! With a persistent data buffer in place things became much easier, like so:
- Receive PASV command from client
- Start listening on a port and tell client which port to connect to
- Once we get hint of a connection (socket connect event), listen for data events and push each data chunk onto a stack
- Receive STOR command from client, stating it’s going to upload a file
- On data connection end, open file
- Loop over buffered data
- Save each buffered chunk before moving to next chunk
- Close the file
It’s less than ideal to buffer each uploaded file in memory first, but it works and is simple. Success!
In addition to the above, I’ve encapsulated things further in my forked repo. The FTP server object itself emits some additional events which you can listen for. My goal was to encapsulate the basic FTP functionality and provide a way to take special actions when:
- A user connects
- A user attempts to log in (listen for these events and handle authentication yourself)
- User uploads a file
Hopefully someone with a slightly different use-case will come along and take things the rest of the way.