Ad

How To Fix My Stream/writing Data Closing Really Slow With NodeJS And Promise-ftp

- 1 answer

I have to download a json file from a ftp server. Getting the file goes really fast but then the read/writing with stream closes really slow. The file is on my desktop almost instantly but its not closing.

I've tryed promise-ftp & just ftp. I updated my nodejs and node to the latest version so I tryed multiple version.

const getFile = () => {
  return new Promise((resolve, reject) => {
    console.log('getting file.');
    ftp.connect(config)
      .then(function (serverMessage) {
        return ftp.get(remoteFile + '.json');
      }).then(function (stream) {
        return new Promise(function (resolve, reject) {
          stream.once('close', resolve);
          stream.once('error', reject);
          stream.pipe(fs.createWriteStream(`${steamid}.json`));
        });
      }).then(async function () {
        console.log('done editing');
        await ftp.end();
        return resolve();
      }).catch(console.log);
  });
};

I expect this to download and close in like 1 second. It takes a solid 10 seconds to close the file but I can already see it instantly on my pc and read it. Its about 700kb for 1 file.

EDIT: The FTP connection closes instantly but the stream.pipe(fs.createWriteStream(${steamid}.json)); takes 10 seconds to call the 'close' event eventhough the file looks fully downloaded instantly.

EDIT 2: There is a gap of 10 seconds after the 680 (thats the chunk.length) and then the Write stream is finished.

Its like it doesn't know there isn't more data coming.

EDIT 3: So the files I download are around 700 bytes, its just 1 chunk maximum everytime. I fixed it by adding a "data" event on the readstream and after the 1 chunk I just destroy it. My whole download, edit, upload works in less than a second now. This isn't probably an ideal solution.

Ad

Answer

Based on the promise-ftp npm docs, if you want to immidiately close the FTP connection you should use the following method:

destroy(): Closes the connection to the server immediately. Returns a boolean indicating whether the connection was connected prior to the call to destroy().

In your case that would be:

const getFile = () => {
  return new Promise((resolve, reject) => {
    console.log('getting file.');
    ftp.connect(config)
      .then(function (serverMessage) {
        return ftp.get(remoteFile + '.json');
      }).then(function (stream) {
        return new Promise(function (resolve, reject) {
          stream.once('close', resolve);
          stream.once('error', reject);
          stream.pipe(fs.createWriteStream(`${steamid}.json`));
        });
      }).then(async function () {
        console.log('done editing');
        await ftp.destroy();
        return resolve();
      }).catch(console.log);
  });
};
Ad
source: stackoverflow.com
Ad