How To Fix My Stream/writing Data Closing Really Slow With NodeJS And Promise-ftp
I have to download a json file from a ftp server. Getting the file goes really fast but then the read/writing with stream closes really slow. The file is on my desktop almost instantly but its not closing.
I've tryed promise-ftp & just ftp. I updated my nodejs and node to the latest version so I tryed multiple version.
const getFile = () => {
return new Promise((resolve, reject) => {
console.log('getting file.');
ftp.connect(config)
.then(function (serverMessage) {
return ftp.get(remoteFile + '.json');
}).then(function (stream) {
return new Promise(function (resolve, reject) {
stream.once('close', resolve);
stream.once('error', reject);
stream.pipe(fs.createWriteStream(`${steamid}.json`));
});
}).then(async function () {
console.log('done editing');
await ftp.end();
return resolve();
}).catch(console.log);
});
};
I expect this to download and close in like 1 second. It takes a solid 10 seconds to close the file but I can already see it instantly on my pc and read it. Its about 700kb for 1 file.
EDIT:
The FTP connection closes instantly but the stream.pipe(fs.createWriteStream(${steamid}.json
)); takes 10 seconds to call the 'close' event eventhough the file looks fully downloaded instantly.
Its like it doesn't know there isn't more data coming.
EDIT 3: So the files I download are around 700 bytes, its just 1 chunk maximum everytime. I fixed it by adding a "data" event on the readstream and after the 1 chunk I just destroy it. My whole download, edit, upload works in less than a second now. This isn't probably an ideal solution.
Answer
Based on the promise-ftp
npm docs, if you want to immidiately close the FTP connection you should use the following method:
destroy(): Closes the connection to the server immediately. Returns a boolean indicating whether the connection was connected prior to the call to destroy().
In your case that would be:
const getFile = () => {
return new Promise((resolve, reject) => {
console.log('getting file.');
ftp.connect(config)
.then(function (serverMessage) {
return ftp.get(remoteFile + '.json');
}).then(function (stream) {
return new Promise(function (resolve, reject) {
stream.once('close', resolve);
stream.once('error', reject);
stream.pipe(fs.createWriteStream(`${steamid}.json`));
});
}).then(async function () {
console.log('done editing');
await ftp.destroy();
return resolve();
}).catch(console.log);
});
};
Related Questions
- → Maximum call stack exceeded when instantiating class inside of a module
- → Browserify api: how to pass advanced option to script
- → Node.js Passing object from server.js to external modules?
- → gulp-rename makes copies, but does not replace
- → requiring RX.js in node.js
- → Remove an ObjectId from an array of objectId
- → Can not connect to Redis
- → React: How to publish page on server using React-starter-kit
- → Express - better pattern for passing data between middleware functions
- → Can't get plotly + node.js to stream data coming through POST requests
- → IsGenerator implementation
- → Async/Await not waiting
- → (Socket.io on nodejs) Updating div with mysql data stops without showing error