I’ved used Axios a ton, but I’ve only ever used it to make AJAX requests. In
fact, I don’t ever remember a time when I’ve needed to download a file and save it to disk in Node.js.
That changed recently with a new side gig I’ve been helping out with. Axios was
already part of the stack, and I knew how to make AJAX requests, but the file
that I needed to save locally was a few hundred megabytes.
With the size what it was, I didn’t want to try loading the file in it’s entirety and then save it to disk, so I got to digging around with streams.
Streams are the best way to handle these type of scenarios, as it allows you to
work on smaller chunks instead of the entirety of something. Smaller chunks
tends to mean less memory overhead and that’s a good thing, especially if you
want to run things in parallel!
Fortunately Axios supports a “stream” response type, but it wasn’t nearly as
straight forward as I would have liked when attempting to use it with async
/await
syntax.
The trick to getting it to work was to promisify
the finished
method fromstream
so that it could be await
ed instead of just plowing through to the
next block of code.
The finished product looked something like this (with the IIFE added for
demonstration purposes):
import * as fs from 'fs';
import * as stream from 'stream';
import axios from 'axios';
import { promisify } from 'util';
(async () => {
const finishedDownload = promisify(stream.finished);
const writer = fs.createWriteStream('/path/to/saved/file');
const response = await axios({
method: 'GET',
url: 'https://url/to/remote/file',
responseType: 'stream',
});
response.data.pipe(writer);
await finishedDownload(writer);
})();
JavaScriptNot a whole lot to it and given the lack of complexity, it’s fairly
self-explanatory code too!