I’ved used Axios a ton, but I’ve only ever used it to make AJAX requests. In fact, I don’t ever remember a time when I’ve needed to download a file and save it to disk in Node.js.
That changed recently with a new side gig I’ve been helping out with. Axios was already part of the stack, and I knew how to make AJAX requests, but the file that I needed to save locally was a few hundred megabytes.
With the size what it was, I didn’t want to try loading the file in it’s entirety and then save it to disk, so I got to digging around with streams.
Streams are the best way to handle these type of scenarios, as it allows you to work on smaller chunks instead of the entirety of something. Smaller chunks tends to mean less memory overhead and that’s a good thing, especially if you want to run things in parallel!
Fortunately Axios supports a “stream” response type, but it wasn’t nearly as
straight forward as I would have liked when attempting to use it with
async
/await
syntax.
The trick to getting it to work was to promisify
the finished
method from
stream
so that it could be await
ed instead of just plowing through to the
next block of code.
The finished product looked something like this (with the IIFE added for demonstration purposes):
import * as fs from 'fs';
import * as stream from 'stream';
import axios from 'axios';
import { promisify } from 'util';
(async () => {
const finishedDownload = promisify(stream.finished);
const writer = fs.createWriteStream('/path/to/saved/file');
const response = await axios({
method: 'GET',
url: 'https://url/to/remote/file',
responseType: 'stream',
});
response.data.pipe(writer);
await finishedDownload(writer);
})();
Not a whole lot to it and given the lack of complexity, it’s fairly self-explanatory code too!