1

This answer has the following way to download a file using the node-fetch library https://stackoverflow.com/a/51302466:

const downloadFile = (async (url, path) => { const res = await fetch(url); const fileStream = fs.createWriteStream(path); await new Promise((resolve, reject) => { res.body.pipe(fileStream); res.body.on("error", reject); fileStream.on("finish", resolve); }); });

How to mock this if a function has this method as part of it? I'm running into weird problems:

unit.test.js import { PassThrough } from 'stream'; import { createWriteStream, WriteStream } from 'fs'; import fetch, { Response } from 'node-fetch'; import mocked = jest.mocked; jest.mock('node-fetch'); jest.mock('fs'); describe('downloadfile', () => { const mockedFetch = fetch as jest.MockedFunction<typeof fetch>; const response = Promise.resolve({ ok: true, status: 200, body: { pipe: jest.fn(), on: jest.fn(), }, }); const mockWriteable = new PassThrough(); mocked(createWriteStream).mockReturnValueOnce(mockWriteable as unknown as WriteStream); mockedFetch.mockImplementation(() => response as unknown as Promise<Response>); it('should work', async () => { await downloadfile(); }), ); });

Throws:

Cannot read property 'length' of undefined TypeError: Cannot read property 'length' of undefined at GlobSync.Object.<anonymous>.GlobSync._readdirEntries (//node_modules/glob/sync.js:300:33) at GlobSync.Object.<anonymous>.GlobSync._readdir (//node_modules/glob/sync.js:288:17) at GlobSync.Object.<anonymous>.GlobSync._processReaddir (//node_modules/glob/sync.js:137:22) at GlobSync.Object.<anonymous>.GlobSync._process (/api/node_modules/glob/sync.js:132:10)

What could the solutions be?

Improve this question
4
  • 1
    Can you please share also the configuration that you have for jest? Commented May 25, 2022 at 21:23
  • There isn't really any jest specific config except the default typescript one. Commented May 26, 2022 at 15:52
  • 2
    Please provide a minimum reproducible example of your issue. In your code "createWriteStream" and "mockWriteable" aren't defined. Commented May 26, 2022 at 21:15
  • Maybe i'm wrong, but your function downloadFile awaits the response, but it's not returning the promise that should be awaited... ¿that could be the problem? Commented May 30, 2022 at 19:50

2 Answers 2

Reset to default
1

I see that the error is not related to tests. It is more about your test runner. Glob is a library to search files in your project. Check your test run command. Try to escape quotes \ double quotes.

Your package JSON will give more information that your test files.

There are some people that report this problem can be fixed by reinstalling node modules Try to

  1. Remove node_modules folder
  2. Remove the file package-lock.json
  3. Run npm install to re-install package dependencies.
Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

I dont think so because changing the test code changes the error, if it was about the test runner it would always throw the glob error.
1

Your base function has a fetch function which expects a URL from which the response body(data) is piped to the write stream and then written to the file system.

Mocking fetch is a bit tedious as jest mock has limited support for ES modules, More info here and here. So better test it by letting fetch download a file from a URL and then test the integrity of the downloaded file.

Now, calling the function inside your test case as follows:

const fileURL = "https://interactive-examples.mdn.mozilla.net/media/examples/lizard.png"; const fileDownloadPath = "/home/<your-username>/Downloads/downloaded_lizard.png"; // ----> Executing the function to be tested. await downloadFile(fileURL, fileDownloadPath);

Next, after downloading the file we can verify if the file exists at the download path and then test for it's integrity.

Integrity checking involves that the downloaded file checksum (md5 in our case) should match the checksum retrieved from the server.

If you are using a AWS S3 file URL then it should most probably return the checksum in the header named "ETag", also note that this is not true in every case but for small files at least this holds true.

If checksum is not available from the URL you can download the file using the specified URL by yourself and calculate its md5 checksum on the local machine then assign this checksum to a const variable inside the test case which can then be further used to test the integrity of the file.

const access_res = await fsp.access(fileDownloadPath, fs.constants.F_OK); // File should be accessible expect(access_res).toEqual(undefined); // denotes success // Compare md5 checksum of downloaded file with server file. const file_buffer = fs.readFileSync(fileDownloadPath); const downloaded_file_md5_checksum = createHash('md5').update(file_buffer).digest('hex'); const response_obj = await fetch(fileURL, { method: 'HEAD' }); const file_etag = JSON.parse(response_obj.headers.get('ETag')); expect(downloaded_file_md5_checksum).toEqual(file_etag); // Should match

Here is the full working example using ES modules:

import { jest } from '@jest/globals'; import fs from 'node:fs'; import fsp from 'node:fs/promises'; const { createHash } = await import('node:crypto'); import fetch from 'node-fetch'; import { downloadFile } from '../index'; // jest.setTimeout(15000); // in ms. Override if downloading big files. describe("Download service", () => { it("should download the file from provided url", async () => { const fileURL = "https://interactive-examples.mdn.mozilla.net/media/examples/lizard.png"; const fileDownloadPath = "/home/<your-username>/Downloads/downloaded_lizard.png"; // ----> Executing the function to be tested. await downloadFile(fileURL, fileDownloadPath); // ----> Testing if the file exists on fs. const access_res = await fsp.access(fileDownloadPath, fs.constants.F_OK); expect(access_res).toEqual(undefined); // denotes success // ----> Comparing downloaded file checksum with server file checksum. const file_buffer = fs.readFileSync(fileDownloadPath); const downloaded_file_md5_checksum = createHash('md5').update(file_buffer).digest('hex'); const response_obj = await fetch(fileURL, { method: 'HEAD' }); // Fetching checksum from server. const file_etag = JSON.parse(response_obj.headers.get('ETag')); expect(downloaded_file_md5_checksum).toEqual(file_etag); // Should match // ----> Finally remove downloaded file from local system. const rm_res = await fsp.unlink(fileDownloadPath); expect(rm_res).toEqual(undefined); // denotes success }) })
Improve this answer

2 Comments

This seems like an integration test. For example without internet it would fail.
If you are not limited to Jest you can use msw. This can act as a mock server, so no need to override fetch. You can set it up to get a file stream in the response body.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.