1

I'm developing a Chrome extension (typescript, browser target) with a dependency to a local npm module (node target).

The problem: my build pipeline is broken, most likely the sourcemap transformations (webpack > browserify > exorcist > sorcery) through various dependencies (my app > core library > node depencendies).

Project Setup & Build Pipeline

Multi module project setup

  • chrome extension "quickbrain.chrome"
    • this is what I want to build
    • browser target
  • core library "quickbrain.core"
  • some other apps that use the "quickbrain.core" module (node express)

Builld pipeline for 'quickbrain.chrome'

Error Message

When building I get the following artifacts and an error at the Sorcery stage.

  • Webpack -> contentscript.js, contentscript.js.map
  • Browserify -> Exorcist -> contentscript.exorcist.js, contentscript.exorcist.js.map
  • Sorcery -> contentscript.final.js, concentscript.final.js.map
(node:30724) UnhandledPromiseRejectionWarning: Error: ENOENT: no such file or directory, open 'C:\QB\com.quickbrain\chrome\dist.development\core\node_modules\underscore\underscore-node-f.cjs.map'

Underscore / underscore-node is a dependency from a node module 'natural', used within 'quikbrain.core' and I didn't have that problem in the past.

Hints

  1. I can see that the root path in the error message is completely wrong:

    • Error Message: com.quickbrain\____chrome\dist.development____\core\node_modules\underscore\underscore-node-f.cjs.map (note the wrong absolute path highlighted by ____)
    • Actual File: com.quickbrain\core\node_modules\underscore\underscore-node-f.cjs.map

  2. When looking into the sourcemap itself I can see the same relative path for other sourcemaps as well, e.g. webpack:///../core/node_modules/axios/index.js vs webpack:///../core/node_modules/underscore/underscore-node-f.cjs, but only the latter causes the error message in sorcery with the wrong absolute path.

  3. Builds in the past didn't include the underscore-node sourcemap. There is already a difference after running webpack, but I can't figure out why it is now included.

  4. Please note that quickbrain.chrome installs quickbrain.core as a local dependency "quickbrain.core": "file:../core"

Reference Files

> Browserified parts of contentscript.js and contentscript.js.map

const _ = __webpack_require__(/*! underscore */ "../core/node_modules/underscore/underscore-node.cjs") 
 "webpack:///../core/node_modules/underscore/underscore-node-f.cjs","webpack:///../core/node_modules/underscore/underscore-node.cjs",

> quickbrain.chrome > package.json (only relevant parts)

{ "name": "quickbrain.chrome", "version": "0.3.0", "description": "", "main": "index.js", "watch": { "webpack-dev": { "patterns": [ "src" ], "extensions": "ts,html" } }, "scripts": { "webpack-dev": "webpack --env NODE_ENV=development --config webpack.config.js && npm run browserify-contentscript-dev && npx node sorcery_finalize_sourcemaps.js", "browserify-contentscript-dev": "browserify dist.development/contentscript.js --debug --ignore-missing | exorcist --error-on-missing --root ../ dist.development/contentscript.exorcist.js.map > dist.development/contentscript.exorcist.js", }, "author": "", "license": "ISC", "devDependencies": { "@types/chrome": "0.0.127", "@types/node": "^14.14.28", "awesome-typescript-loader": "^5.2.1", "browserify": "^17.0.0", "clean-webpack-plugin": "^3.0.0", "convert-source-map": "^1.7.0", "copy-webpack-plugin": "^7.0.0", "exorcist": "^1.0.1", "node-sass": "^5.0.0", "npm": "^6.14.10", "npm-run-all": "^4.1.5", "npm-watch": "^0.7.0", "sass-loader": "^10.1.0", "sorcery": "^0.10.0", "sourceify": "^1.0.0", "ts-node": "^9.1.1", "typescript": "^4.1.3", "webpack": "^5.10.3", "webpack-cli": "^4.2.0" }, "dependencies": { "@types/ejs": "^3.0.5", "@types/jquery": "^3.5.5", "axios": "^0.21.0", "quickbrain.core": "file:../core", "ejs": "^3.1.5", "html-loader": "^1.3.2", "jquery": "^3.5.1", "nodejs-base64-converter": "^1.0.5" } }

> quickbrain.chrome > tsconfig.json

{ "compileOnSave": false, "compilerOptions": { "target": "es5", "module": "commonjs", "sourceMap": true, "inlineSourceMap": false, "inlineSources": false, "types": ["@types/chrome", "node", "jquery"], "typeRoots": ["node_modules/@types"], "moduleResolution": "node", "esModuleInterop": true, "lib": ["es2017", "dom"] }, "include": ["src/background", "src/contentscript-chrome"] }

> quickbrain.chrome > sorcery_finalize_sourcemaps.js (only relevant parts)

var sorcery = require("sorcery"); let i = 0; function doMagic(src, dst) { console.log(`loading ` + src); sorcery.load(src).then(function(chain) { // generate a flattened sourcemap var map = chain.apply(); // { version: 3, file: 'code.min.js', ... } // get a JSON representation of the sourcemap map.toString(); // '{"version":3,"file":"code.min.js",...}' // get a data URI representation map.toUrl(); // 'data:application/json;charset=utf-8;base64,eyJ2ZXJ...' // write to a new file, but append the flattened sourcemap as a data URI chain.write(dst, { inline: true }); }); } try { doMagic( "dist.development/contentscript.exorcist.js", "dist.development/contentscript.final.js" ); console.log("\n\n_,.~*`( sorcery done )`*~.,_\n"); } catch (e) { console.error("Computer says no: " + e + " :("); }

(When I just use npx sorcery --input dist.development/contentscript.exorcist.js --output dist.development/contentscript.sorcery.js I get the same error message)

> quickbrain.core > package.json

{ "name": "quickbrain.core", "version": "0.2.0", "description": "", "main": "index.js", "types": "index.d.ts", "scripts": { "test": "echo \"Error: no test specified\" && exit 1", "build": "npx tsc" }, "author": "", "license": "ISC", "dependencies": { "@types/axios": "^0.14.0", "@types/mongoose": "^5.10.3", "@types/natural": "^2.1.1", "@types/wink-tokenizer": "^4.0.2", "atlassian-jwt": "^1.0.3", "axios": "^0.21.1", "compute-cosine-similarity": "^1.0.0", "express": "^4.17.1", "fuzzball": "^1.3.1", "jwt-decode": "^3.1.2", "moment": "^2.29.1", "mongoose": "^5.11.17", "natural": "^4.0.3", "stopwords-iso": "^1.1.0", "string-strip-html": "^8.2.2", "ts-lib": "0.0.5", "tslib": "^2.1.0", "wink-tokenizer": "^5.2.3" }, "devDependencies": { "mongodb-memory-server": "^6.9.6", "typescript": "^4.2.3" } }

> quickbrain.core > tsconfig.json

{ "compilerOptions": { "target": "es5", /* Specify ECMAScript target version: 'ES3' (default), 'ES5', 'ES2015', 'ES2016', 'ES2017', 'ES2018', 'ES2019', 'ES2020', or 'ESNEXT'. */ "module": "commonjs", /* Specify module code generation: 'none', 'commonjs', 'amd', 'system', 'umd', 'es2015', 'es2020', or 'ESNext'. */ "declaration": true, /* Generates corresponding '.d.ts' file. */ "outDir": "./dist", /* Redirect output structure to the directory. */ "strict": true, /* Enable all strict type-checking options. */ "esModuleInterop": true, /* Enables emit interoperability between CommonJS and ES Modules via creation of namespace objects for all imports. Implies 'allowSyntheticDefaultImports'. */ "skipLibCheck": true, /* Skip type checking of declaration files. */ "forceConsistentCasingInFileNames": true /* Disallow inconsistently-cased references to the same file. */ } }
Improve this question
5
  • 1
    This PR (from 2016!) seems to aim to make sure that sorcery.js does not just crash when encountering a missing source map file. Commented Apr 24, 2021 at 8:04
  • 1
    This issue (from 2017!) complains about that very same problem. Commented Apr 24, 2021 at 8:05
  • Thanks. What puzzles me is that (1) the sourcemap is actually there and that in the input file to sorcery (2) it has the same relative path '../core/node_modules/..' as others where no problem occurs. Commented Apr 25, 2021 at 6:43
  • 1
    Tought to say. Might have to do with the *.cjs file ending. Try to dig deeper: How does sorcery load the files? Why does it load one, and not the other? If it is missing, it is probably a problem in the previous steps, where one guy in the pipeline dropped the ball. Commented Apr 25, 2021 at 8:32
  • 1
    Your guess went into the right direcion. I was able to fix this in the early steps of the pipeline, influencing browserify to ignore precompiled sourcemaps within node_module. Thanks! Commented Apr 25, 2021 at 8:53

1 Answer 1

Reset to default
0

Edit: Unfortunately not solved. The pipeline works, but I am missing most of the sourcemaps

I was able to fix this using https://stackoverflow.com/a/55356078/5244937 (and https://github.com/browserify/browserify/issues/772) about Browserify and modules with precompiled sourcemaps (like underscore that is causing the trouble).

According to the post, Browserify will ignore 3rd party sourcemaps when tsconfig.json is set to inline-sourcemaps.

This means I have changed the tsconfig.json of quickbrain.chrome to

 "sourceMap": false, "inlineSourceMap": true, "inlineSources": true,

.. and Browserify (2st step of the pipeline) drops the sourcemaps of natural, underscore, etc avoiding the problem at the end of the pipeline altogether.

Fun fact: "inlineSourceMap": true is not even used by my pipeline, since I compile using webpack with external sourcemaps. The only effect is influencing Browserfify to drop other sourcemaps.

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.