improve: wasm build by Uzlopak · Pull Request #3074 · nodejs/undici

Uzlopak · 2024-04-09T00:35:30Z

should shrink the wasm file. removes build artifacts and other files we dont need.

build/wasm.js

+ execSync(`${WASM_CC} ${WASM_CFLAGS} ${WASM_LDFLAGS} \
+ ${join(WASM_SRC, 'src')}/*.c \
+ -I${join(WASM_SRC, 'include')} \
+ -o ${join(WASM_OUT, 'llhttp.wasm')} \
+ ${WASM_LDLIBS}`, { stdio: 'inherit' })


build/wasm.js

+ -o ${join(WASM_OUT, 'llhttp.wasm')} \
+ ${WASM_LDLIBS}`, { stdio: 'inherit' })
+
+ execSync(`./wasm-opt ${WASM_OPT_FLAGS} -o ${join(WASM_OUT, 'llhttp.wasm')} ${join(WASM_OUT, 'llhttp.wasm')}`, { stdio: 'inherit' })


build/wasm.js

+ execSync(`${WASM_CC} ${WASM_CFLAGS} -msimd128 ${WASM_LDFLAGS} \
+ ${join(WASM_SRC, 'src')}/*.c \
+ -I${join(WASM_SRC, 'include')} \
+ -o ${join(WASM_OUT, 'llhttp_simd.wasm')} \
+ ${WASM_LDLIBS}`, { stdio: 'inherit' })


build/wasm.js

+ -o ${join(WASM_OUT, 'llhttp_simd.wasm')} \
+ ${WASM_LDLIBS}`, { stdio: 'inherit' })
+
+ execSync(`./wasm-opt ${WASM_OPT_FLAGS} --enable-simd -o ${join(WASM_OUT, 'llhttp_simd.wasm')} ${join(WASM_OUT, 'llhttp_simd.wasm')}`, { stdio: 'inherit' })


codecov-commenter · 2024-04-09T00:43:14Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 94.06%. Comparing base (7a94682) to head (4a8f4a7).

Additional details and impacted files

@@ Coverage Diff @@ ## main #3074 +/- ## ======================================= Coverage 94.06% 94.06% ======================================= Files 89 89 Lines 24327 24331 +4 ======================================= + Hits 22883 22887 +4  Misses 1444 1444

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Uzlopak · 2024-04-09T00:46:45Z

@mcollina

Can you please this PR on your special benchmarking PC please :)?

mcollina · 2024-04-09T06:19:58Z

how does it shrink the wasm file?

Uzlopak

@mcollina

By using -04 we optimize the wasm further.

Uzlopak · 2024-04-13T12:47:08Z

build/wasm.js

 WASM_LDFLAGS += ' -Wl,--allow-undefined -Wl,--export-dynamic -Wl,--export-table'
 WASM_LDFLAGS += ' -Wl,--export=malloc -Wl,--export=free -Wl,--no-entry'

+const WASM_OPT_FLAGS = '-O4 --converge --strip-debug --strip-dwarf --strip-producers'


@mcollina

This is where we define O4 for optimizing the wasm file.

mcollina

lgtm

mochaaP · 2024-05-21T18:28:35Z

Hi @Uzlopak,

The external wasm codepath was used by Fedora due to our package policy. This is blocking us from updating to v6.14.0 and onwards.

If you think some code / files is redundant, please explain it further to me and I'll be glad to update my code!

Uzlopak · 2024-05-21T19:36:47Z

What do you mean with external wasm codepath?

mochaaP · 2024-05-22T06:09:42Z

build/wasm.js

- writeFileSync(join(WASM_OUT, 'wasm_build_env.txt'), buildInfo)
-}
-
-const writeWasmChunk = EXTERNAL_PATH
- ? (path, dest) => {
- const base64 = readFileSync(join(WASM_OUT, path)).toString('base64')
- writeFileSync(join(WASM_OUT, dest), `
-const { Buffer } = require('node:buffer')
-
-module.exports = Buffer.from('${base64}', 'base64')
-`)
- }
- : (path, dest) => {
- writeFileSync(join(WASM_OUT, dest), `
-const { fs } = require('node:fs')
-
-module.exports = fs.readFileSync(require.resolve('./${basename(path)}'))
-`)
- }
-
-// Build wasm binary
-execSync(`${WASM_CC} ${WASM_CFLAGS} ${WASM_LDFLAGS} \
- ${join(WASM_SRC, 'src')}/*.c \
- -I${join(WASM_SRC, 'include')} \
- -o ${join(WASM_OUT, 'llhttp.wasm')} \
- ${WASM_LDLIBS}`, { stdio: 'inherit' })
-
-writeWasmChunk('llhttp.wasm', 'llhttp-wasm.js')
-
-// Build wasm simd binary
-execSync(`${WASM_CC} ${WASM_CFLAGS} -msimd128 ${WASM_LDFLAGS} \
- ${join(WASM_SRC, 'src')}/*.c \
- -I${join(WASM_SRC, 'include')} \
- -o ${join(WASM_OUT, 'llhttp_simd.wasm')} \
- ${WASM_LDLIBS}`, { stdio: 'inherit' })
-
-writeWasmChunk('llhttp_simd.wasm', 'llhttp_simd-wasm.js')
-
-if (EXTERNAL_PATH) {
- writeFileSync(join(ROOT, 'loader.js'), `
-'use strict'
-
-globalThis.__UNDICI_IS_NODE__ = true
-module.exports = require('node:module').createRequire('${EXTERNAL_PATH}/loader.js')('./index-fetch.js')
-delete globalThis.__UNDICI_IS_NODE__
-`)


^ I meant this.

Ok, then it has to be added back + a big fat comment to not remove it because of fedora

Uzlopak added 2 commits April 9, 2024 02:33

improve: wasm build

1695a15

Merge branch 'main' into wasm-builder

211e1d5

github-advanced-security bot found potential problems Apr 9, 2024

View reviewed changes

Merge branch 'main' into wasm-builder

4a8f4a7

Uzlopak commented Apr 13, 2024

View reviewed changes

mcollina approved these changes Apr 22, 2024

View reviewed changes

mcollina requested a review from ronag April 22, 2024 13:10

mcollina merged commit 3f9e147 into nodejs:main Apr 22, 2024

github-actions bot mentioned this pull request Apr 22, 2024

[Release] v6.14.0 #3147

Merged

Uzlopak deleted the wasm-builder branch April 22, 2024 13:31

mochaaP reviewed May 22, 2024

View reviewed changes

renovate bot mentioned this pull request Jun 11, 2024

chore(deps): update all non-major dependencies unjs/node-fetch-native#123

Merged

1 task

mhdawson mentioned this pull request Jun 19, 2024

Ability to externalize WASM was broken #3345

Closed

richardlau mentioned this pull request Jul 26, 2024

fix: restore externalized Node.js dep compatibility #3421

Merged

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

improve: wasm build#3074

improve: wasm build#3074
mcollina merged 3 commits intonodejs:mainfrom
Uzlopak:wasm-builder

Uzlopak commented Apr 9, 2024

Check warning

Check warning

Check warning

Check warning

codecov-commenter commented Apr 9, 2024 •

edited

Loading

Uzlopak commented Apr 9, 2024

mcollina commented Apr 9, 2024

Uzlopak left a comment

Uzlopak Apr 13, 2024

mcollina left a comment

mochaaP commented May 21, 2024

Uzlopak commented May 21, 2024

mochaaP May 22, 2024

Uzlopak May 22, 2024

Labels

4 participants

Uh oh!

Conversation

Uzlopak commented Apr 9, 2024

Check warning

Check warning

Check warning

Check warning

codecov-commenter commented Apr 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uzlopak commented Apr 9, 2024

mcollina commented Apr 9, 2024

Uzlopak left a comment

Choose a reason for hiding this comment

Uzlopak Apr 13, 2024

Choose a reason for hiding this comment

mcollina left a comment

Choose a reason for hiding this comment

mochaaP commented May 21, 2024

Uzlopak commented May 21, 2024

mochaaP May 22, 2024

Choose a reason for hiding this comment

Uzlopak May 22, 2024

Choose a reason for hiding this comment

Labels

4 participants

codecov-commenter commented Apr 9, 2024 •

edited

Loading