Skip to main content
Emacs

Return to Question

Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
added 215 characters in body
Source Link
fap
fap
  • 161
  • 5

Is there any way to make the output the same on every run again? SoSo far I've only found the backtrace-on-error-noninteractive variable to deactivate the backtrace, which is not what we want because we don't set that flag in production, making the test less useful.

I've also tried to set the variable debugger, but that doesn't seem to have an effect. (I later saw that it's already set to debug in batch mode, so relevant code doesn't seem to care about that variable.)

Is there any way to make the output the same on every run again?

Is there any way to make the output the same on every run again? So far I've only found the backtrace-on-error-noninteractive variable to deactivate the backtrace, which is not what we want because we don't set that flag in production, making the test less useful.

So far I've only found the backtrace-on-error-noninteractive variable to deactivate the backtrace, which is not what we want because we don't set that flag in production, making the test less useful.

I've also tried to set the variable debugger, but that doesn't seem to have an effect. (I later saw that it's already set to debug in batch mode, so relevant code doesn't seem to care about that variable.)

Is there any way to make the output the same on every run again?

Source Link
fap
fap
  • 161
  • 5

Emacs 29 ert batch run: Error output changes between runs because of debug-early

We have a job running in CI that verifies the output of an ert batch run. The job also includes examples of failing tests which produce a backtrace. The job tests if the error message output matches our expectation.

This has worked well with Emacs 28.2 with a simple call to diff, checking if the error message is exactly the same. But after upgrading to Emacs 29.3 the error message changes between runs.

Emacs 28.2:

Loading /solution/example-syntax-error.el (source)... Debugger entered--Lisp error: (void-variable DEF&*@) eval-buffer(#<buffer > nil "/solution/exampl..." nil t) ; Reading at buffer position 41 load-with-code-conversion("/solution/exampl..." "/solution/exampl..." nil nil) load("/solution/exampl..." nil nil t) load-file("example-syntax-error.el") eval-buffer(#<buffer > nil "/solution/exampl..." nil t) ; Reading at buffer position 151 load-with-code-conversion("/solution/exampl..." "/solution/exampl..." nil t) load("/solution/exampl..." nil t) command-line-1(("-l" "ert" "-l" "/solution/exampl..." "-f" "ert-run-tests-batch-and-exit")) command-line() normal-top-level()

Emacs 29.3:

Loading /solution/example-syntax-error.el (source)... Error: void-variable (DEF&*@) mapbacktrace(#f(compiled-function (evald func args flags) #<bytecode -0x77b476f04510f1b>)) <--- the bytecode output changes debug-early-backtrace() debug-early(error (void-variable DEF&*@)) eval-buffer(#<buffer > nil "/solution/example-syntax-error.el" nil t) load-with-code-conversion("/solution/example-syntax-error.el" "/solution/example-syntax-error.el" nil nil) load("/solution/example-syntax-error.el" nil nil t) load-file("example-syntax-error.el") eval-buffer(#<buffer > nil "/solution/example-syntax-error-test.el" nil t) load-with-code-conversion("/solution/example-syntax-error-test.el" "/solution/example-syntax-error-test.el" nil t) load("/solution/example-syntax-error-test.el" nil t) command-line-1(("-l" "ert" "-l" "/solution/example-syntax-error-test.el" "-f" "ert-run-tests-batch-and-exit")) command-line() normal-top-level() Symbol's value as variable is void: DEF&*@

I've traced back the problem to the introduction of debug-early.el, which is now always used in batch mode. This lambda changes the output every run of the #<bytecode -0x77b476f04510f1b> part, line 3 of the quote above.

Is there any way to make the output the same on every run again? So far I've only found the backtrace-on-error-noninteractive variable to deactivate the backtrace, which is not what we want because we don't set that flag in production, making the test less useful.