3

I'm making a couple of functions via exec, that could possibly error. However when Python errors out it doesn't display the line that the error occurred on.

For example using:

fn_str = '''\ def fn(): raise Exception() ''' globs = {} exec(fn_str, globs) fn = globs['fn'] fn()

Gives us the output:

Traceback (most recent call last): File "...", line 10, in <module> fn() File "<string>", line 2, in fn Exception

If we however, don't use eval. Then we get the line that the program errored on:

def fn(): raise Exception() fn() 
Traceback (most recent call last): File "...", line 4, in <module> fn() File "...", line 2, in fn raise Exception() Exception

I've looked into using the __traceback__, however I couldn't find a way to add to the traceback under the 'File' line. And so the best I could get was this:

fn_str = '''\ def fn(): try: raise Exception() except BaseException as e: tb = e.__traceback__ if 1 <= tb.tb_lineno <= len(fn_lines): e.args = ((e.args[0] if e.args else '') + ' - ' + fn_lines[tb.tb_lineno - 1].strip(),) raise ''' globs = {'fn_lines': fn_str.split('\n')} exec(fn_str, globs) fn = globs['fn'] fn() 
Traceback (most recent call last): File "...", line 16, in <module> fn() File "<string>", line 3, in fn Exception: - raise Exception()

The biggest problem with this is if the eval calls other code, it becomes confusing where the - raise Exception() comes from.

Is there a way to make the eval code provide the line that it errored on?

Improve this question
0

3 Answers 3

Reset to default
3

The missing lines are a symptom of the fact that Python cannot find a file on disk named <string>, which is the filename built into your compiled snippet of code. (Though if you create a file with exactly that name, Python will print lines from it!)

Approach 1. You can catch exceptions yourself, whether at the top level of your application or elsewhere, and instead of letting the default builtin traceback routine fire you can call the Standard Library routine traceback.print_exc() which pulls lines from the Standard Library module linecache. Because the linecache cache is a simple public Python dictionary, you can pre-populate it with the source code it needs to print. See:

Why does the Python linecache affect the traceback module but not regular tracebacks?

The resulting code:

import linecache import traceback source = 'print("Hello, world" + 1)' source_name = 'Example' lines = source.splitlines(True) linecache.cache[source_name] = len(source), None, lines, source_name compiled = compile(source, source_name, 'exec') try: eval(compiled) except Exception: traceback.print_exc()

Approach 2. You can also avoid the indirection of populating a global cache by simply taking charge of printing the exception yourself: you can have Python return the traceback data as a list of tuples, step through them adding the missing lines, and then finally print them as usual.

Here is a fill_in_lines() function that fills out the traceback with the missing information, in a small program that prints the full traceback:

import sys import traceback def fill_in_lines(frames, source_name, source): lines = source.splitlines() for filename, line_number, function_name, text in frames: if filename == source_name: text = lines[line_number - 1] yield filename, line_number, function_name, text source = 'print("Hello, world" + 1)' source_name = 'Example' compiled = compile(source, source_name, 'exec') try: eval(compiled) except Exception as e: _, _, tb = sys.exc_info() frames = traceback.extract_tb(tb) frames = fill_in_lines(frames, source_name, source) print('Traceback (most recent call last):') print(''.join(traceback.format_list(frames)), end='') print('{}: {}'.format(type(e).__name__, str(e)))

I am able to use the fancy name “Example” here because I set it using compile(). In your case you would instead want to pass the bare string '<string>' as the source_name argument.

Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

Interesting. I've verified that this works but I'm off to bed. Thank you for the answer, I'll look into it more tomorrow
I'm glad it worked! Upon waking this morning I thought, "so why didn't it work when I pre-populated the line cache," and did a quick search, and found the Stack Overflow answer that already tackles the subject. No idea why I didn't find it yesterday! I have updated my answer to include a link to that approach too.
0

Here is a helper function that puts all of these ideas together, to automatically write a file and rewrite the backtrace to point to it when you have an exception that has a frame:

import traceback import sys from types import TracebackType import tempfile import contextlib import inspect @contextlib.contextmanager def report_compile_source_on_error(): try: yield except Exception as exc: tb = exc.__traceback__ # Walk the traceback, looking for frames that have # source attached stack = [] while tb is not None: filename = tb.tb_frame.f_code.co_filename source = tb.tb_frame.f_globals.get("__compile_source__") if filename == "<string>" and source is not None: # Don't delete the temporary file so the user can expect it with tempfile.NamedTemporaryFile(mode='w', delete=False) as f: f.write(source) # Create a frame. Python doesn't let you construct # FrameType directly, so just make one with compile frame = tb.tb_frame code = compile('__inspect_currentframe()', f.name, 'eval') # Python 3.8 only. In earlier versions of Python # just have less accurate name info if hasattr(code, 'replace'): code = code.replace(co_name=frame.f_code.co_name) fake_frame = eval( code, frame.f_globals, { **frame.f_locals, '__inspect_currentframe': inspect.currentframe } ) fake_tb = TracebackType( None, fake_frame, tb.tb_lasti, tb.tb_lineno ) stack.append(fake_tb) else: stack.append(tb) tb = tb.tb_next # Reconstruct the linked list tb_next = None for tb in reversed(stack): tb.tb_next = tb_next tb_next = tb raise exc.with_traceback(tb_next)

Full file with a test at https://gist.github.com/ezyang/ed041c0302d4c2a63cc51be5b10660da

Improve this answer

Comments

-2

This is a repost of vaultah's deleted answer.

That happens when the interpreter is unable to find the source code for that line for whatever reason. This is the case for built-in modules, compiled files, exec strings, etc. More specifically, in the traceback you can see that the filename for the fn's code object is set to <string>

File "<string>", line 2, in fn

Because <string> is not a valid filename, the reference to the source code is lost.

One option is to create a temporary file, write fn_str in there, compile fn_str to set the filename, execute the compiled code, and finally call the function. Note that you'll need to keep the file alive at least until the source lines are cached by the traceback-printing facility

from tempfile import NamedTemporaryFile import traceback with NamedTemporaryFile('w') as temp: code = compile(fn_str, temp.name, 'exec') print(fn_str, file=temp, flush=True) globs = {} exec(code, globs) fn = globs['fn'] try: fn() except: traceback.print_exc()

prints

Traceback (most recent call last): File "test.py", line 16, in <module> fn() File "/tmp/tmp9q2bogm6", line 2, in fn raise Exception() Exception

Since we already create a "real" file, we can delegate the compilation and execution of code to runpy.run_path:

from tempfile import NamedTemporaryFile import runpy, traceback with NamedTemporaryFile('w') as temp: print(fn_str, file=temp, flush=True) fn = runpy.run_path(temp.name)['fn'] try: fn() except: traceback.print_exc()
Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.