How to push exceptions up the stack?

Question

Possible Duplicate:
How can I modify a Python traceback object when raising an exception?

Consider this toy example:

def twice(n):
    _validate_twice_args(n)
    return 2*n

def _validate_twice_args(n):
    if type(n) != int:
        raise TypeError('n must be an int')

twice(None)
--
Traceback (most recent call last):
  File "demo.py", line 9, in <module>
    twice(None)
  File "demo.py", line 2, in twice
    _validate_twice_args(n)
  File "demo.py", line 7, in _validate_twice_args
    raise TypeError('n must be an int')
TypeError: n must be an int

Even though the locus of the error is the call twice(None), the traceback refers to code that the person responsible for the error is not even aware of, ("_validate who? I never called that! I don't even know what it is!"), and unnecessarily exposes code that should be "behind the API". Even in the absence of "private" helper functions like _validate_twice_args, the stacktrace printed in response to bad arguments unnecessarily exposes internal code, and obscures the locus of the error.

For example, if one inlines the code of _validate_twice_args, the stack trace looks like this:

Traceback (most recent call last):
  File "demo.py", line 10, in <module>
    twice(None)
  File "demo.py", line 3, in twice
    raise TypeError('n must be an int')
TypeError: n must be an int

To get an idea of what the stack trace for this sort of error should look like, here's the one resulting from a very similar type of error, the call twice() instead of twice(None), but one that is raised by Python, before control is passed to twice:

Traceback (most recent call last):
  File "demo.py", line 9, in <module>
    twice()
TypeError: twice() takes exactly 1 argument (0 given)

In this case too, the error lies in a call to twice with an invalid arguments list. Accordingly the stack trace points directly to the locus of the error (and not to the line in the underlying [most likely C] code where the error was first detected, thank goodness). This is as it should be, IMO¹.

How can I modify twice_args and/or _validate_twice_args so that the last line of the stacktrace refers to the source code line of the buggy call to twice?

---

¹I realize that opinions on what the stacktrace should say in such cases differ from mine, but these philosophical considerations are not the subject of this thread.

Edit: emphasized remarks in the original post, but apparently overlooked by the first two responders.

Answer 1

Well, you can change twice to this:

def twice(n):
    try:
        _validate_twice_args(n)
    except Exception, e:
        raise e
    return 2*n

But of course, if you're going to put a try/except block in twice, you might as well do the actual check in there. Which is, of course, what you really should do if you want the exception raised there, despite your footnote. An exception happens where it happens. You can never call a function and expect that any exception will occur in the actual body of that particular function. Functions call other functions all the time. It's fine to have validator functions that do stuff like that, but if so, you just have to accept that the exceptions are going to be raised there and not somewhere else.

In response to your edit: It may seem like you are "exposing private code" when you raise the exception where it occurs, but doing it the other way hides bugs, which is way, way worse. The traceback easily allows you to see up the stack and look for the place where you called the function. Doing what you describe, however, would chop off the traceback too high, making it impossible to see where the error actually happened. If code could raise an exception somewhere else, buggy code could hide itself very easily.

More generally, what you describe as "the locus of the error" is a red herring. An exception (which may or may not be an "error") should be raised at the point where the exceptional condition is detected. "Where the error is" is a more complex question that is not something you should try to paper over by creating "spaghetti exceptions" that raise themselves somewhere else. You as a human can see that twice(None) is the "place where a mistake was made" in this particular call chain, but that doesn't mean that you should try to make your program clever and have it figure that out. It is likely to go wrong at some point and cause greater confusion.

So to answer what you have emphasized: no, you can't do that. An exception is raised where it is raised. You can't raise an exception in one place and have it show up as if it was raised somewhere else.

Answer 2

Long answer short: without writing a C extension or using ctypes magic, you cannot manipulate a normal Python traceback to pretend that an error came from anywhere but a raise statement in your code.

Exceptions raised from Python will have a raise statement at the very top of their traceback, since that is by definition where the exception originated -- you can't change that (without any of the aforementioned trickery). In your case, you want the exception to look like it came from a function call (as the topmost frame), and I am saying it is impossible from pure Python.

If you still want to see a ctypes or CPython way to do this, I could show you. But you will not get what you want (and you evidently want what you want) by sticking to pure Python code.

EDIT: OK, this isn't 100% accurate. Of course you can always call some extension method that throws an exception -- the exception "comes from" the call to the extension method. Consequently, if you wrote twice as an extension method you could implement the exact semantics you wanted (this is what I mean by "writing a C extension").

Owing to the existence of generator.throw(), you can also cause exceptions to appear to come out of the def line in a generator function, or a yield statement, all from Python code. Neither, of course, will help you implement the semantics you want, but I wanted to mention these for completeness.

Answer 3

See How can I modify a Python traceback object when raising an exception?

There you'll find more advice not to do this, and also a link to some ugly code in jinja2 that shows you how to do it if you really must.

Answer 4

You could modify your code to something like this:

def twice(n):
    error = _validate_twice_args(n)
    if error:
        raise error
    return 2*n

def _validate_twice_args(n):
    if type(n) != int:
        return TypeError('n must be an int')
    return None

twice(None)

This way, the error is raised from twice. Unfortunately, it requires a bit more code in the twice method than a simple method call.

Traceback (most recent call last):
  File "<module1>", line 28, in <module>
  File "<module1>", line 25, in main
  File "<module1>", line 16, in twice
TypeError: n must be an int