Is there a way to map error messages to the source file for generated python scripts?
I have a python script that reads some input, generates a python script, and then runs the generated script. What I would like is that if there are errors in the generated script, they are reported at the file/lineno of the input file, not in the generated script.
If python supported a #line directive as in C or perl, I'd just put those in the generated script, and that would be all I needed. It doesn't look like that is supported, however.
I tried having my generated script sit in a try block, with the idea that I could catch exceptions and use the traceback module to tweak how the stack trace is printed. The problem is that the most common type of error is a parse error, and those seem to happen and get reported outside my control (i.e., the exception is thrown before the try, so my exception handler is never invoked).
The best bet I've been able to come up with is to use compile() on my generated script and then try to catch errors. That looks like it would force me to slurp in my entire generated script as a string (which could be large), though. Also, what I really want is to be able to run the generated script in its own sandbox, so it's not affected at all by the calling script (if not for error reporting, I would invoke the script as a new python process). If I use compile/exec, it's unclear to me that would be the case. Would passing an empty dictionary for the globals argument of exec() do the trick?
Is there a better way to do this sort of thing?
Edit:
The generated script could be almost anything. The input file allows the user to insert arbitrary python code in the generated script. What I'd like to do is generate a script like:
temp.py:
#line 10 original_file
for i in range(0,3)::
and have the syntax error be reported on line 10 of "original_file" and not line 2 of temp.py (which would mean nothing to the user).
The best I have so far is to insert #line directives in the generated script and have have the original script (i.e., the one that generated temp.py) execute the generated script as:
import runpy
try:
runpy.run_path(generated_file)
except:
print_stack_trace(traceback.format_exc(), generated_file)
The "print_stack_trace()" function parses the traceback and uses the #line directives to translate back to the original source.
This seems to work okay for simple cases, but I'm not sure of the implications of running the generated script (whose contents are arbitrary) in the same python that is running my generator script.
python
add a comment |
I have a python script that reads some input, generates a python script, and then runs the generated script. What I would like is that if there are errors in the generated script, they are reported at the file/lineno of the input file, not in the generated script.
If python supported a #line directive as in C or perl, I'd just put those in the generated script, and that would be all I needed. It doesn't look like that is supported, however.
I tried having my generated script sit in a try block, with the idea that I could catch exceptions and use the traceback module to tweak how the stack trace is printed. The problem is that the most common type of error is a parse error, and those seem to happen and get reported outside my control (i.e., the exception is thrown before the try, so my exception handler is never invoked).
The best bet I've been able to come up with is to use compile() on my generated script and then try to catch errors. That looks like it would force me to slurp in my entire generated script as a string (which could be large), though. Also, what I really want is to be able to run the generated script in its own sandbox, so it's not affected at all by the calling script (if not for error reporting, I would invoke the script as a new python process). If I use compile/exec, it's unclear to me that would be the case. Would passing an empty dictionary for the globals argument of exec() do the trick?
Is there a better way to do this sort of thing?
Edit:
The generated script could be almost anything. The input file allows the user to insert arbitrary python code in the generated script. What I'd like to do is generate a script like:
temp.py:
#line 10 original_file
for i in range(0,3)::
and have the syntax error be reported on line 10 of "original_file" and not line 2 of temp.py (which would mean nothing to the user).
The best I have so far is to insert #line directives in the generated script and have have the original script (i.e., the one that generated temp.py) execute the generated script as:
import runpy
try:
runpy.run_path(generated_file)
except:
print_stack_trace(traceback.format_exc(), generated_file)
The "print_stack_trace()" function parses the traceback and uses the #line directives to translate back to the original source.
This seems to work okay for simple cases, but I'm not sure of the implications of running the generated script (whose contents are arbitrary) in the same python that is running my generator script.
python
Can you give us some code? I think seeing an example of a typical generated script would help, but seeing your main program's code (especially the part around which you see the error) would also be helpful.
– Green Cloak Guy
Nov 28 '18 at 21:05
1
Use theinspect
package?
– s3cur3
Nov 28 '18 at 21:27
I see that the inspect package may help me do something more powerful regarding printing information once I've found an error. The difficulty I've had is finding the right place to catch the error in the first place. I haven't found a way to make the generated script catch its own syntax errors. Runtime errors are easy, but I don't know how to use a try block to catch parse-time errors, unless the generated script is executed in the context of the calling script, which I'm trying to avoid (or at least find a way to do safely).
– user1806566
Nov 29 '18 at 16:28
add a comment |
I have a python script that reads some input, generates a python script, and then runs the generated script. What I would like is that if there are errors in the generated script, they are reported at the file/lineno of the input file, not in the generated script.
If python supported a #line directive as in C or perl, I'd just put those in the generated script, and that would be all I needed. It doesn't look like that is supported, however.
I tried having my generated script sit in a try block, with the idea that I could catch exceptions and use the traceback module to tweak how the stack trace is printed. The problem is that the most common type of error is a parse error, and those seem to happen and get reported outside my control (i.e., the exception is thrown before the try, so my exception handler is never invoked).
The best bet I've been able to come up with is to use compile() on my generated script and then try to catch errors. That looks like it would force me to slurp in my entire generated script as a string (which could be large), though. Also, what I really want is to be able to run the generated script in its own sandbox, so it's not affected at all by the calling script (if not for error reporting, I would invoke the script as a new python process). If I use compile/exec, it's unclear to me that would be the case. Would passing an empty dictionary for the globals argument of exec() do the trick?
Is there a better way to do this sort of thing?
Edit:
The generated script could be almost anything. The input file allows the user to insert arbitrary python code in the generated script. What I'd like to do is generate a script like:
temp.py:
#line 10 original_file
for i in range(0,3)::
and have the syntax error be reported on line 10 of "original_file" and not line 2 of temp.py (which would mean nothing to the user).
The best I have so far is to insert #line directives in the generated script and have have the original script (i.e., the one that generated temp.py) execute the generated script as:
import runpy
try:
runpy.run_path(generated_file)
except:
print_stack_trace(traceback.format_exc(), generated_file)
The "print_stack_trace()" function parses the traceback and uses the #line directives to translate back to the original source.
This seems to work okay for simple cases, but I'm not sure of the implications of running the generated script (whose contents are arbitrary) in the same python that is running my generator script.
python
I have a python script that reads some input, generates a python script, and then runs the generated script. What I would like is that if there are errors in the generated script, they are reported at the file/lineno of the input file, not in the generated script.
If python supported a #line directive as in C or perl, I'd just put those in the generated script, and that would be all I needed. It doesn't look like that is supported, however.
I tried having my generated script sit in a try block, with the idea that I could catch exceptions and use the traceback module to tweak how the stack trace is printed. The problem is that the most common type of error is a parse error, and those seem to happen and get reported outside my control (i.e., the exception is thrown before the try, so my exception handler is never invoked).
The best bet I've been able to come up with is to use compile() on my generated script and then try to catch errors. That looks like it would force me to slurp in my entire generated script as a string (which could be large), though. Also, what I really want is to be able to run the generated script in its own sandbox, so it's not affected at all by the calling script (if not for error reporting, I would invoke the script as a new python process). If I use compile/exec, it's unclear to me that would be the case. Would passing an empty dictionary for the globals argument of exec() do the trick?
Is there a better way to do this sort of thing?
Edit:
The generated script could be almost anything. The input file allows the user to insert arbitrary python code in the generated script. What I'd like to do is generate a script like:
temp.py:
#line 10 original_file
for i in range(0,3)::
and have the syntax error be reported on line 10 of "original_file" and not line 2 of temp.py (which would mean nothing to the user).
The best I have so far is to insert #line directives in the generated script and have have the original script (i.e., the one that generated temp.py) execute the generated script as:
import runpy
try:
runpy.run_path(generated_file)
except:
print_stack_trace(traceback.format_exc(), generated_file)
The "print_stack_trace()" function parses the traceback and uses the #line directives to translate back to the original source.
This seems to work okay for simple cases, but I'm not sure of the implications of running the generated script (whose contents are arbitrary) in the same python that is running my generator script.
python
python
edited Nov 29 '18 at 16:23
user1806566
asked Nov 28 '18 at 21:01
user1806566user1806566
203210
203210
Can you give us some code? I think seeing an example of a typical generated script would help, but seeing your main program's code (especially the part around which you see the error) would also be helpful.
– Green Cloak Guy
Nov 28 '18 at 21:05
1
Use theinspect
package?
– s3cur3
Nov 28 '18 at 21:27
I see that the inspect package may help me do something more powerful regarding printing information once I've found an error. The difficulty I've had is finding the right place to catch the error in the first place. I haven't found a way to make the generated script catch its own syntax errors. Runtime errors are easy, but I don't know how to use a try block to catch parse-time errors, unless the generated script is executed in the context of the calling script, which I'm trying to avoid (or at least find a way to do safely).
– user1806566
Nov 29 '18 at 16:28
add a comment |
Can you give us some code? I think seeing an example of a typical generated script would help, but seeing your main program's code (especially the part around which you see the error) would also be helpful.
– Green Cloak Guy
Nov 28 '18 at 21:05
1
Use theinspect
package?
– s3cur3
Nov 28 '18 at 21:27
I see that the inspect package may help me do something more powerful regarding printing information once I've found an error. The difficulty I've had is finding the right place to catch the error in the first place. I haven't found a way to make the generated script catch its own syntax errors. Runtime errors are easy, but I don't know how to use a try block to catch parse-time errors, unless the generated script is executed in the context of the calling script, which I'm trying to avoid (or at least find a way to do safely).
– user1806566
Nov 29 '18 at 16:28
Can you give us some code? I think seeing an example of a typical generated script would help, but seeing your main program's code (especially the part around which you see the error) would also be helpful.
– Green Cloak Guy
Nov 28 '18 at 21:05
Can you give us some code? I think seeing an example of a typical generated script would help, but seeing your main program's code (especially the part around which you see the error) would also be helpful.
– Green Cloak Guy
Nov 28 '18 at 21:05
1
1
Use the
inspect
package?– s3cur3
Nov 28 '18 at 21:27
Use the
inspect
package?– s3cur3
Nov 28 '18 at 21:27
I see that the inspect package may help me do something more powerful regarding printing information once I've found an error. The difficulty I've had is finding the right place to catch the error in the first place. I haven't found a way to make the generated script catch its own syntax errors. Runtime errors are easy, but I don't know how to use a try block to catch parse-time errors, unless the generated script is executed in the context of the calling script, which I'm trying to avoid (or at least find a way to do safely).
– user1806566
Nov 29 '18 at 16:28
I see that the inspect package may help me do something more powerful regarding printing information once I've found an error. The difficulty I've had is finding the right place to catch the error in the first place. I haven't found a way to make the generated script catch its own syntax errors. Runtime errors are easy, but I don't know how to use a try block to catch parse-time errors, unless the generated script is executed in the context of the calling script, which I'm trying to avoid (or at least find a way to do safely).
– user1806566
Nov 29 '18 at 16:28
add a comment |
0
active
oldest
votes
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53528035%2fis-there-a-way-to-map-error-messages-to-the-source-file-for-generated-python-scr%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53528035%2fis-there-a-way-to-map-error-messages-to-the-source-file-for-generated-python-scr%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Can you give us some code? I think seeing an example of a typical generated script would help, but seeing your main program's code (especially the part around which you see the error) would also be helpful.
– Green Cloak Guy
Nov 28 '18 at 21:05
1
Use the
inspect
package?– s3cur3
Nov 28 '18 at 21:27
I see that the inspect package may help me do something more powerful regarding printing information once I've found an error. The difficulty I've had is finding the right place to catch the error in the first place. I haven't found a way to make the generated script catch its own syntax errors. Runtime errors are easy, but I don't know how to use a try block to catch parse-time errors, unless the generated script is executed in the context of the calling script, which I'm trying to avoid (or at least find a way to do safely).
– user1806566
Nov 29 '18 at 16:28