0

In Python 3.6.5, this works fine:

command = "ffmpeg -i {0} -vsync 0 -q:v 2 -vf select=\"eq(pict_type\,PICT_TYPE_I)\" -r 30 {1}/frame%03d.jpg".format(file_path, output_path)

That's obviously a long line, so I used a line continuation:

command = "ffmpeg -i {0} -vsync 0 -q:v 2 -vf select=\"eq(pict_type\,PICT_TYPE_I)\" -r 30 {1}/frame%03d.jpg"\
    .format(file_path, output_path)

However, at startup, this generates a DeprecationWarning:

 DeprecationWarning: invalid escape sequence \,
  command = "ffmpeg -i {0} -vsync 0 -q:v 2 -vf select=\"eq(pict_type\,PICT_TYPE_I)\" -r 30 {1}/frame%03d.jpg"\

This does not, however:

command = "foo {0} bar {1}"\
    .format(file_path, output_path)

I use line continuations all over the rest of the project; none have resulted in DeprecationWarning. Other questions like this one mention this warning, but none for continuation characters that I can find.

What causes this warning, and why does it only appear in this very narrow case?

Edit: This has nothing to do with the line continuation. The reason the error presented to me only some of the time has to do with Django's runserver. The first time runserver is run, the error is not reported. But if a change causes a reload, then the error is reported when the reloader runs.

Athena
  • 3,200
  • 3
  • 27
  • 35

2 Answers2

5

This has nothing to do with your line continuation, it's with using \, as an escape sequence in your string.

The warning explicitly includes \,:

DeprecationWarning: invalid escape sequence \,

And that's why your later example doesn't warn: because the string doesn't have a \, or other unrecognized escape sequence in it.

As explained in the docs for String and Byte literals:

Changed in version 3.6: Unrecognized escape sequences produce a DeprecationWarning. In some future version of Python they will be a SyntaxError.

This change is mentioned in What's New in Python 3.6, with a link to issue #27364, which links to an earlier discussion on the -dev mailing list.

Traditionally, Python allowed unrecognized escape sequences in string literals and just processed them as if they weren't escapes, so \, literally means a backslash and a comma, because this makes it easier to see what's going wrong when you print out the string in the debugger.

But this leads to all kinds of confusion, especially for Windows users (who get away with 'C:\Spam', 'C:\spam', and 'C:\Vikings', but get errors with 'C:\vikings'), and for people coming from any of the huge number of languages that follow C-style escape rules (where \, would mean just a comma—although most C compilers will generate a warning for that), which is presumably why you now get a warning.

abarnert
  • 354,177
  • 51
  • 601
  • 671
  • That explains it; I figured it was something like that. But why does the presence of a line continuation character impact whether that's outputted or not? – Athena Jul 25 '18 at 20:13
  • @Ares I don't think it does. I can't reproduce that behavior; any situation where I get a warning for the second one, I get a warning if I change it to the first one. Can you provide a reproducible test case that shows a difference? If so, it's probably a bug that should be reported. – abarnert Jul 25 '18 at 20:18
  • @Ares: It doesn't matter whether the line continuation character is there or not (I reproduce either way as long as `DeprecationWarning`s are enabled). If you're launching or using the code in different ways, you might have different warning levels set though (by default, `DeprecationWarning`s aren't emitted, but one of your tests might be enabling it). – ShadowRanger Jul 25 '18 at 20:18
  • 1
    @abarnert et al; I figured it out; the issue has to do with Django's reloader behavior. The first time `runserver` executes, no warning appears, but when the reloader runs, the warning appears. I incorrectly attributed the behavior to the changes made in my code rather than Django's reloader. – Athena Jul 25 '18 at 20:22
  • 2
    @Ares: They likely set the handling behavior for `DeprecationWarning` to either `module` (prints when first raised in a given module, then silent thereafter) or `once` (prints each class of warning the first time it occurs program-wide, then never again). Either way would get the behavior you describe. – ShadowRanger Jul 25 '18 at 20:28
2

try using lists for commands:

command = [
    "ffmpeg",
    "-i", file_path,
    "-vsync", "0", 
    "-q:v", "2",
    "-vf", 'select="eq(pict_type,PICT_TYPE_I)"',
    "-r", "30", 
    os.path.join(output_path, "frame%03d.jpg"),
]

Then call it without using shell=True:

subprocess.run(command)

By doing this, you have multiple advantages:

  1. You are free from quote and escape hell - you don't have to add quotes or escape anything. Spaces are no longer delimiters, the parameters will be passed as in the list.
  2. You don't have to use string interpolation (.format) because you can just pass the parameter separately to the list.
  3. By not using the shell, you avoid executing one extra process for nothing - why execute a shell to run your command when you can run it directly?
nosklo
  • 217,122
  • 57
  • 293
  • 297
  • 1
    While this is 100% true, it doesn't explain the cause of the problem, and it's not necessarily the correct answer (e.g. if `subprocess` isn't being used directly, it's possible they're either printing this for some reason, or passing it to some third party API). – ShadowRanger Jul 25 '18 at 20:21
  • @ShadowRanger Agreed, but I think this answer will really help the OP, most people use commands with `subprocess` and (argh) `os.system`. – nosklo Jul 25 '18 at 20:24
  • @nosklo I suggested `os.system` should be moved to `subprocess.system` in 3.0. Guido's reply was something like: Nice idea, but nothing humanly possible will stop people from finding that function, calling it, and then asking why they can't capture the output. It's a law of nature. – abarnert Jul 25 '18 at 20:37