Adopting the black Python code style

Hello everyone,

If you don't write Python code in mozilla-central, you can stop reading now..

On October 19, 2020 we will be officially adopting the black Python style for all our Python code in mozilla-central.

black (https://black.readthedocs.io/en/stable/) is an opinionated, fast, and correct auto-formatter for Python. It is an increasingly popular autoformatter which might be considered the de facto standard for Python code (like clang-format and jslint are for C++ and JS). It is already used by several Mozilla projects, including Release Engineering, Lando, and moz-phab.

black makes it easy for us to reliably format all our Python code in a consistent way, making the codebase easier to read on the whole and allowing us to spend more time in code review discussing substantive issues over trivial formatting matters.

This policy change will affect all Python code in-tree, including sandboxed Python code used by the build system (.configure, .build, and .mozbuild files).

As part of this policy change, we plan on doing a one-time auto-reformat on October 19 of all Python code in the entire repository. In addition, mach lint (https://firefox-source-docs.mozilla.org/code-quality/lint/linters/black.html) and reviewbot will be updated to print warnings for Python source files that violate the black style. Just like with C/C++ or Rust, we wont backout offending changes but instead will do regular refreshes of the tree.

If there are any questions, please let me know!

Ricky

Is there some way we can see the results of this change before it lands in the tree? I would like to review the results of formatting for Python files

I care about to make sure nothing gets too messed up by it. For instance, clang-format did some kind of line wrapping for wide comments that broke a lot of ASCII diagram comments, and I and others had to spend a fair amount of time fixing that.

Relatedly, is there some way to disable formatting for specific lines of code, in case there's something that gets mangled by the formatting and there's no easy way to fix it?

What does "all Python code in-tree" mean? Does that mean that this is going to be run on all files that end in .py? I assume that this doesn't apply to any third party code we have in the tree (either under the third-party directory, or included in whatever other list I think we have)?

Thanks,
Andrew

black does not reformat comments: https://github.com/psf/black/issues/181

Third-party code is exempt from being auto-formatted.

 Thanks for driving this Ricky! I'm *very* excited for it.

Just want to call out that you can use:
$ ./mach lint -wo --fix

to reformat all the files you have touched (either in the working directory
or outgoing commits). For more usage docs, see:
https://firefox-source-docs.mozilla.org/code-quality/lint/usage.html

-Andrew

Is black still opiniated about string types and insisting to use double quotes, when we mostly settled on single quotes?

No, black now has a `--skip-string-normalization` flag, which I would be all for using in our code base. Not sure if that was the plan here or not.

-Andrew

p.s It took a great deal of convincing from the community to get this flag added, as the maintainers precisely wanted to prevent conversations like this current one from happening :p.

--skip-string-normalization does exist, but the plan is not to use it. While there is a cost associated with reformatting many (but not all) of the strings in our Python code to be consistent with the black style, we don't think that that cost outweighs the benefit of doing so.

I'm disappointed by that.

FWIW last time I looked at black, I found that the compromises it made to be fully automatic and with minimal configuration meant that it was liable to produce ugly or difficult to read code in some situations.

I understand that we've decided that people will get used to reading any code style over time, and therefore eliminating formatting concerns from the code writing process is a net win for productivity. So I'm not taking a position in opposition to this proposal, but it is not something I would have advocated personally.

Well we generally don't seek consensus anymore for these sorts of things, it seems, but it's reassuring that I'm not alone.