Adopting the black Python code style

Hello everyone,

If you don't write Python code in mozilla-central, you can stop reading now=
..

On October 19, 2020 we will be officially adopting the black Python style f=
or all our Python code in mozilla-central.

black (https://black.readthedocs.io/en/stable/) is an opinionated, fast, an=
d correct auto-formatter for Python. It is an increasingly popular autoform=
atter which might be considered the de facto standard for Python code (like=
 clang-format and jslint are for C++ and JS). It is already used by several=
 Mozilla projects, including Release Engineering, Lando, and moz-phab.

black makes it easy for us to reliably format all our Python code in a cons=
istent way, making the codebase easier to read on the whole and allowing us=
 to spend more time in code review discussing substantive issues over trivi=
al formatting matters.

This policy change will affect all Python code in-tree, including sandboxed=
 Python code used by the build system (.configure, .build, and .mozbuild fi=
les).

As part of this policy change, we plan on doing a one-time auto-reformat on=
 October 19 of all Python code in the entire repository. In addition, mach =
lint (https://firefox-source-docs.mozilla.org/code-quality/lint/linters/bla=
ck.html) and reviewbot will be updated to print warnings for Python source =
files that violate the black style. Just like with C/C++ or Rust, we won=E2=
=80=99t backout offending changes but instead will do regular refreshes of =
the tree.

If there are any questions, please let me know!

Ricky
0
Ricky
10/12/2020 5:00:56 PM
mozilla.dev.platform 6644 articles. 0 followers. Post Follow

9 Replies
14 Views

Similar Articles

[PageSpeed] 2

On Mon, Oct 12, 2020 at 10:05 AM Ricky Stewart <rstewart@mozilla.com> wrote=
:

> Hello everyone,
>
> If you don't write Python code in mozilla-central, you can stop reading
> now.
>
> On October 19, 2020 we will be officially adopting the black Python style
> for all our Python code in mozilla-central.
>
> black (https://black.readthedocs.io/en/stable/) is an opinionated, fast,
> and correct auto-formatter for Python. It is an increasingly popular
> autoformatter which might be considered the de facto standard for Python
> code (like clang-format and jslint are for C++ and JS). It is already use=
d
> by several Mozilla projects, including Release Engineering, Lando, and
> moz-phab.
>
> black makes it easy for us to reliably format all our Python code in a
> consistent way, making the codebase easier to read on the whole and
> allowing us to spend more time in code review discussing substantive issu=
es
> over trivial formatting matters.
>
> This policy change will affect all Python code in-tree, including
> sandboxed Python code used by the build system (.configure, .build, and
> .mozbuild files).
>
> As part of this policy change, we plan on doing a one-time auto-reformat
> on October 19 of all Python code in the entire repository. In addition,
> mach lint (
> https://firefox-source-docs.mozilla.org/code-quality/lint/linters/black.h=
tml)
> and reviewbot will be updated to print warnings for Python source files
> that violate the black style. Just like with C/C++ or Rust, we won=E2=80=
=99t
> backout offending changes but instead will do regular refreshes of the tr=
ee.
>
> If there are any questions, please let me know!
>

Is there some way we can see the results of this change before it lands in
the tree? I would like to review the results of formatting for Python files
I care about to make sure nothing gets too messed up by it. For instance,
clang-format did some kind of line wrapping for wide comments that broke a
lot of ASCII diagram comments, and I and others had to spend a fair amount
of time fixing that.

Relatedly, is there some way to disable formatting for specific lines of
code, in case there's something that gets mangled by the formatting and
there's no easy way to fix it?

What does "all Python code in-tree" mean? Does that mean that this is going
to be run on all files that end in .py? I assume that this doesn't apply to
any third party code we have in the tree (either under the third-party
directory, or included in whatever other list I think we have)?

Thanks,
Andrew


> Ricky
> _______________________________________________
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>
0
Andrew
10/12/2020 5:22:01 PM
On Monday, October 12, 2020 at 12:22:22 PM UTC-5, Andrew McCreight wrote:
> On Mon, Oct 12, 2020 at 10:05 AM Ricky Stewart <rste...@mozilla.com> wrot=
e:=20
>=20
> > Hello everyone,=20
> >=20
> > If you don't write Python code in mozilla-central, you can stop reading=
=20
> > now.=20
> >=20
> > On October 19, 2020 we will be officially adopting the black Python sty=
le=20
> > for all our Python code in mozilla-central.=20
> >=20
> > black (https://black.readthedocs.io/en/stable/) is an opinionated, fast=
,=20
> > and correct auto-formatter for Python. It is an increasingly popular=20
> > autoformatter which might be considered the de facto standard for Pytho=
n=20
> > code (like clang-format and jslint are for C++ and JS). It is already u=
sed=20
> > by several Mozilla projects, including Release Engineering, Lando, and=
=20
> > moz-phab.=20
> >=20
> > black makes it easy for us to reliably format all our Python code in a=
=20
> > consistent way, making the codebase easier to read on the whole and=20
> > allowing us to spend more time in code review discussing substantive is=
sues=20
> > over trivial formatting matters.=20
> >=20
> > This policy change will affect all Python code in-tree, including=20
> > sandboxed Python code used by the build system (.configure, .build, and=
=20
> > .mozbuild files).=20
> >=20
> > As part of this policy change, we plan on doing a one-time auto-reforma=
t=20
> > on October 19 of all Python code in the entire repository. In addition,=
=20
> > mach lint (=20
> > https://firefox-source-docs.mozilla.org/code-quality/lint/linters/black=
..html)=20
> > and reviewbot will be updated to print warnings for Python source files=
=20
> > that violate the black style. Just like with C/C++ or Rust, we won=E2=
=80=99t=20
> > backout offending changes but instead will do regular refreshes of the =
tree.=20
> >=20
> > If there are any questions, please let me know!=20
> >
> Is there some way we can see the results of this change before it lands i=
n=20
> the tree? I would like to review the results of formatting for Python fil=
es=20
> I care about to make sure nothing gets too messed up by it. For instance,=
=20
> clang-format did some kind of line wrapping for wide comments that broke =
a=20
> lot of ASCII diagram comments, and I and others had to spend a fair amoun=
t=20
> of time fixing that.=20
>=20
> Relatedly, is there some way to disable formatting for specific lines of=
=20
> code, in case there's something that gets mangled by the formatting and=
=20
> there's no easy way to fix it?=20
>=20
> What does "all Python code in-tree" mean? Does that mean that this is goi=
ng=20
> to be run on all files that end in .py? I assume that this doesn't apply =
to=20
> any third party code we have in the tree (either under the third-party=20
> directory, or included in whatever other list I think we have)?=20
>=20
> Thanks,=20
> Andrew=20
>=20
>=20
> > Ricky=20
> > _______________________________________________=20
> > dev-platform mailing list=20
> > dev-pl...@lists.mozilla.org=20
> > https://lists.mozilla.org/listinfo/dev-platform=20
> >
black does not reformat comments: https://github.com/psf/black/issues/181

Third-party code is exempt from being auto-formatted.
0
Ricky
10/14/2020 8:52:42 PM
 Thanks for driving this Ricky! I'm *very* excited for it.

Just want to call out that you can use:
$ ./mach lint -wo --fix

to reformat all the files you have touched (either in the working directory
or outgoing commits). For more usage docs, see:
https://firefox-source-docs.mozilla.org/code-quality/lint/usage.html

-Andrew

On Mon, Oct 12, 2020 at 1:05 PM Ricky Stewart <rstewart@mozilla.com> wrote:

> Hello everyone,
>
> If you don't write Python code in mozilla-central, you can stop reading
> now.
>
> On October 19, 2020 we will be officially adopting the black Python style
> for all our Python code in mozilla-central.
>
> black (https://black.readthedocs.io/en/stable/) is an opinionated, fast,
> and correct auto-formatter for Python. It is an increasingly popular
> autoformatter which might be considered the de facto standard for Python
> code (like clang-format and jslint are for C++ and JS). It is already use=
d
> by several Mozilla projects, including Release Engineering, Lando, and
> moz-phab.
>
> black makes it easy for us to reliably format all our Python code in a
> consistent way, making the codebase easier to read on the whole and
> allowing us to spend more time in code review discussing substantive issu=
es
> over trivial formatting matters.
>
> This policy change will affect all Python code in-tree, including
> sandboxed Python code used by the build system (.configure, .build, and
> .mozbuild files).
>
> As part of this policy change, we plan on doing a one-time auto-reformat
> on October 19 of all Python code in the entire repository. In addition,
> mach lint (
> https://firefox-source-docs.mozilla.org/code-quality/lint/linters/black.h=
tml)
> and reviewbot will be updated to print warnings for Python source files
> that violate the black style. Just like with C/C++ or Rust, we won=E2=80=
=99t
> backout offending changes but instead will do regular refreshes of the tr=
ee.
>
> If there are any questions, please let me know!
>
> Ricky
> _______________________________________________
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>
0
Andrew
10/15/2020 3:49:05 PM
Is black still opiniated about string types and insisting to use double
quotes, when we mostly settled on single quotes?

On Mon, Oct 12, 2020 at 10:00:56AM -0700, Ricky Stewart wrote:
> Hello everyone,
> 
> If you don't write Python code in mozilla-central, you can stop reading now.
> 
> On October 19, 2020 we will be officially adopting the black Python style for all our Python code in mozilla-central.
> 
> black (https://black.readthedocs.io/en/stable/) is an opinionated, fast, and correct auto-formatter for Python. It is an increasingly popular autoformatter which might be considered the de facto standard for Python code (like clang-format and jslint are for C++ and JS). It is already used by several Mozilla projects, including Release Engineering, Lando, and moz-phab.
> 
> black makes it easy for us to reliably format all our Python code in a consistent way, making the codebase easier to read on the whole and allowing us to spend more time in code review discussing substantive issues over trivial formatting matters.
> 
> This policy change will affect all Python code in-tree, including sandboxed Python code used by the build system (.configure, .build, and .mozbuild files).
> 
> As part of this policy change, we plan on doing a one-time auto-reformat on October 19 of all Python code in the entire repository. In addition, mach lint (https://firefox-source-docs.mozilla.org/code-quality/lint/linters/black.html) and reviewbot will be updated to print warnings for Python source files that violate the black style. Just like with C/C++ or Rust, we won’t backout offending changes but instead will do regular refreshes of the tree.
> 
> If there are any questions, please let me know!
> 
> Ricky
> _______________________________________________
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
0
Mike
10/15/2020 9:23:17 PM
No, black now has a `--skip-string-normalization` flag, which I would be
all for using in our code base. Not sure if that was the plan here or not.

-Andrew

p.s It took a great deal of convincing from the community to get this flag
added, as the maintainers precisely wanted to prevent conversations like
this current one from happening :p.

On Thu, Oct 15, 2020 at 5:23 PM Mike Hommey <mh@glandium.org> wrote:

> Is black still opiniated about string types and insisting to use double
> quotes, when we mostly settled on single quotes?
>
> On Mon, Oct 12, 2020 at 10:00:56AM -0700, Ricky Stewart wrote:
> > Hello everyone,
> >
> > If you don't write Python code in mozilla-central, you can stop reading
> now.
> >
> > On October 19, 2020 we will be officially adopting the black Python
> style for all our Python code in mozilla-central.
> >
> > black (https://black.readthedocs.io/en/stable/) is an opinionated,
> fast, and correct auto-formatter for Python. It is an increasingly popula=
r
> autoformatter which might be considered the de facto standard for Python
> code (like clang-format and jslint are for C++ and JS). It is already use=
d
> by several Mozilla projects, including Release Engineering, Lando, and
> moz-phab.
> >
> > black makes it easy for us to reliably format all our Python code in a
> consistent way, making the codebase easier to read on the whole and
> allowing us to spend more time in code review discussing substantive issu=
es
> over trivial formatting matters.
> >
> > This policy change will affect all Python code in-tree, including
> sandboxed Python code used by the build system (.configure, .build, and
> .mozbuild files).
> >
> > As part of this policy change, we plan on doing a one-time auto-reforma=
t
> on October 19 of all Python code in the entire repository. In addition,
> mach lint (
> https://firefox-source-docs.mozilla.org/code-quality/lint/linters/black.h=
tml)
> and reviewbot will be updated to print warnings for Python source files
> that violate the black style. Just like with C/C++ or Rust, we won=E2=80=
=99t
> backout offending changes but instead will do regular refreshes of the tr=
ee.
> >
> > If there are any questions, please let me know!
> >
> > Ricky
> > _______________________________________________
> > dev-platform mailing list
> > dev-platform@lists.mozilla.org
> > https://lists.mozilla.org/listinfo/dev-platform
> _______________________________________________
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>
0
Andrew
10/19/2020 1:53:38 PM
On Monday, October 19, 2020 at 8:53:59 AM UTC-5, Andrew Halberstadt wrote:
> No, black now has a `--skip-string-normalization` flag, which I would be=
=20
> all for using in our code base. Not sure if that was the plan here or not=
..=20
>=20
> -Andrew=20
>=20
> p.s It took a great deal of convincing from the community to get this fla=
g=20
> added, as the maintainers precisely wanted to prevent conversations like=
=20
> this current one from happening :p.
> On Thu, Oct 15, 2020 at 5:23 PM Mike Hommey <m...@glandium.org> wrote:=20
>=20
> > Is black still opiniated about string types and insisting to use double=
=20
> > quotes, when we mostly settled on single quotes?=20
> >=20
> > On Mon, Oct 12, 2020 at 10:00:56AM -0700, Ricky Stewart wrote:
> > > Hello everyone,=20
> > >=20
> > > If you don't write Python code in mozilla-central, you can stop readi=
ng=20
> > now.=20
> > >=20
> > > On October 19, 2020 we will be officially adopting the black Python=
=20
> > style for all our Python code in mozilla-central.=20
> > >=20
> > > black (https://black.readthedocs.io/en/stable/) is an opinionated,=20
> > fast, and correct auto-formatter for Python. It is an increasingly popu=
lar=20
> > autoformatter which might be considered the de facto standard for Pytho=
n=20
> > code (like clang-format and jslint are for C++ and JS). It is already u=
sed=20
> > by several Mozilla projects, including Release Engineering, Lando, and=
=20
> > moz-phab.=20
> > >=20
> > > black makes it easy for us to reliably format all our Python code in =
a=20
> > consistent way, making the codebase easier to read on the whole and=20
> > allowing us to spend more time in code review discussing substantive is=
sues=20
> > over trivial formatting matters.=20
> > >=20
> > > This policy change will affect all Python code in-tree, including=20
> > sandboxed Python code used by the build system (.configure, .build, and=
=20
> > .mozbuild files).=20
> > >=20
> > > As part of this policy change, we plan on doing a one-time auto-refor=
mat=20
> > on October 19 of all Python code in the entire repository. In addition,=
=20
> > mach lint (=20
> > https://firefox-source-docs.mozilla.org/code-quality/lint/linters/black=
..html)=20
> > and reviewbot will be updated to print warnings for Python source files=
=20
> > that violate the black style. Just like with C/C++ or Rust, we won=E2=
=80=99t=20
> > backout offending changes but instead will do regular refreshes of the =
tree.=20
> > >=20
> > > If there are any questions, please let me know!=20
> > >=20
> > > Ricky
> > > _______________________________________________=20
> > > dev-platform mailing list=20
> > > dev-pl...@lists.mozilla.org=20
> > > https://lists.mozilla.org/listinfo/dev-platform=20
> > _______________________________________________=20
> > dev-platform mailing list=20
> > dev-pl...@lists.mozilla.org=20
> > https://lists.mozilla.org/listinfo/dev-platform=20
> >

--skip-string-normalization does exist, but the plan is not to use it. Whil=
e there is a cost associated with reformatting many (but not all) of the st=
rings in our Python code to be consistent with the black style, we don't th=
ink that that cost outweighs the benefit of doing so.
0
Ricky
10/19/2020 8:55:04 PM
I'm disappointed by that.

On Mon, Oct 19, 2020 at 2:00 PM Ricky Stewart <rstewart@mozilla.com> wrote:
>
> On Monday, October 19, 2020 at 8:53:59 AM UTC-5, Andrew Halberstadt wrote=
:
> > No, black now has a `--skip-string-normalization` flag, which I would b=
e
> > all for using in our code base. Not sure if that was the plan here or n=
ot.
> >
> > -Andrew
> >
> > p.s It took a great deal of convincing from the community to get this f=
lag
> > added, as the maintainers precisely wanted to prevent conversations lik=
e
> > this current one from happening :p.
> > On Thu, Oct 15, 2020 at 5:23 PM Mike Hommey <m...@glandium.org> wrote:
> >
> > > Is black still opiniated about string types and insisting to use doub=
le
> > > quotes, when we mostly settled on single quotes?
> > >
> > > On Mon, Oct 12, 2020 at 10:00:56AM -0700, Ricky Stewart wrote:
> > > > Hello everyone,
> > > >
> > > > If you don't write Python code in mozilla-central, you can stop rea=
ding
> > > now.
> > > >
> > > > On October 19, 2020 we will be officially adopting the black Python
> > > style for all our Python code in mozilla-central.
> > > >
> > > > black (https://black.readthedocs.io/en/stable/) is an opinionated,
> > > fast, and correct auto-formatter for Python. It is an increasingly po=
pular
> > > autoformatter which might be considered the de facto standard for Pyt=
hon
> > > code (like clang-format and jslint are for C++ and JS). It is already=
 used
> > > by several Mozilla projects, including Release Engineering, Lando, an=
d
> > > moz-phab.
> > > >
> > > > black makes it easy for us to reliably format all our Python code i=
n a
> > > consistent way, making the codebase easier to read on the whole and
> > > allowing us to spend more time in code review discussing substantive =
issues
> > > over trivial formatting matters.
> > > >
> > > > This policy change will affect all Python code in-tree, including
> > > sandboxed Python code used by the build system (.configure, .build, a=
nd
> > > .mozbuild files).
> > > >
> > > > As part of this policy change, we plan on doing a one-time auto-ref=
ormat
> > > on October 19 of all Python code in the entire repository. In additio=
n,
> > > mach lint (
> > > https://firefox-source-docs.mozilla.org/code-quality/lint/linters/bla=
ck.html)
> > > and reviewbot will be updated to print warnings for Python source fil=
es
> > > that violate the black style. Just like with C/C++ or Rust, we won=E2=
=80=99t
> > > backout offending changes but instead will do regular refreshes of th=
e tree.
> > > >
> > > > If there are any questions, please let me know!
> > > >
> > > > Ricky
> > > > _______________________________________________
> > > > dev-platform mailing list
> > > > dev-pl...@lists.mozilla.org
> > > > https://lists.mozilla.org/listinfo/dev-platform
> > > _______________________________________________
> > > dev-platform mailing list
> > > dev-pl...@lists.mozilla.org
> > > https://lists.mozilla.org/listinfo/dev-platform
> > >
>
> --skip-string-normalization does exist, but the plan is not to use it. Wh=
ile there is a cost associated with reformatting many (but not all) of the =
strings in our Python code to be consistent with the black style, we don't =
think that that cost outweighs the benefit of doing so.
> _______________________________________________
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
0
Jeff
10/19/2020 9:01:41 PM
On 19/10/2020 22:01, Jeff Gilbert wrote:
> I'm disappointed by that.

FWIW last time I looked at black, I found that the compromises it made 
to be fully automatic and with minimal configuration meant that it was 
liable to produce ugly or difficult to read code in some situations.

I understand that we've decided that people will get used to reading any 
code style over time, and therefore eliminating formatting concerns from 
the code writing process is a net win for productivity. So I'm not 
taking a position in opposition to this proposal, but it is not 
something I would have advocated personally.
0
James
10/20/2020 8:17:20 AM
Well we generally don't seek consensus anymore for these sorts of
things, it seems, but it's reassuring that I'm not alone.

On Tue, Oct 20, 2020 at 1:17 AM James Graham <james@hoppipolla.co.uk> wrote:
>
> On 19/10/2020 22:01, Jeff Gilbert wrote:
> > I'm disappointed by that.
>
> FWIW last time I looked at black, I found that the compromises it made
> to be fully automatic and with minimal configuration meant that it was
> liable to produce ugly or difficult to read code in some situations.
>
> I understand that we've decided that people will get used to reading any
> code style over time, and therefore eliminating formatting concerns from
> the code writing process is a net win for productivity. So I'm not
> taking a position in opposition to this proposal, but it is not
> something I would have advocated personally.
> _______________________________________________
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
0
Jeff
10/20/2020 8:26:09 PM
Reply: