Exploring requiring Node.js to build Firefox

--001a11449a3c1eef7b05637ad72b
Content-Type: text/plain; charset="UTF-8"

Speaking as a maintainer of the Firefox build system, we try to be
conservative about the set of dependencies required to build Firefox from
source. We recognize that every required dependency is a potential source
of failure and complexity. Dependencies can create complications for
packagers. From a build system perspective, it's in our best interest to
minimize the dependencies required to build Firefox.

Our desire to keep things simple can be at odds with the wishes of Firefox
developers who wish to leverage new and exciting tools and technologies. In
short, the policy of minimizing build dependencies can externalize costs
onto overall Firefox development by hindering people from using better
tools. This adversely affects the development velocity of Firefox and can
cause product quality to suffer.

For a few years now, various pockets of Firefox development have wanted to
use Node.js as part of the development workflow. I wouldn't say they
directly want to use Node.js: they want to use the large ecosystem of tools
built around Node.js. But that's splitting hairs. Our build system policy
has been that leveraging Node.js and its ecosystem for supplemental
workflows is fine, but shipping a build dependency on Node.js is not. Many
Firefox developers are now using Node.js in their day-to-day workflow for
things like running ESLint. We're using Node.js in CI. But we're not
forcing people to have Node.js installed to build Firefox.

Various groups have routed around the limitation that Node.js can't be
required to build Firefox. There are now Firefox features that do require
Node.js to build. However, the output from the Node.js tools is checked
into the Firefox source repository. So from the perspective of the Firefox
build system, Node.js doesn't exist and therefore isn't a build dependency.

The status quo is not ideal. The more people I speak with, the more
apparent it is that our current policy of not allowing Node.js tooling in
the build system is causing more problems than it is preventing. Speaking
as the build system module owner and someone who cares about developer
workflows, tooling, and developer productivity, I don't think the current
policy is good for Mozilla.

I'd like to start a discussion about requiring Node.js to build Firefox.

What do I mean by "require Node.js?" Let's assume I mean having a usable
Node.js executable on the host system to be used during a Firefox build.

What about npm or a package manager? I would strongly prefer to limit the
required dependency to Node.js itself. While the Firefox build system would
depend on 3rd party packages and tools (such as Babel), I'm pretty
insistent (as a build system maintainer) that these dependencies be
vendored into the Firefox source repository so as to not incur a run-time
dependency on a packaging service. I've seen the chaos that "left-pad"
caused. I don't fully trust the security model of JavaScript package
distribution. I don't think we can risk the ability to build Firefox or the
integrity of the Firefox product by the availability and integrity of a 3rd
party packaging service. That may sound like a harsh thing to say. But it's
the posture we've applied elsewhere (such as to Python packages and PyPI).
So, this means that all JavaScript executed by Node.js as part of the build
would either be provided by Node.js itself or the Firefox source
repository. If we needed to use a package manager as part of the build,
that package manager could be vendored in the Firefox repository along with
other JavaScript libraries (not unlike how we currently vendor Python's pip
package manager).

A few people at Mozilla have poked at this problem already. We have a
general sense of where some pain points for us will be. We know that
getting modern versions of Node.js installed on various distributions
requires using 3rd party package repositories. We know that Windows support
could be painful. We know that installing common packages can result of
dozens if not hundreds of dependencies being added. We know this could lead
to us having to install thousands of files as part of the Firefox build -
an overhead I'm not keen on seeing. We know all of this can add up to a
significant amount of overhead to support. (Yet it still feels like a
lesser problem than having people work around not being able to use Node.js
directly.)

What we don't generally know is the impact requiring Node.js would have on
downstream packagers. Our adoption of Rust last year was a long and
sometimes painful process. I have a feeling that requiring Node.js would be
a similar experience. But like Rust, I feel that adopting Node.js is in the
best long-term interest for Firefox development velocity and product
quality. I'm reluctant to cause more hardship by introducing a new build
dependency. But it's very difficult to keep saying we can't use Node.js in
the Firefox build system. I wish I could say "we'll build SpiderMonkey and
use that instead." Unfortunately, many Node.js tools don't work with
SpiderMonkey, so that's not an option. Plus there are difficulties with
cross-compilation. As sad as it makes me to say it, SpiderMonkey is not an
option: Node.js is the only viable option.

If we require Node.js to build Firefox, what are the requirements, desires,
and hardships of downstream packagers and consumers of the Firefox build
system? Keep in mind that mozilla-central right now is Firefox 60. That
will become ESR 60 in May.

Gregory Szorc
gps@mozilla.com
Firefox build system module owner

--001a11449a3c1eef7b05637ad72b
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>Speaking as a maintainer of the Firefox build system,=
 we try to be conservative about the set of dependencies required to build =
Firefox from source. We recognize that every required dependency is a poten=
tial source of failure and complexity. Dependencies can create complication=
s for packagers. From a build system perspective, it&#39;s in our best inte=
rest to minimize the dependencies required to build Firefox.</div><div><br>=
</div><div>Our desire to keep things simple can be at odds with the wishes =
of Firefox developers who wish to leverage new and exciting tools and techn=
ologies. In short, the policy of minimizing build dependencies can external=
ize costs onto overall Firefox development by hindering people from using b=
etter tools. This adversely affects the development velocity of Firefox and=
 can cause product quality to suffer.</div><div><br></div><div>For a few ye=
ars now, various pockets of Firefox development have wanted to use Node.js =
as part of the development workflow. I wouldn&#39;t say they directly want =
to use Node.js: they want to use the large ecosystem of tools built around =
Node.js. But that&#39;s splitting hairs. Our build system policy has been t=
hat leveraging Node.js and its ecosystem for supplemental workflows is fine=
, but shipping a build dependency on Node.js is not. Many Firefox developer=
s are now using Node.js in their day-to-day workflow for things like runnin=
g ESLint. We&#39;re using Node.js in CI. But we&#39;re not forcing people t=
o have Node.js installed to build Firefox.</div><div><br></div><div>Various=
 groups have routed around the limitation that Node.js can&#39;t be require=
d to build Firefox. There are now Firefox features that do require Node.js =
to build. However, the output from the Node.js tools is checked into the Fi=
refox source repository. So from the perspective of the Firefox build syste=
m, Node.js doesn&#39;t exist and therefore isn&#39;t a build dependency.</d=
iv><div><br></div><div>The status quo is not ideal. The more people I speak=
 with, the more apparent it is that our current policy of not allowing Node=
..js tooling in the build system is causing more problems than it is prevent=
ing. Speaking as the build system module owner and someone who cares about =
developer workflows, tooling, and developer productivity, I don&#39;t think=
 the current policy is good for Mozilla.</div><div><br></div><div>I&#39;d l=
ike to start a discussion about requiring Node.js to build Firefox.</div><d=
iv><br></div><div>What do I mean by &quot;require Node.js?&quot; Let&#39;s =
assume I mean having a usable Node.js executable on the host system to be u=
sed during a Firefox build.</div><div><br></div><div>What about npm or a pa=
ckage manager? I would strongly prefer to limit the required dependency to =
Node.js itself. While the Firefox build system would depend on 3rd party pa=
ckages and tools (such as Babel), I&#39;m pretty insistent (as a build syst=
em maintainer) that these dependencies be vendored into the Firefox source =
repository so as to not incur a run-time dependency on a packaging service.=
 I&#39;ve seen the chaos that &quot;left-pad&quot; caused. I don&#39;t full=
y trust the security model of JavaScript package distribution. I don&#39;t =
think we can risk the ability to build Firefox or the integrity of the Fire=
fox product by the availability and integrity of a 3rd party packaging serv=
ice. That may sound like a harsh thing to say. But it&#39;s the posture we&=
#39;ve applied elsewhere (such as to Python packages and PyPI). So, this me=
ans that all JavaScript executed by Node.js as part of the build would eith=
er be provided by Node.js itself or the Firefox source repository. If we ne=
eded to use a package manager as part of the build, that package manager co=
uld be vendored in the Firefox repository along with other JavaScript libra=
ries (not unlike how we currently vendor Python&#39;s pip package manager).=
<br></div><div><br></div><div>A few people at Mozilla have poked at this pr=
oblem already. We have a general sense of where some pain points for us wil=
l be. We know that getting modern versions of Node.js installed on various =
distributions requires using 3rd party package repositories. We know that W=
indows support could be painful. We know that installing common packages ca=
n result of dozens if not hundreds of dependencies being added. We know thi=
s could lead to us having to install thousands of files as part of the Fire=
fox build - an overhead I&#39;m not keen on seeing. We know all of this can=
 add up to a significant amount of overhead to support. (Yet it still feels=
 like a lesser problem than having people work around not being able to use=
 Node.js directly.)</div><div><br></div><div>What we don&#39;t generally kn=
ow is the impact requiring Node.js would have on downstream packagers. Our =
adoption of Rust last year was a long and sometimes painful process. I have=
 a feeling that requiring Node.js would be a similar experience. But like R=
ust, I feel that adopting Node.js is in the best long-term interest for Fir=
efox development velocity and product quality. I&#39;m reluctant to cause m=
ore hardship by introducing a new build dependency. But it&#39;s very diffi=
cult to keep saying we can&#39;t use Node.js in the Firefox build system. I=
 wish I could say &quot;we&#39;ll build SpiderMonkey and use that instead.&=
quot; Unfortunately, many Node.js tools don&#39;t work with SpiderMonkey, s=
o that&#39;s not an option. Plus there are difficulties with cross-compilat=
ion. As sad as it makes me to say it, SpiderMonkey is not an option: Node.j=
s is the only viable option.<br></div><div><br></div><div>If we require Nod=
e.js to build Firefox, what are the requirements, desires, and hardships of=
 downstream packagers and consumers of the Firefox build system? Keep in mi=
nd that mozilla-central right now is Firefox 60. That will become ESR 60 in=
 May.<br></div><div><br></div><div>Gregory Szorc</div><div><a href=3D"mailt=
o:gps@mozilla.com">gps@mozilla.com</a></div><div>Firefox build system modul=
e owner<br></div></div>

--001a11449a3c1eef7b05637ad72b--
0
Gregory
1/24/2018 12:34:49 AM
mozilla.dev.builds 1693 articles. 0 followers. Post Follow

0 Replies
13 Views

Similar Articles

[PageSpeed] 39

Reply: