Avoiding the hard build dependency on NodeJS

I understand the desire for developers do use the best tool for each job, a=
nd integrating NodeJS in the regular work flow during development might be =
a good idea.

However, requiring NodeJS for each build (including the official releases) =
causes great problems: NodeJS itself is way less portable than Firefox, and=
 I am building one of the strangler platforms - on sparc64. NodeJS seems to=
 have no support at all for sparc CPUs, and I am not yet sure how hard it w=
ould be to add. I will investigate (as time permits).

So I have a concrete question: are the outputs of the NodeJS runs during a =
build actually target machine / OS dependent? At first I would hope not, an=
d in this case: would it be possible to pre-generate the NodeJS output file=
s for each official release, and then allow building with those pre-generat=
ed files instead of a local NodeJS installation?

Thanks,

Martin
0
Martin
11/7/2018 1:32:40 PM
mozilla.dev.builds 1720 articles. 0 followers. Post Follow

7 Replies
19 Views

Similar Articles

[PageSpeed] 21

On 11/7/18 8:32 AM, Martin Husemann wrote:
> NodeJS seems to have no support at all for sparc CPUs, and I am not yet sure how hard it would be to add.

You'd need to port V8 to work on sparc, right?  That sounds fairly 
painful...

-Boris
0
Boris
11/7/2018 2:22:56 PM
On Wednesday, November 7, 2018 at 3:23:02 PM UTC+1, Boris Zbarsky wrote:

> You'd need to port V8 to work on sparc, right?  That sounds fairly 
> painful...

Or port node to Spidermonkey, might be easier :-)
0
Martin
11/7/2018 2:30:56 PM
On 11/7/18 9:30 AM, Martin Husemann wrote:
> Or port node to Spidermonkey, might be easier :-)

True.  https://github.com/mozilla/spidernode/ is not really active, but 
might be revivable...

-Boris
0
Boris
11/7/2018 2:42:58 PM
--000000000000cf2998057a181f18
Content-Type: text/plain; charset="UTF-8"

On Wed, Nov 7, 2018 at 5:35 AM Martin Husemann <martin@duskware.de> wrote:

> I understand the desire for developers do use the best tool for each job,
> and integrating NodeJS in the regular work flow during development might be
> a good idea.
>
> However, requiring NodeJS for each build (including the official releases)
> causes great problems: NodeJS itself is way less portable than Firefox, and
> I am building one of the strangler platforms - on sparc64. NodeJS seems to
> have no support at all for sparc CPUs, and I am not yet sure how hard it
> would be to add. I will investigate (as time permits).
>
> So I have a concrete question: are the outputs of the NodeJS runs during a
> build actually target machine / OS dependent? At first I would hope not,
> and in this case: would it be possible to pre-generate the NodeJS output
> files for each official release, and then allow building with those
> pre-generated files instead of a local NodeJS installation?
>

I understand your concerns and empathize with your position. One of the
reasons we held off on integrating NodeJS in Firefox's build system for so
many years were concerns like this. We recognize that Firefox is the only
viable modern web browser that can still be built on and for some
platforms. Losing that could be devastating.

You asked a concrete question about build outputs. At this time, I don't
foresee the outputs of NodeJS being target machine / OS dependent. i.e. I
highly doubt we'll be producing executables, shared/static libraries, etc
with NodeJS. I imagine anything using NodeJS will be related to JavaScript
processing. And that should be machine / OS agnostic. Our go to language
for build support tooling will likely continue to be Python for the
foreseeable future. And we may start using more Rust in this space.

Would it be possible to pre-generate files requiring Node and provide a
Firefox source distribution that doesn't require Node to build? In theory,
yes. In fact, this is what some Firefox components have been doing! Before
there was Node support, people would run Node out-of-band with the Firefox
build system and check in the generated files into source control, where
they were processed by the Node-less Firefox build system! (This workflow
was not very efficient, hence one of the reasons for wanting to incorporate
Node into the build system.)

While such a source distribution would be possible, maintaining it
constitutes work. When we decided to require Node, we had to balance the
loss of productivity that the lack of Node was costing us against the cost
of supporting Node and other factors like the loss of some platforms being
able to build Firefox natively. In the end, we decided that lack of Node
was causing too much of a burden on Firefox developers and we needed native
support for Node. While there was grumbling about this decision from some,
overall the negative reaction seems relatively muted. At this time, I don't
think an officially supported source distribution that supports building
without Node can be justified. Maintaining such a distribution will
constitute a non-trivial amount of work and will put a burden on our
limited number of build system maintainers.

That being said, once the initial dust from requiring Node has settled and
the build system Node integration bits are a bit more well-defined, it's
quite possible that maintaining such a source distribution would be
relatively easy. If someone were to contribute patches and offer to fix
things when they break, we could try to support such a distribution. But
even this could have future costs on build system maintenance. I'm hesitant
to give blanket commitment to accepting patches if someone does this work.
It's much easier to say "any new code not immediately beneficial to
Mozilla/Firefox creates a maintenance burden and therefore should be
rejected." I think a good first step would be for someone to produce a
proof-of-concept demonstrating what things would like look. Take the
existing Firefox source code, modify it to do what you want, and let's see
what that looks like. In the worst case, you've re-enabled building Firefox
on platforms like SPARC with a few patches. In the best case, the patches
are accepted upstream and we commit to supporting a Firefox source
distribution that doesn't require Node.

Gregory

--000000000000cf2998057a181f18
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div class=3D"gmail_quote"><div dir=3D"ltr">On Wed, Nov 7,=
 2018 at 5:35 AM Martin Husemann &lt;<a href=3D"mailto:martin@duskware.de">=
martin@duskware.de</a>&gt; wrote:<br></div><blockquote class=3D"gmail_quote=
" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">I=
 understand the desire for developers do use the best tool for each job, an=
d integrating NodeJS in the regular work flow during development might be a=
 good idea.<br>
<br>
However, requiring NodeJS for each build (including the official releases) =
causes great problems: NodeJS itself is way less portable than Firefox, and=
 I am building one of the strangler platforms - on sparc64. NodeJS seems to=
 have no support at all for sparc CPUs, and I am not yet sure how hard it w=
ould be to add. I will investigate (as time permits).<br>
<br>
So I have a concrete question: are the outputs of the NodeJS runs during a =
build actually target machine / OS dependent? At first I would hope not, an=
d in this case: would it be possible to pre-generate the NodeJS output file=
s for each official release, and then allow building with those pre-generat=
ed files instead of a local NodeJS installation?<br></blockquote><div><br><=
/div><div>I understand your concerns and empathize with your position. One =
of the reasons we held off on integrating NodeJS in Firefox&#39;s build sys=
tem for so many years were concerns like this. We recognize that Firefox is=
 the only viable modern web browser that can still be built on and for some=
 platforms. Losing that could be devastating.</div><div><br></div><div>You =
asked a concrete question about build outputs. At this time, I don&#39;t fo=
resee the outputs of NodeJS being target machine / OS dependent. i.e. I hig=
hly doubt we&#39;ll be producing executables, shared/static libraries, etc =
with NodeJS. I imagine anything using NodeJS will be related to JavaScript =
processing. And that should be machine / OS agnostic. Our go to language fo=
r build support tooling will likely continue to be Python for the foreseeab=
le future. And we may start using more Rust in this space.<br></div><div><b=
r></div><div>Would it be possible to pre-generate files requiring Node and =
provide a Firefox source distribution that doesn&#39;t require Node to buil=
d? In theory, yes. In fact, this is what some Firefox components have been =
doing! Before there was Node support, people would run Node out-of-band wit=
h the Firefox build system and check in the generated files into source con=
trol, where they were processed by the Node-less Firefox build system! (Thi=
s workflow was not very efficient, hence one of the reasons for wanting to =
incorporate Node into the build system.)</div><div><br></div><div>While suc=
h a source distribution would be possible, maintaining it constitutes work.=
 When we decided to require Node, we had to balance the loss of productivit=
y that the lack of Node was costing us against the cost of supporting Node =
and other factors like the loss of some platforms being able to build Firef=
ox natively. In the end, we decided that lack of Node was causing too much =
of a burden on Firefox developers and we needed native support for Node. Wh=
ile there was grumbling about this decision from some, overall the negative=
 reaction seems relatively muted. At this time, I don&#39;t think an offici=
ally supported source distribution that supports building without Node can =
be justified. Maintaining such a distribution will constitute a non-trivial=
 amount of work and will put a burden on our limited number of build system=
 maintainers.</div><div><br></div><div>That being said, once the initial du=
st from requiring Node has settled and the build system Node integration bi=
ts are a bit more well-defined, it&#39;s quite possible that maintaining su=
ch a source distribution would be relatively easy. If someone were to contr=
ibute patches and offer to fix things when they break, we could try to supp=
ort such a distribution. But even this could have future costs on build sys=
tem maintenance. I&#39;m hesitant to give blanket commitment to accepting p=
atches if someone does this work. It&#39;s much easier to say &quot;any new=
 code not immediately beneficial to Mozilla/Firefox creates a maintenance b=
urden and therefore should be rejected.&quot; I think a good first step wou=
ld be for someone to produce a proof-of-concept demonstrating what things w=
ould like look. Take the existing Firefox source code, modify it to do what=
 you want, and let&#39;s see what that looks like. In the worst case, you&#=
39;ve re-enabled building Firefox on platforms like SPARC with a few patche=
s. In the best case, the patches are accepted upstream and we commit to sup=
porting a Firefox source distribution that doesn&#39;t require Node.</div><=
div><br></div><div>Gregory</div></div></div>

--000000000000cf2998057a181f18--
0
Gregory
11/7/2018 7:27:56 PM
--0000000000001eb49c057a286e4e
Content-Type: text/plain; charset="UTF-8"

I wonder if ChakraCore (the JS engine used inside Edge) has a JavaScript
interpreter that would make it viable to run node-chakracore (
https://github.com/nodejs/node-chakracore) to run on sparc64.  Have you
tried that by any chance?  (Of course even if that works, how compatible
the node_modules used in mozilla-central would be with node-chakracore
remains to be seen...)

Cheers,
Ehsan

On Wed, Nov 7, 2018 at 8:35 AM Martin Husemann <martin@duskware.de> wrote:

> I understand the desire for developers do use the best tool for each job,
> and integrating NodeJS in the regular work flow during development might be
> a good idea.
>
> However, requiring NodeJS for each build (including the official releases)
> causes great problems: NodeJS itself is way less portable than Firefox, and
> I am building one of the strangler platforms - on sparc64. NodeJS seems to
> have no support at all for sparc CPUs, and I am not yet sure how hard it
> would be to add. I will investigate (as time permits).
>
> So I have a concrete question: are the outputs of the NodeJS runs during a
> build actually target machine / OS dependent? At first I would hope not,
> and in this case: would it be possible to pre-generate the NodeJS output
> files for each official release, and then allow building with those
> pre-generated files instead of a local NodeJS installation?
>
> Thanks,
>
> Martin
> _______________________________________________
> dev-builds mailing list
> dev-builds@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-builds
>


-- 
Ehsan

--0000000000001eb49c057a286e4e
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr">I wonder if ChakraCore (the JS engine use=
d inside Edge) has a JavaScript interpreter that would make it viable to ru=
n node-chakracore (<a href=3D"https://github.com/nodejs/node-chakracore">ht=
tps://github.com/nodejs/node-chakracore</a>) to run on sparc64.=C2=A0 Have =
you tried that by any chance?=C2=A0 (Of course even if that works, how comp=
atible the node_modules used in mozilla-central would be with node-chakraco=
re remains to be seen...)</div><div dir=3D"ltr"><br></div><div>Cheers,</div=
><div>Ehsan<br></div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr">=
On Wed, Nov 7, 2018 at 8:35 AM Martin Husemann &lt;<a href=3D"mailto:martin=
@duskware.de">martin@duskware.de</a>&gt; wrote:<br></div><blockquote class=
=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padd=
ing-left:1ex">I understand the desire for developers do use the best tool f=
or each job, and integrating NodeJS in the regular work flow during develop=
ment might be a good idea.<br>
<br>
However, requiring NodeJS for each build (including the official releases) =
causes great problems: NodeJS itself is way less portable than Firefox, and=
 I am building one of the strangler platforms - on sparc64. NodeJS seems to=
 have no support at all for sparc CPUs, and I am not yet sure how hard it w=
ould be to add. I will investigate (as time permits).<br>
<br>
So I have a concrete question: are the outputs of the NodeJS runs during a =
build actually target machine / OS dependent? At first I would hope not, an=
d in this case: would it be possible to pre-generate the NodeJS output file=
s for each official release, and then allow building with those pre-generat=
ed files instead of a local NodeJS installation?<br>
<br>
Thanks,<br>
<br>
Martin<br>
_______________________________________________<br>
dev-builds mailing list<br>
<a href=3D"mailto:dev-builds@lists.mozilla.org" target=3D"_blank">dev-build=
s@lists.mozilla.org</a><br>
<a href=3D"https://lists.mozilla.org/listinfo/dev-builds" rel=3D"noreferrer=
" target=3D"_blank">https://lists.mozilla.org/listinfo/dev-builds</a><br>
</blockquote></div><br clear=3D"all"><br>-- <br><div dir=3D"ltr" class=3D"g=
mail_signature" data-smartmail=3D"gmail_signature"><div dir=3D"ltr">Ehsan<b=
r></div></div>

--0000000000001eb49c057a286e4e--
0
Ehsan
11/8/2018 2:55:12 PM
On Wed, Nov 07, 2018 at 05:32:40AM -0800, Martin Husemann wrote:
> I understand the desire for developers do use the best tool for each job, and integrating NodeJS in the regular work flow during development might be a good idea.
> 
> However, requiring NodeJS for each build (including the official releases) causes great problems: NodeJS itself is way less portable than Firefox, and I am building one of the strangler platforms - on sparc64. NodeJS seems to have no support at all for sparc CPUs, and I am not yet sure how hard it would be to add. I will investigate (as time permits).
> 
> So I have a concrete question: are the outputs of the NodeJS runs during a build actually target machine / OS dependent? At first I would hope not, and in this case: would it be possible to pre-generate the NodeJS output files for each official release, and then allow building with those pre-generated files instead of a local NodeJS installation?
> 

Note that for your _immediate_ needs, you can just --disable-nodejs,
because node is currently *not* actually used for anything. That won't
be the case for very long, though.

Mike
0
Mike
11/8/2018 9:55:24 PM
For better and for worse, I believe that's no longer true: devtools is
now using it to transpile JSX files to JS files as part of the normal
build.

Dan
Am Do., 8. Nov. 2018 um 13:55 Uhr schrieb Mike Hommey <mh@glandium.org>:
>
> On Wed, Nov 07, 2018 at 05:32:40AM -0800, Martin Husemann wrote:
> > I understand the desire for developers do use the best tool for each jo=
b, and integrating NodeJS in the regular work flow during development might=
 be a good idea.
> >
> > However, requiring NodeJS for each build (including the official releas=
es) causes great problems: NodeJS itself is way less portable than Firefox,=
 and I am building one of the strangler platforms - on sparc64. NodeJS seem=
s to have no support at all for sparc CPUs, and I am not yet sure how hard =
it would be to add. I will investigate (as time permits).
> >
> > So I have a concrete question: are the outputs of the NodeJS runs durin=
g a build actually target machine / OS dependent? At first I would hope not=
, and in this case: would it be possible to pre-generate the NodeJS output =
files for each official release, and then allow building with those pre-gen=
erated files instead of a local NodeJS installation?
> >
>
> Note that for your _immediate_ needs, you can just --disable-nodejs,
> because node is currently *not* actually used for anything. That won't
> be the case for very long, though.
>
> Mike
> _______________________________________________
> dev-builds mailing list
> dev-builds@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-builds
0
Dan
11/8/2018 10:08:46 PM
Reply: