How do I trace objects on the heap with JS 59

Hi,

I'm currently trying to update my embedding from a very old version (1.8.5)=
 to 59. I have lots of objects which I used to root using JS_AddObjectRoot.
These are stored on heap structures.
I'm trying to understand how to do the rooting in 59.

In the GC rooting guide on MDN at https://developer.mozilla.org/en-US/docs/=
Mozilla/Projects/SpiderMonkey/GC_Rooting_Guide#GC_things_on_the_heap it say=
s:

GC thing pointers on the heap must be wrapped in a JS::Heap<T>. The only ex=
ception to this is if they are added as roots with the JS_Add<T>Root() func=
tions or JS::PersistentRooted class, but don't do this unless it's really n=
ecessary.  JS::Heap<T> pointers must also continue to be traced in the norm=
al way, which is not covered here.

OK, so I need to use JS::Heap<JSObject *> and these need to be traced but h=
ow to do it isn't covered in the guide... Please can someone point me to so=
me documentation/reference which says how to do it. I can't find any inform=
ation anywhere...

Alternatively it looks like I could use the JS::PersistentRooted class to d=
o what I want but it specifically says not to do this unless it's really ne=
cessary. Why is that then? Is it some sort of performance issue?

Many thanks

Miles
0
Miles
8/9/2018 8:40:01 AM
mozilla.dev.tech.js-engine 2024 articles. 0 followers. Post Follow

16 Replies
41 Views

Similar Articles

[PageSpeed] 56

On 8/9/18 4:40 AM, Miles wrote:
> Alternatively it looks like I could use the JS::PersistentRooted class to do what I want but it specifically says not to do this unless it's really necessary. Why is that then? Is it some sort of performance issue?

Well, if nothing else it makes it easy to leak.  Say you have two C/C++ 
data structures X and Y.  X holds on to a PersistentRooted for object A, 
object A keeps Y alive, Y holds a PersistentRooted for object B, B keeps 
X alive.  Now you have a leak.

As far as how to trace things goes, it depends on what owns the heap 
data structures that are pointing to the JS objects.  If they are owned 
(possibly indirectly) by some SpiderMonkey object, its trace class hook 
should do the tracing.  If they just exist independently, then 
JS_AddExtraGCRootsTracer might be the right thing?

-Boris
0
Boris
8/9/2018 3:12:25 PM
On 08/09/2018 01:40 AM, Miles wrote:
> Hi,
> 
> I'm currently trying to update my embedding from a very old version (1.8.5) to 59. I have lots of objects which I used to root using JS_AddObjectRoot.
> These are stored on heap structures.
> I'm trying to understand how to do the rooting in 59.
> 
> In the GC rooting guide on MDN at https://developer.mozilla.org/en-US/docs/Mozilla/Projects/SpiderMonkey/GC_Rooting_Guide#GC_things_on_the_heap it says:
> 
> GC thing pointers on the heap must be wrapped in a JS::Heap<T>. The only exception to this is if they are added as roots with the JS_Add<T>Root() functions or JS::PersistentRooted class, but don't do this unless it's really necessary.  JS::Heap<T> pointers must also continue to be traced in the normal way, which is not covered here.
> 
> OK, so I need to use JS::Heap<JSObject *> and these need to be traced but how to do it isn't covered in the guide... Please can someone point me to some documentation/reference which says how to do it. I can't find any information anywhere...
> 
> Alternatively it looks like I could use the JS::PersistentRooted class to do what I want but it specifically says not to do this unless it's really necessary. Why is that then? Is it some sort of performance issue?

The exact equivalent of JS_Add*Root is JS::PersistentRooted. It is 
discouraged because it is ridiculously easy to keep things alive 
forever. For example, if the referent of a PersistentRooted field 
creates a cycle with its container, then nothing in or reachable from 
that cycle will ever be freed up. And since global objects are reachable 
from pretty much everything, that means that if your structure is 
reachable from any JS object, you'll have a cycle.

You want such fields to be traced, not rooted. That generally means that 
you define a trace() method on your structures that calls JS::TraceEdge 
on each of your Heap<T> fields. That is enough to put your structure in 
a Rooted<yourstruct> on the stack, btw, if you happen to want to.

The exact signature of trace() is

     void trace(JSTracer* trc, const char* name);

Actually, this spurred me to update the MDN documentation to include 
tracing: 
https://developer.mozilla.org/en-US/docs/Mozilla/Projects/SpiderMonkey/GC_Rooting_Guide#GC_things_on_the_heap
0
Steve
8/9/2018 5:46:54 PM
I'd like to add a question on this topic.

What if I have JSObjects that are already rooted? (Either as global
variables, or properties of rooted objects) And I have references to those
JSObject* pointers in my backend?

Do I need to change those JSObject* pointers to JS::Heap<JSObject*> in that
case? I always assumed that would be the case, but does it affect those
rooted JSObjects? Are they still the same object? If so, how do I convert
Heap<T> back to Rooted<T>?

I am a little confused. Do I need to call some kind of GC tracer API on
construction/deconstruction of my backend objects?

Thanks,
James
0
James
8/10/2018 3:44:53 AM
So, here's an actual example to clarify. Please tell me if this simple
concept is correct.

1.) On my backend objects, I set a pointer to a `new FrontEnd(cx,
rooted_obj_js)`
//----------------------------------------------
    void trace_obj(JSTracer* tracer, void* data) {
        JS::TraceEdge(tracer, (JS::Heap<JSObject*>*)data, "obj_js");
    };

    struct FrontEnd {
           FrontEnd(JSContext *cx, JS::HandleObject obj) {
               this->cx = cx;
               this->obj = obj;
               if (this->obj.get())
                   JS_AddExtraGCRootsTracer(this->cx, trace_obj,
&this->obj);
           };
          ~FrontEnd() {
               if (this->obj.get())
                   JS_RemoveExtraGCRootsTracer(this->cx, trace_obj,
&this->obj);
           };

        JSContext *cx;
        JS::Heap<JSObject*> obj;
    };
//----------------------------------------------
2.) back in the frontend (JS), I root that JSObject again like so:
//----------------------------------------------
    JS::RootedObject obj(cx, ((FrontEnd*)backend_obj->frontend)->obj);
//----------------------------------------------


3.) and in the finalizeOp of my frontend object (JS), I delete the FrontEnd
(and BackEnd) wrappers.
//----------------------------------------------
    void obj_finalize(JSFreeOp *fop, JSObject *obj) {
        auto *backend = (BackEnd*)JS_GetPrivate(obj);
        auto *frontend = (FrontEnd*)backend->obj->frontend;
        delete frontend;
        delete backend;
    };
//----------------------------------------------


Am I using the GC/tracer API correctly? I don't want to derive from
JSObject, just link the frontend and backend with wrappers. Want to know if
`JS_AddExtraGCRootsTracer`,`JS_RemmoveExtraGCRootsTracer`, and
`JS::TraceEdge` are used appropriately for this.

Hopefully this will help Miles too.

Thanks,
James
0
James
8/10/2018 9:34:49 AM
Thanks for the replies and for updating the documentation to include tracing. Much appreciated. It sounds like JS_AddExtraGCRootsTracer, JS::Heap<JSObject *> and JS::TraceEdge are what I need then.

One final question if I may...
GS thing pointers that are parameters to functions now use JS::Handle<T>. i.e. JS::Handle<JSObject *> (or JS::HandleObject) for objects.
If I use JS::Heap<JSObject *> can these be automatically coerced into JS::Handle<JSObject *> for passing to functions or do I have to do something else to be able to pass them as function parameters?

Thanks

Miles
0
Miles
8/13/2018 11:34:32 AM
On 8/13/18 7:34 AM, Miles wrote:
> If I use JS::Heap<JSObject *> can these be automatically coerced into JS::Handle<JSObject *> for passing to functions or do I have to do something else to be able to pass them as function parameters?

You generally need to put them in a Rooted on the stack.

The reason for that is that Handle<JSObject*> is really a JSObject** 
under the hood.  Heap<JSObject*> stores a JSObject*.  If we allowed 
creation of a Handle from it, that Handle would point inside the Heap. 
Then if the code you are calling triggers some JS that changes the value 
of your member (updating the JSObject* value stored), the handle would 
suddenly be pointing to a different object, which is pretty confusing.

-Boris
0
Boris
8/14/2018 4:25:34 PM
On 8/9/18 11:44 PM, James Stortz wrote:
> What if I have JSObjects that are already rooted? (Either as global
> variables, or properties of rooted objects) And I have references to those
> JSObject* pointers in my backend?

You must not store JSObject* directly, because that doesn't play nice 
with the GC moving objects.

If you're storing JSObject** pointing into a rooted thing, that might be 
OK as long as you don't write to it, I think.  So a "JSObject* const*", 
basically.

> Do I need to change those JSObject* pointers to JS::Heap<JSObject*> in that
> case? I always assumed that would be the case, but does it affect those
> rooted JSObjects? Are they still the same object? If so, how do I convert
> Heap<T> back to Rooted<T>?

I'm not quite sure what you're asking here... :(

-Boris
0
Boris
8/14/2018 6:14:48 PM
On 8/10/18 5:34 AM, James Stortz wrote:
> So, here's an actual example to clarify. Please tell me if this simple
> concept is correct.

What you have here will keep the thing stored in the "obj" member of a 
FrontEnd struct alive as long as that FrontEnd struct is alive, I 
believe.  That's assuming there is no API to mutate the value of that 
member other than the FrontEnd constructor.

Given that, deleting the FrontEnd in the object's finalizer doesn't 
really make sense: the FrontEnd will keep the object alive as long as 
the FrontEnd is alive, as far as I can tell.

What is the desired lifetime relationship here between the FrontEnd, the 
BackEnd, and the JSObject that the FrontEnd points to?

-Boris
0
Boris
8/14/2018 6:19:26 PM
Miles, well I'm not entirely sure my example is 100% correct. I was hoping
to get some answers as well. (I also made a typo with `auto *` D:)

I went digging through some Mozilla/JSAPI code on the DXR to see how it is
used, and put that together with some info from StackOverflow.

I think I can answer your question about Heap/Handle. I believe Handles are
always for Rooted Objects, which are on the stack. Heap and Stack Objects
are handled differently, so you would have to root it before use.

Again, I'm trying to learn these things myself. These are good questions.
0
James
8/14/2018 6:31:54 PM
On Tue, Aug 14, 2018 at 2:19 PM, Boris Zbarsky <bzbarsky@mit.edu> wrote:

>
> Given that, deleting the FrontEnd in the object's finalizer doesn't really
> make sense: the FrontEnd will keep the object alive as long as the FrontEnd
> is alive, as far as I can tell.
>
> What is the desired lifetime relationship here between the FrontEnd, the
> BackEnd, and the JSObject that the FrontEnd points to?
>
>
Ah, thank you for catching that! Ok, so the JSObject has a C++ Object that
it mirrors, as a high-level interface. The desired relationship is that C++
or JS can work on their respective counterpart and have changes reflected
accordingly. (FrontEnd and BackEnd are simply wrappers that contain
pointers to them.)

The problem is when the JS Object goes out of scope, it never really goes
out of scope! They all get GC'd at Context Destruction. These JSObjects are
created in my JS API with `new`, so in C++ that would be normal to keep
them alive until `delete`, but in JS, we don't want this!

I can't think of any elegant solutions off the top of my head. I don't
think storing raw JSObject* pointers in the C++ is viable. Neither is
reworking away the C++ mirror entirely. It would be great if there was a
distinction between Heap/Rooted objects that I could test for during some
GC trace/hook, but there's not a better place to delete the Heap<JSObject*>
wrapper, is there? Is there a way I can check for when a JSObject
"would've" gone out of scope? (such as: upon RootedObject normally going
out of scope)

If anybody knows of a better/common methodology, I would gladly take some
advice. I don't want to impose otherwise, I'm sure I can figure something
out. (Seems like, at least in my program, I can do some kind of a check in
a GC trace somewhere, to see if the JSObject was actually used by my API.)

Again, I really appreciate the replies, everyone! Thanks for catching that,
Boris.
Thanks,
James
0
James
8/15/2018 11:08:55 PM
Please can I clarify how tracing interacts with garbage collection?

Hi, thanks for all the help and patience so far but I'm struggling with thi=
s somewhat as I don't really understand how tracing works. There is very li=
ttle information on it in the JSAPI reference. I would be really grateful i=
f someone could give some more clarification so I can try to understand how=
 to use it properly.

In my embedding that currently uses JS 1.8.5 I have some classes defined us=
ing JSClass (e.g. I have Widget, WidgetItem and Window classes). Each of th=
ese has a constructor and this stores a structure in the private data field=
 of the object using JS_SetPrivate. I define a JSfinalizeOp for the class w=
here I then return the structure for this memory.

In my embedding this structure stores the pointer back to the JSObject. For=
 example the structure I store for my 'Widget' class looks something like

typedef struct db_js_widget_struct {

    JSObject         *jsobject;      /* The Widget object in javascript */
    JSContext        *context;       /* The context in javascript */
    int               id;            /* The id of the widget */
    int               type;          /* The type of widget */
    int               bg_col;        /* The background colour */
....
} DB_JS_WIDGET;

This enables me to get back to the object.

When using JS 1.8.5 normally when the object goes out of scope the finalize=
Op will be called when GC is done and my structure will be returned.

However, in some special cases I wanted to ensure that the object was not g=
arbage collected so I rooted the object using JS_AddObjectRoot.
When I had finished with the object I could then call JS_RemoveObjectRoot a=
nd then when GC was done the object would not be rooted anymore and would b=
e freed.

So I had two different use cases. The 'normal' case where I wanted the obje=
ct to be garbage collected when it went out of scope and the 'special' case=
 where I wanted to protect the object from being garbage collected by rooti=
ng. However in both cases I wanted to store the (JSObject *) pointer.

Now moving to JS 59 from my understanding I cannot store the (JSObject *) p=
ointer directly on the structure as GC is now done with a moving GC so (JSO=
bject *) pointers can 'move'. Correct?
So I have two options.
1. use JS::PersistentRootedObject
2. use JS::Heap<JSObject *> and trace it.

If I use JS::PersistentRootedObject then I think that my object will *alway=
s* be rooted so will *never* go 'out of scope' and be garbage collected. i.=
e. I cannot use this for my 'normal' use case.

So I think I have to use JS::Heap<JSObject *> and trace it.
My structure now becomes

typedef struct db_js_widget_struct {

    JS::Heap<JSObject *> jsobject;      /* The Widget object in javascript =
*/
    JSContext        *context;       /* The context in javascript */
    int               id;            /* The id of the widget */
    int               type;          /* The type of widget */
    int               bg_col;        /* The background colour */
....
} DB_JS_WIDGET;

and on my JSClass I now define a traceOp which looks something like

   static void dj_widget_trace(JSTracer *trc, JSObject *obj)
/* =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D
 *
 * Tracing for JS::Heap<T> GC things for Widget
 */
{
    DB_JS_WIDGET     *js_widget;

/* Get the private data for the widget and return it. */
    js_widget =3D (DB_JS_WIDGET *) JS_GetPrivate(obj);

    JS::TraceEdge(trc, &js_widget->jsobject, "widget");
}

This is where I don't understand the process.
What does calling JS::TraceEdge do? Does it
1. Ensure that jsobject is kept up to date to refer to the correct (JSObjec=
t *) pointer
or
2. Root the object to stop GC on it.

Or perhaps it is both?

Basically, how can I do my two 'normal' and 'special' use cases using traci=
ng? Can I somehow store the JSObject pointer in a structure but still allow=
 it to go out of scope as normal for my 'normal' case and also prevent GC i=
n some circumstances for my 'special' case. Is it possible?

I *think* this is the same question that James is asking...

I hope this makes sense. Apologies if I haven't explained it clearly.
Many thanks in advance for any information someone might be able to give.

Miles
0
Miles
8/16/2018 2:33:40 PM
On Thu, Aug 16, 2018 at 10:33 AM, Miles <miles.thornton@arup.com> wrote:

> What does calling JS::TraceEdge do? Does it
> 1. Ensure that jsobject is kept up to date to refer to the correct
> (JSObject *) pointer
> or
> 2. Root the object to stop GC on it.
>
> Or perhaps it is both?
>
> Basically, how can I do my two 'normal' and 'special' use cases using
> tracing? Can I somehow store the JSObject pointer in a structure but still
> allow it to go out of scope as normal for my 'normal' case and also prevent
> GC in some circumstances for my 'special' case. Is it possible?
>
> I *think* this is the same question that James is asking...
>


Miles, did anybody help answer this? (Sometimes I get these emails delayed,
but I thought I'd try to help in some small way, since I was able to get
mine working, as far as I can tell.)

Yes, I have the same questions, and I believe our understanding is correct.
My advice is this: "You have to know when it is ok to keep the [heap]
object alive." For example, say if you had an event/request object, and
wanted to keep it alive so the backend can do work, and then return control
to the JS. Upon returning control to JS, you would be ok to free it from
the heap/destroy private, as once the object is rooted again [on the stack]
it would be treated as per usual, and able to GC.

If your special case is as straight-forward as that, then that'll work. If
not, you should try to work it out so you can know exactly when it needs to
be alive. My case was more complicated, but I was able to do that, and I
tested by calling `JS_GC(cx)` in the middle of my program, and it did
indeed GC those objects.

I like how you are using `TraceOp` on your `JSClass`, but unfortunately I
don't know how that works. I think it just adds/removes the tracer
automatically, whereas I used
`JS_AddExtraGCRootsTracer`/`JS_RemoveExtraRootsTracer`, as well as a
`FinalizeOp`. That means, elsewhere in my program, I would NULL the
`JS::Heap<>` and remove it's tracer, and that allowed FinalizeOp to delete
the private, and complete GC as per usual.

I think your approach would be cleaner, but similar using `TraceOp` ...?

Good luck and Regards,
James
0
James
8/18/2018 5:16:48 PM
On 8/15/18 7:08 PM, James Stortz wrote:
> Ok, so the JSObject has a C++ Object that
> it mirrors, as a high-level interface. The desired relationship is that C++
> or JS can work on their respective counterpart and have changes reflected
> accordingly.

So you want the JS object to keep the C++ object alive as needed _and_ 
the C++ object to keep the JS object alive?  But for both to go away 
once both become unreachable (in JS and C++ respectively)?

> I can't think of any elegant solutions off the top of my head.

Well, there's what Firefox does, which is having C++ objects trace their 
JS objects as needed, etc, but breaking the cycles when the C++ object 
is conceptually dead.

> I don't think storing raw JSObject* pointers in the C++ is viable.

That's correct.

> It would be great if there was a
> distinction between Heap/Rooted objects that I could test for during some
> GC trace/hook, but there's not a better place to delete the Heap<JSObject*>
> wrapper, is there?

I'm not sure what you mean here.

> Is there a way I can check for when a JSObject "would've" gone out of scope?

Sure, you can run a GC marking pass and see what color it ended up being.

-Boris
0
Boris
8/27/2018 5:07:12 PM
On 8/16/18 10:33 AM, Miles wrote:
> Please can I clarify how tracing interacts with garbage collection?

Tracing is how you tell the GC about edges in the object (well, gcthing) 
graph.

> In my embedding this structure stores the pointer back to the JSObject.

That might be fine, depending.  If you don't need the structure to keep 
the JSObject alive, then you can probably just store a raw pointer here 
and update it when the class objectMovedOp is called.  See documentation 
at 
https://searchfox.org/mozilla-central/rev/e126996d9b0a3d7653de205898517f4f5b632e7f/js/public/Class.h#718-733

> and on my JSClass I now define a traceOp which looks something like

This seems like it would work ok too, to me.

> What does calling JS::TraceEdge do? Does it
> 1. Ensure that jsobject is kept up to date to refer to the correct (JSObject *) pointer

Yes.

> or
> 2. Root the object to stop GC on it.

Yes, if it gets called.

In your case, it'll only get called if the object is getting traced 
anyway (since you're doing it in the class trace hook), so the object is 
known to be alive and only your #1 is really happening.

> Basically, how can I do my two 'normal' and 'special' use cases using tracing? Can I somehow store the JSObject pointer in a structure but still allow it to go out of scope as normal for my 'normal' case and also prevent GC in some circumstances for my 'special' case.

For your 'special' case, rooting things as you used to seems correct.

-Boris
0
Boris
8/27/2018 5:13:34 PM
On 8/27/18 1:07 PM, Boris Zbarsky wrote:
>> I don't think storing raw JSObject* pointers in the C++ is viable.
> 
> That's correct.

Well, unless you're OK with the C++ struct not keeping the JS object 
alive.  In which case, as I said in response to Miles, you can use the 
objectMovedOp on the JSClass to update weak raw pointers as needed.  See 
documentation at 
https://searchfox.org/mozilla-central/rev/e126996d9b0a3d7653de205898517f4f5b632e7f/js/public/Class.h#718-733

-Boris
0
Boris
8/27/2018 5:14:33 PM
On Mon, Aug 27, 2018 at 1:07 PM, Boris Zbarsky <bzbarsky@mit.edu> wrote:

>
> [a]
> Well, there's what Firefox does, which is having C++ objects trace their
> JS objects as needed, etc, but breaking the cycles when the C++ object is
> conceptually dead.
> [b]
> Sure, you can run a GC marking pass and see what color it ended up being.
> [c] You can use the objectMovedOp on the JSClass to update weak raw
> pointers as needed.
>
>
Ah, that's great! Thanks, Boris. I ended up using the first suggestion, the
way firefox does, and it works nicely with the design pattern.
0
James
8/27/2018 10:19:06 PM
Reply: