What is PLDHashTable used for?

Hi everyone,


While doing some random testing of Firefox, I encountered the assertion failure MOZ_ASSERT(IsIdle(oldState) || IsRead(oldState)) in Checker::StartReadOp() in xpcom/glue/pldhash.h.  It seems that this failure is the result of concurrent readers and writers to the same instance of PLDHashTable.


It seemed odd to me that there would be code to check for concurrent readers and writers but no code to prevent such concurrent accesses, so I decided to add some basic synchronization with a pthread_rwlock_t.  While this approach did fix the assertion failure, it also slowed Firefox down to a crawl.  Is this the reason that PLDHashTable is currently left unsynchronized?  What is PLDHashTable being used for that requires so many concurrent accesses?


Thank you,

Matt Morehouse
0
Matt
11/18/2015 11:22:52 PM
mozilla.dev.tech.xpcom 1345 articles. 0 followers. Post Follow

5 Replies
506 Views

Similar Articles

[PageSpeed] 10

On 11/18/15 6:22 PM, Matt Morehouse wrote:
> While doing some random testing of Firefox, I encountered the assertion failure MOZ_ASSERT(IsIdle(oldState) || IsRead(oldState)) in Checker::StartReadOp() in xpcom/glue/pldhash.h.  It seems that this failure is the result of concurrent readers and writers to the same instance of PLDHashTable.

Or, possibly, reentry.  What did the stack look like?

> It seemed odd to me that there would be code to check for concurrent readers and writers but no code to prevent such concurrent accesses

PLDHashTable is single-threaded.  It's not meant to be used in any sort 
of concurrent way at all.  This code is checking for reentry, really.

> so I decided to add some basic synchronization with a pthread_rwlock_t.

Note that in the reentry case this would probably deadlock....

> While this approach did fix the assertion failure

Sounds like you were actually getting concurrent access, then.  I'd 
_really_ like to see the stack at that assertion failure.

> Is this the reason that PLDHashTable is currently left unsynchronized?

Well, that, and that it's not meant to be used in any sort of concurrent 
manner that involves thread.

> What is PLDHashTable being used for that requires so many concurrent accesses?

It's the main hashtable implementation used outside the JS engine.  Used 
for all sorts of stuff in the DOM, layout, and so forth.

-Boris
0
Boris
11/19/2015 3:54:45 AM
--Boundary_(ID_13sBidO7vMJLGDK4uY58SA)
Content-type: text/plain; CHARSET=US-ASCII
Content-transfer-encoding: 7BIT

Hi Boris,

The backtrace is attached.  The assertion failure occurs at #6.  Doesn't look like reentry to me.

- Matt

________________________________________
From: dev-tech-xpcom <dev-tech-xpcom-bounces+mm=cs.wisc.edu@lists.mozilla.org> on behalf of Boris Zbarsky <bzbarsky@mit.edu>
Sent: Wednesday, November 18, 2015 9:54 PM
To: dev-tech-xpcom@lists.mozilla.org
Subject: Re: What is PLDHashTable used for?

On 11/18/15 6:22 PM, Matt Morehouse wrote:
> While doing some random testing of Firefox, I encountered the assertion failure MOZ_ASSERT(IsIdle(oldState) || IsRead(oldState)) in Checker::StartReadOp() in xpcom/glue/pldhash.h.  It seems that this failure is the result of concurrent readers and writers to the same instance of PLDHashTable.

Or, possibly, reentry.  What did the stack look like?

> It seemed odd to me that there would be code to check for concurrent readers and writers but no code to prevent such concurrent accesses

PLDHashTable is single-threaded.  It's not meant to be used in any sort
of concurrent way at all.  This code is checking for reentry, really.

> so I decided to add some basic synchronization with a pthread_rwlock_t.

Note that in the reentry case this would probably deadlock....

> While this approach did fix the assertion failure

Sounds like you were actually getting concurrent access, then.  I'd
_really_ like to see the stack at that assertion failure.

> Is this the reason that PLDHashTable is currently left unsynchronized?

Well, that, and that it's not meant to be used in any sort of concurrent
manner that involves thread.

> What is PLDHashTable being used for that requires so many concurrent accesses?

It's the main hashtable implementation used outside the JS engine.  Used
for all sorts of stuff in the DOM, layout, and so forth.

-Boris
_______________________________________________
dev-tech-xpcom mailing list
dev-tech-xpcom@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-tech-xpcom

--Boundary_(ID_13sBidO7vMJLGDK4uY58SA)
Content-type: text/plain; name=pldhash-stack.txt
Content-transfer-encoding: 7BIT
Content-disposition: attachment; filename=pldhash-stack.txt; size=6127;
 creation-date="Thu, 19 Nov 2015 14:27:45 GMT";
 modification-date="Thu, 19 Nov 2015 14:27:45 GMT"
Content-description: pldhash-stack.txt

#0  0x00007f52ba455f3d in nanosleep () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x00007f52ba455dd4 in sleep () from /lib/x86_64-linux-gnu/libc.so.6
#2  0x00007f52b4006425 in ah_crap_handler (signum=11)
    at /home/matt/Downloads/firefox-42.0/toolkit/xre/nsSigHandlers.cpp:103
#3  0x00007f52b3fe9312 in nsProfileLock::FatalSignalHandler (signo=11, info=0x7fff8150cef0, context=0x7fff8150cdc0)
    at /home/matt/Downloads/firefox-42.0/toolkit/profile/nsProfileLock.cpp:195
#4  0x00007f52b4e67a39 in AsmJSFaultHandler (signum=11, info=0x7fff8150cef0, context=0x7fff8150cdc0)
    at /home/matt/Downloads/firefox-42.0/js/src/asmjs/AsmJSSignalHandlers.cpp:1135
#5  <signal handler called>
#6  0x00007f52b00e4c7a in Checker::StartReadOp (this=0x7f528b5a0073)
    at /home/matt/Downloads/firefox-42.0/xpcom/build/../glue/pldhash.h:128
#7  0x00007f52b00e4806 in PLDHashTable::Iterator::Iterator (this=0x7fff8150d2e0, aTable=0x7f528b5a004b)
    at /home/matt/Downloads/firefox-42.0/xpcom/glue/pldhash.cpp:848
#8  0x00007f52b0d897a6 in Native2WrappedNativeMap::Iter (this=0x7f529a0d4b30)
    at /home/matt/Downloads/firefox-42.0/js/xpconnect/src/XPCMaps.h:152
#9  0x00007f52b0d7d259 in XPCWrappedNativeScope::TraceWrappedNativesInAllScopes (trc=0x7fff8150d438, 
    rt=0x7f52a064e000) at /home/matt/Downloads/firefox-42.0/js/xpconnect/src/XPCWrappedNativeScope.cpp:471
#10 0x00007f52b0d1ddf5 in XPCJSRuntime::TraceAdditionalNativeGrayRoots (this=0x7f52a064e000, trc=0x7fff8150d438)
    at /home/matt/Downloads/firefox-42.0/js/xpconnect/src/XPCJSRuntime.cpp:565
#11 0x00007f52affd3159 in mozilla::CycleCollectedJSRuntime::TraceNativeGrayRoots (this=0x7f52a064e000, 
    aTracer=0x7fff8150d438) at /home/matt/Downloads/firefox-42.0/xpcom/base/CycleCollectedJSRuntime.cpp:802
#12 0x00007f52affd2e0d in mozilla::CycleCollectedJSRuntime::TraceGrayJS (aTracer=0x7fff8150d438, 
    aData=0x7f52a064e000) at /home/matt/Downloads/firefox-42.0/xpcom/base/CycleCollectedJSRuntime.cpp:683
#13 0x00007f52b524230a in js::gc::GCRuntime::bufferGrayRoots (this=0x7f52a06743d8)
    at /home/matt/Downloads/firefox-42.0/js/src/gc/RootMarking.cpp:442
#14 0x00007f52b5647823 in js::gc::GCRuntime::beginMarkPhase (this=0x7f52a06743d8, 
    reason=JS::gcreason::SET_NEW_DOCUMENT) at /home/matt/Downloads/firefox-42.0/js/src/jsgc.cpp:3943
#15 0x00007f52b564e5f7 in js::gc::GCRuntime::incrementalCollectSlice (this=0x7f52a06743d8, budget=..., 
    reason=JS::gcreason::SET_NEW_DOCUMENT) at /home/matt/Downloads/firefox-42.0/js/src/jsgc.cpp:5818
#16 0x00007f52b564ef45 in js::gc::GCRuntime::gcCycle (this=0x7f52a06743d8, incremental=true, budget=..., 
    reason=JS::gcreason::SET_NEW_DOCUMENT) at /home/matt/Downloads/firefox-42.0/js/src/jsgc.cpp:6063
#17 0x00007f52b564f681 in js::gc::GCRuntime::collect (this=0x7f52a06743d8, incremental=true, budget=..., 
    reason=JS::gcreason::SET_NEW_DOCUMENT) at /home/matt/Downloads/firefox-42.0/js/src/jsgc.cpp:6177
#18 0x00007f52b564faea in js::gc::GCRuntime::startGC (this=0x7f52a06743d8, gckind=GC_NORMAL, 
    reason=JS::gcreason::SET_NEW_DOCUMENT, millis=0) at /home/matt/Downloads/firefox-42.0/js/src/jsgc.cpp:6246
#19 0x00007f52b5652836 in JS::StartIncrementalGC (rt=0x7f52a0674000, gckind=GC_NORMAL, 
    reason=JS::gcreason::SET_NEW_DOCUMENT, millis=0) at /home/matt/Downloads/firefox-42.0/js/src/jsgc.cpp:7096
#20 0x00007f52b16d8556 in nsJSContext::GarbageCollectNow (aReason=JS::gcreason::SET_NEW_DOCUMENT, 
    aIncremental=nsJSContext::IncrementalGC, aShrinking=nsJSContext::NonShrinkingGC, aSliceMillis=0)
    at /home/matt/Downloads/firefox-42.0/dom/base/nsJSEnvironment.cpp:1332
#21 0x00007f52b16d96c5 in GCTimerFired (aTimer=0x7f529a0cdec0, aClosure=0x2a)
    at /home/matt/Downloads/firefox-42.0/dom/base/nsJSEnvironment.cpp:1832
#22 0x00007f52b008a97e in nsTimerImpl::Fire (this=0x7f529a0cdec0)
    at /home/matt/Downloads/firefox-42.0/xpcom/threads/nsTimerImpl.cpp:437
#23 0x00007f52b0081567 in nsTimerEvent::Run (this=0x7f528d35f0b0)
    at /home/matt/Downloads/firefox-42.0/xpcom/threads/TimerThread.cpp:267
#24 0x00007f52b008611a in nsThread::ProcessNextEvent (this=0x7f52ba13d7c0, aMayWait=false, aResult=0x7fff8150daef)
    at /home/matt/Downloads/firefox-42.0/xpcom/threads/nsThread.cpp:867
#25 0x00007f52b00e2bec in NS_ProcessNextEvent (aThread=0x7f52ba13d7c0, aMayWait=false)
    at /home/matt/Downloads/firefox-42.0/xpcom/glue/nsThreadUtils.cpp:277
#26 0x00007f52b05cd53c in mozilla::ipc::MessagePump::Run (this=0x7f52a6b57e00, aDelegate=0x7f52ba172da0)
    at /home/matt/Downloads/firefox-42.0/ipc/glue/MessagePump.cpp:95
#27 0x00007f52b055d965 in MessageLoop::RunInternal (this=0x7f52ba172da0)
    at /home/matt/Downloads/firefox-42.0/ipc/chromium/src/base/message_loop.cc:234
#28 0x00007f52b055d8fa in MessageLoop::RunHandler (this=0x7f52ba172da0)
    at /home/matt/Downloads/firefox-42.0/ipc/chromium/src/base/message_loop.cc:227
#29 0x00007f52b055d88b in MessageLoop::Run (this=0x7f52ba172da0)
    at /home/matt/Downloads/firefox-42.0/ipc/chromium/src/base/message_loop.cc:201
#30 0x00007f52b3146dec in nsBaseAppShell::Run (this=0x7f529d010480)
    at /home/matt/Downloads/firefox-42.0/widget/nsBaseAppShell.cpp:165
#31 0x00007f52b3f5c17b in nsAppStartup::Run (this=0x7f529d026920)
    at /home/matt/Downloads/firefox-42.0/toolkit/components/startup/nsAppStartup.cpp:280
#32 0x00007f52b3ffd34b in XREMain::XRE_mainRun (this=0x7fff8150dec0)
    at /home/matt/Downloads/firefox-42.0/toolkit/xre/nsAppRunner.cpp:4287
#33 0x00007f52b3ffd71c in XREMain::XRE_main (this=0x7fff8150dec0, argc=4, argv=0x7fff8150f3c8, 
    aAppData=0x7fff8150e0e0) at /home/matt/Downloads/firefox-42.0/toolkit/xre/nsAppRunner.cpp:4380
#34 0x00007f52b3ffd9f9 in XRE_main (argc=4, argv=0x7fff8150f3c8, aAppData=0x7fff8150e0e0, aFlags=0)
    at /home/matt/Downloads/firefox-42.0/toolkit/xre/nsAppRunner.cpp:4482
#35 0x00000000004051a4 in do_main (argc=4, argv=0x7fff8150f3c8, xreDirectory=0x7f52ba154840)
    at /home/matt/Downloads/firefox-42.0/browser/app/nsBrowserApp.cpp:212
#36 0x00000000004055af in main (argc=4, argv=0x7fff8150f3c8)
    at /home/matt/Downloads/firefox-42.0/browser/app/nsBrowserApp.cpp:399

--Boundary_(ID_13sBidO7vMJLGDK4uY58SA)--
0
Matt
11/19/2015 2:27:49 PM
On 11/19/15 9:27 AM, Matt Morehouse wrote:
> The backtrace is attached.  The assertion failure occurs at #6.  Doesn't look like reentry to me.

Indeed, it doesn't.

This particular hashtable (XPCWrappedNativeScope::mWrappedNativeMap) 
should _certainly_ not be touched from multiple threads.  If that's 
happening, something is very wrong...

-Boris
0
Boris
11/19/2015 2:45:15 PM
On Thursday, November 19, 2015 at 4:28:31 PM UTC+2, Matt Morehouse wrote:
> Hi Boris,
> 
> The backtrace is attached.  The assertion failure occurs at #6.  Doesn't look like reentry to me.



And that looks like main thread which is accessing the hashtable, which is expected.
Do you have steps to reproduce? Did you have some addons installed when testing?
0
olli
11/19/2015 2:58:29 PM
Hi again everyone,

I just figured out the real issue, and it was completely my fault.

My testing technique involved modifying the pthreads library to induce unusual thread scheduling patterns.  One modification I made was to keep track of all running threads in a dynamically-allocated data structure.  Turns out concurrent calls to malloc between Firefox and my modified library were causing some memory chunks to be assigned to both Firefox and my data structure, thereby causing data corruption.  After switching to a statically-allocated data structure, the assertion failure went away.

Thank you for your help Boris and Olli, and sorry for wasting your time.

- Matt 

________________________________________
From: dev-tech-xpcom <dev-tech-xpcom-bounces+mm=cs.wisc.edu@lists.mozilla.org> on behalf of olli.pettay@gmail.com <olli.pettay@gmail.com>
Sent: Thursday, November 19, 2015 8:58 AM
To: dev-tech-xpcom@lists.mozilla.org
Subject: Re: What is PLDHashTable used for?

On Thursday, November 19, 2015 at 4:28:31 PM UTC+2, Matt Morehouse wrote:
> Hi Boris,
>
> The backtrace is attached.  The assertion failure occurs at #6.  Doesn't look like reentry to me.



And that looks like main thread which is accessing the hashtable, which is expected.
Do you have steps to reproduce? Did you have some addons installed when testing?
_______________________________________________
dev-tech-xpcom mailing list
dev-tech-xpcom@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-tech-xpcom
0
Matt
11/19/2015 5:09:24 PM
Reply:

Similar Artilces:

Purpose of dev-tech-xpcom
What's the purpose of this list? I used to think it was about XPCOM itself, basically for questions about the code in xpcom/**. However, that doesn't seem to be how everyone else uses it. In fact, it seems to be about all coding questions that are somewhat Firefox- or XULRunner-related. Basically what the platform or extension lists are supposed to be. Should we, therefore, delete this list/newsgroup? As it is used, it is redundant. -christian -- All the world's a stage, And all the men and women merely players: They have their exits and their entrances...

superreview denied: [Bug 182758] freebl PRNG hashes netstat and /dev/urandom data rather than just using /dev/urandom : [Attachment 235522] On Solaris, use only /dev/urandom if it is available. If not
Wan-Teh Chang <wtchang@redhat.com> has denied Julien Pierre <julien.pierre.bugs@sun.com>'s request for superreview: Bug 182758: freebl PRNG hashes netstat and /dev/urandom data rather than just using /dev/urandom https://bugzilla.mozilla.org/show_bug.cgi?id=182758 Attachment 235522: On Solaris, use only /dev/urandom if it is available. If not, use libkstat https://bugzilla.mozilla.org/attachment.cgi?id=235522&action=edit ------- Additional Comments from Wan-Teh Chang <wtchang@redhat.com> These are just some minor problems. But since there are many, I wa...

superreview requested: [Bug 182758] freebl PRNG hashes netstat and /dev/urandom data rather than just using /dev/urandom : [Attachment 235522] On Solaris, use only /dev/urandom if it is available. If
Julien Pierre <julien.pierre.bugs@sun.com> has asked Wan-Teh Chang <wtchang@redhat.com> for superreview: Bug 182758: freebl PRNG hashes netstat and /dev/urandom data rather than just using /dev/urandom https://bugzilla.mozilla.org/show_bug.cgi?id=182758 Attachment 235522: On Solaris, use only /dev/urandom if it is available. If not, use libkstat https://bugzilla.mozilla.org/attachment.cgi?id=235522&action=edit ------- Additional Comments from Julien Pierre <julien.pierre.bugs@sun.com> In the libkstat case, I am feeding all kernel statistics to the PRNG, 4 KB ...

XPCOM that use another XPCOM
Hi all, I have the following problem: I want to create a standalone application that uses an XPCOM module "A" written in C++ (a library). I want to create plugins for this application (.xpi package) that install another XPCOM module "B" that use the A XPCom library. I think I doesn't need to cross the interface and to write javascript code. How can I do this in a cross-platform manner? Does mozilla support the dynamic linking? Does it exist some pages of documentation about it? Thank you for your help, Lorenzo On 23 Mar, 14:48, Lorenzo <nos...@pl...

Merging dev-tech-layout into dev-platform
Today I found out that I had missed a post from David Baron on dev-tech-layout, because I had no idea that this list exists. I think dev-platform is a better place to have the conversation related to the layout module (people are already having discussions about other modules over there). Does anybody have any objections? Cheers, Ehsan ...

Merging dev-tech-layout into dev-platform
Today I found out that I had missed a post from David Baron on dev-tech-layout, because I had no idea that this list exists. I think dev-platform is a better place to have the conversation related to the layout module (people are already having discussions about other modules over there). Does anybody have any objections? Cheers, Ehsan ...

Using Using
I have just completed reading a chapter in my book on performance. The books says to speed up performance on your web site you should use the using statement when opening connections. My question is how do you catch errors if you are using this as apposed to a try catch block.thanksBryan  Why would you use it as an alternative to a try/catch statement. Couldn't you just do something like  public void UsingSomeMethod() { try { SomeMethod(); } catch(SomeException e) { /// do something useful }}public void SomeMethod() { using (Something) { ...

When to use ( and when to use ((?
I'm just not getting it. When do you use ( in a statement, when do you need to use ((, and is the space after either or both mandatory? For example, I coded this statement: if (( $_ eq $bad_guys_ip )) { more_stuff_here }; Why the two (( in an if? And why does it seem to not work when I miss the space after the ((? Thanx! Mark me as... /Corn-fused|Dense/ > I'm just not getting it. >=20 > When do you use ( in a statement, when do you need to use ((, and is > the space after either or both mandatory? >=20 > For example, I coded this sta...

How Can I use the compliers used under Dev-C++ in C++ builder?
How Can I use the compliers (for instance GCC, ...) used under Dev-C++ in C++ builder? In my program,I used boost lambda expression under Dev-C++ due to the fact that C++ builder 2009 do not support such expression. But Dev-C++ do not support RAD(rapid application development). It is hard to establish the common graphic GUI under windows.So I want to use the compliers of Dev-C++ in C++ builder. And I do not know how to manage it. Where can I get the related materials or advices? Can any one with kindness help me? Il Tue, 11 Aug 2009 23:29:59 -0700, lserjt lserjt <> ha scritto:...

can i use XPCOM emit signal, and use javascript in extension to handle event?
Hi list, i try to write a C++ XPCOM for receive Windows message, and pass this message from XPCOM to javascript in extension to handle event. does any XPCOM API can do this thing? Thanks, Yuren ...

How to use the arguments to use() in the package being used
Howdy,=20 The subject says it all believe it or not :) What I'm trying to figure out is how to pass an argument=20 (pragma I believe is the proper term) to use() and do=20 sonethign in the package based on it. I've looked at CGI.pm source but can't seem to track it down.=20 (Similar idea as to CGIs -oldstyle_urls -newstyel_urls) http://search.cpan.org/~lds/CGI.pm-3.04/CGI.pm#PRAGMAS What I'd like to do is something like this: # for old time's sake we'll just use our favorite module use Foo::Monkey qw(:Foo :Bar -doamazingthings); #then in Foo::Monkey...

superreview requested: [Bug 182758] freebl PRNG hashes netstat and /dev/urandom data rather than just using /dev/urandom : [Attachment 235586] Update
Julien Pierre <julien.pierre.bugs@sun.com> has asked Nelson Bolyard <nelson@bolyard.com> for superreview: Bug 182758: freebl PRNG hashes netstat and /dev/urandom data rather than just using /dev/urandom https://bugzilla.mozilla.org/show_bug.cgi?id=182758 Attachment 235586: Update https://bugzilla.mozilla.org/attachment.cgi?id=235586&action=edit ------- Additional Comments from Julien Pierre <julien.pierre.bugs@sun.com> Wan-Teh, This was written from scratch, it wasn't sample code. I switched from assert to PORT_Assert, as well as from malloc/free to ...

superreview requested: [Bug 182758] freebl PRNG hashes netstat and /dev/urandom data rather than just using /dev/urandom : [Attachment 236342] update
Julien Pierre <julien.pierre.bugs@sun.com> has asked Wan-Teh Chang <wtchang@redhat.com> for superreview: Bug 182758: freebl PRNG hashes netstat and /dev/urandom data rather than just using /dev/urandom https://bugzilla.mozilla.org/show_bug.cgi?id=182758 Attachment 236342: update https://bugzilla.mozilla.org/attachment.cgi?id=236342&action=edit ------- Additional Comments from Julien Pierre <julien.pierre.bugs@sun.com> This patch contains several changes : 1) More comments 2) CollectEntropy and RNG_kstat are both changed to return a SECStatus . This is to ea...

superreview granted: [Bug 182758] freebl PRNG hashes netstat and /dev/urandom data rather than just using /dev/urandom : [Attachment 236342] update
Wan-Teh Chang <wtchang@redhat.com> has granted Julien Pierre <julien.pierre.bugs@sun.com>'s request for superreview: Bug 182758: freebl PRNG hashes netstat and /dev/urandom data rather than just using /dev/urandom https://bugzilla.mozilla.org/show_bug.cgi?id=182758 Attachment 236342: update https://bugzilla.mozilla.org/attachment.cgi?id=236342&action=edit ------- Additional Comments from Wan-Teh Chang <wtchang@redhat.com> r=wtc. Please remove the 4 extraneous semicolons after closing curly braces. Just search for "};" in the file and remove ...

Web resources about - What is PLDHashTable used for? - mozilla.dev.tech.xpcom

Resources last updated: 1/2/2016 8:40:04 PM