Commit 284cc3d6 authored by oliver@apple.com's avatar oliver@apple.com

fourthTier: DFG should be able to run on a separate thread

https://bugs.webkit.org/show_bug.cgi?id=112839

Source/JavaScriptCore:

Reviewed by Geoffrey Garen.

This is the final bit of concurrent JITing. The idea is that there is a
single global worklist, and a single global thread, that does all
optimizing compilation. This is the DFG::Worklist. It contains a queue of
DFG::Plans, and a map from CodeBlock* (the baseline code block we're
trying to optimize) to DFG::Plan. If the DFGDriver tries to concurrently
compile something, it puts the Plan on the Worklist. The Worklist's
thread will compile that Plan eventually, and when it's done, it will
signal its completion by (1) notifying anyone waiting for the Worklist to
be done, and (2) forcing the CodeBlock::m_jitExecuteCounter to take slow
path. The next Baseline JIT cti_optimize call will then install all ready
(i.e. compiled) Plans for that VM. Note that (1) is only for the GC and
VM shutdown, which will want to ensure that there aren't any outstanding
async compilations before proceeding. They do so by simply waiting for
all of the plans for the current VM to complete. (2) is the actual way
that code typically gets installed.

This is all very racy by design. For example, just as we try to force the
execute counter to take slow path, the main thread may be setting the
execute counter to some other value. The main thread must set it to
another value because (a) JIT code is constantly incrementing the counter
in a racy way, (b) the cti_optimize slow path will set it to some
large-ish negative value to ensure that cti_optimize isn't called
repeatedly, and (c) OSR exits from previously jettisoned code blocks may
still want to reset the counter values. This "race" is made benign, by
ensuring that while there is an asynchronous compilation, we at worse set
the counter to optimizeAfterWarmUp and never to deferIndefinitely. Hence
if the race happens then the worst case is that we wait another ~1000
counts before installing the optimized code. Another defense is that if
any CodeBlock calls into cti_optimize, then it will check for all ready
plans for the VM - so even if a code block has to wait another ~1000
executions before it calls cti_optimize to do the installation, it may
actually end up being installed sooner because a different code block had
called cti_optimize, potentially for an unrelated reason.

Special care is taken to ensure that installing plans informs the GC
about the increased memory usage, but also ensures that we don't recurse
infinitely - since at start of GC we try to install outstanding plans.
This is done by introducing a new GC deferral mechanism (the DeferGC
block-scoped thingy), which will ensure that GCs don't happen in the
scope but are allowed to happen after. This still leaves the strange
corner case that cti_optimize may install outstanding plans, then GC, and
that GC may jettison the code block that was installed. This, and the
fact that the plan that we took slow path to install could have been a
failed or invalid compile, mean that we have to take special precautions
in cti_optimize.

This patch also fixes a number of small concurrency bugs that I found
when things started running. There are probably more of those bugs still
left to fix. This patch just fixes the ones I know about.

Concurrent compilation is right now only enabled on X86_64 Mac. We need
platforms that are sufficiently CAStastic so that we can do the various
memory fence and CAS tricks that make this safe. We also need a platform
that uses JSVALUE64. And we need pthread_once. So, that pretty much means
just X64_64 for now. Enabling Linux-64_64 should be a breeze, but I'll
leave that up to the Qt and GTK+ ports to do at their discretion.

This is a solid speed-up on SunSpider (8-9%) and V8Spider (16%), our two
main compile-time benchmarks. Most peculiarly, this also appears to
reduce measurement noise, rather than increasing it as you would have
expected. I don't understand that result but I like it anyway. On the
other hand, this is a slight (1%) slow-down on V8v7. I will continue to
investigate this but I think that the results are already good enough
that we should land this as-is. So far, it appears that the slow-down is
due to this breaking the don't-compile-inlineables heuristics. See
investigation in https://bugs.webkit.org/show_bug.cgi?id=116556 and the
bug https://bugs.webkit.org/show_bug.cgi?id=116557.

* JavaScriptCore.xcodeproj/project.pbxproj:
* bytecode/CodeBlock.cpp:
(JSC):
(JSC::CodeBlock::finalizeUnconditionally):
(JSC::CodeBlock::resetStubInternal):
(JSC::CodeBlock::baselineVersion):
(JSC::CodeBlock::hasOptimizedReplacement):
(JSC::CodeBlock::optimizationThresholdScalingFactor):
(JSC::CodeBlock::checkIfOptimizationThresholdReached):
(JSC::CodeBlock::optimizeNextInvocation):
(JSC::CodeBlock::dontOptimizeAnytimeSoon):
(JSC::CodeBlock::optimizeAfterWarmUp):
(JSC::CodeBlock::optimizeAfterLongWarmUp):
(JSC::CodeBlock::optimizeSoon):
(JSC::CodeBlock::forceOptimizationSlowPathConcurrently):
(JSC::CodeBlock::setOptimizationThresholdBasedOnCompilationResult):
(JSC::CodeBlock::updateAllPredictionsAndCountLiveness):
(JSC::CodeBlock::updateAllArrayPredictions):
(JSC::CodeBlock::shouldOptimizeNow):
* bytecode/CodeBlock.h:
(CodeBlock):
(JSC::CodeBlock::jitCompile):
* bytecode/CodeBlockLock.h:
(JSC):
* bytecode/ExecutionCounter.cpp:
(JSC::ExecutionCounter::forceSlowPathConcurrently):
(JSC):
(JSC::ExecutionCounter::setThreshold):
* bytecode/ExecutionCounter.h:
(ExecutionCounter):
* debugger/Debugger.cpp:
(JSC::Debugger::recompileAllJSFunctions):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::injectLazyOperandSpeculation):
(JSC::DFG::ByteCodeParser::getArrayMode):
(JSC::DFG::ByteCodeParser::getArrayModeAndEmitChecks):
* dfg/DFGCommon.h:
(JSC::DFG::enableConcurrentJIT):
(DFG):
* dfg/DFGDriver.cpp:
(JSC::DFG::compile):
* dfg/DFGGraph.cpp:
(JSC::DFG::Graph::Graph):
* dfg/DFGGraph.h:
(Graph):
* dfg/DFGOSREntry.cpp:
(JSC::DFG::prepareOSREntry):
* dfg/DFGOperations.cpp:
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::Plan):
(JSC::DFG::Plan::compileInThread):
(JSC::DFG::Plan::key):
(DFG):
* dfg/DFGPlan.h:
(DFG):
(Plan):
* dfg/DFGWorklist.cpp: Added.
(DFG):
(JSC::DFG::Worklist::Worklist):
(JSC::DFG::Worklist::~Worklist):
(JSC::DFG::Worklist::finishCreation):
(JSC::DFG::Worklist::create):
(JSC::DFG::Worklist::enqueue):
(JSC::DFG::Worklist::compilationState):
(JSC::DFG::Worklist::waitUntilAllPlansForVMAreReady):
(JSC::DFG::Worklist::removeAllReadyPlansForVM):
(JSC::DFG::Worklist::completeAllReadyPlansForVM):
(JSC::DFG::Worklist::completeAllPlansForVM):
(JSC::DFG::Worklist::queueLength):
(JSC::DFG::Worklist::dump):
(JSC::DFG::Worklist::runThread):
(JSC::DFG::Worklist::threadFunction):
(JSC::DFG::initializeGlobalWorklistOnce):
(JSC::DFG::globalWorklist):
* dfg/DFGWorklist.h: Added.
(DFG):
(Worklist):
* heap/CopiedSpaceInlines.h:
(JSC::CopiedSpace::allocateBlock):
* heap/DeferGC.h: Added.
(JSC):
(DeferGC):
(JSC::DeferGC::DeferGC):
(JSC::DeferGC::~DeferGC):
* heap/Heap.cpp:
(JSC::Heap::Heap):
(JSC::Heap::reportExtraMemoryCostSlowCase):
(JSC::Heap::collectAllGarbage):
(JSC::Heap::collect):
(JSC::Heap::collectIfNecessaryOrDefer):
(JSC):
(JSC::Heap::incrementDeferralDepth):
(JSC::Heap::decrementDeferralDepthAndGCIfNeeded):
* heap/Heap.h:
(Heap):
(JSC::Heap::isCollecting):
(JSC):
* heap/MarkedAllocator.cpp:
(JSC::MarkedAllocator::allocateSlowCase):
* jit/JIT.cpp:
(JSC::JIT::privateCompile):
* jit/JIT.h:
* jit/JITStubs.cpp:
(JSC::DEFINE_STUB_FUNCTION):
* llint/LLIntSlowPaths.cpp:
(JSC::LLInt::jitCompileAndSetHeuristics):
(JSC::LLInt::entryOSR):
(JSC::LLInt::LLINT_SLOW_PATH_DECL):
* profiler/ProfilerBytecodes.h:
* runtime/ConcurrentJITLock.h: Added.
(JSC):
* runtime/ExecutionHarness.h:
(JSC::replaceWithDeferredOptimizedCode):
* runtime/JSSegmentedVariableObject.cpp:
(JSC::JSSegmentedVariableObject::findRegisterIndex):
(JSC::JSSegmentedVariableObject::addRegisters):
* runtime/JSSegmentedVariableObject.h:
(JSSegmentedVariableObject):
* runtime/Options.h:
(JSC):
* runtime/Structure.h:
(Structure):
* runtime/StructureInlines.h:
(JSC::Structure::propertyTable):
* runtime/SymbolTable.h:
(SymbolTable):
* runtime/VM.cpp:
(JSC::VM::VM):
(JSC::VM::~VM):
(JSC::VM::prepareToDiscardCode):
(JSC):
(JSC::VM::discardAllCode):
(JSC::VM::releaseExecutableMemory):
* runtime/VM.h:
(DFG):
(VM):

Source/WTF:

Reviewed by Geoffrey Garen.

* wtf/ByteSpinLock.h:
Make it non-copyable. We previously had bugs where we used ByteSpinLock as a locker.
Clearly that's bad.

* wtf/MetaAllocatorHandle.h:
Make it thread-safe ref-counted, since we may now be passing them between the
concurrent JIT thread and the main thread.

* wtf/Vector.h:
(WTF::Vector::takeLast):
I've wanted this method for ages, and now I finally added.

git-svn-id: http://svn.webkit.org/repository/webkit/trunk@153169 268f45cc-cd09-0410-ab3c-d52691b4dbfc
parent 7b2d139b
This diff is collapsed.
......@@ -75,6 +75,7 @@
0F0CD4C415F6B6BB0032F1C0 /* SparseArrayValueMap.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F0CD4C315F6B6B50032F1C0 /* SparseArrayValueMap.cpp */; };
0F0D85B21723455400338210 /* CodeBlockLock.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F0D85B11723455100338210 /* CodeBlockLock.h */; settings = {ATTRIBUTES = (Private, ); }; };
0F0FC45A14BD15F500B81154 /* LLIntCallLinkInfo.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F0FC45814BD15F100B81154 /* LLIntCallLinkInfo.h */; settings = {ATTRIBUTES = (Private, ); }; };
0F136D4D174AD69E0075B354 /* DeferGC.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F136D4B174AD69B0075B354 /* DeferGC.h */; settings = {ATTRIBUTES = (Private, ); }; };
0F13912916771C33009CCB07 /* ProfilerBytecodeSequence.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F13912416771C30009CCB07 /* ProfilerBytecodeSequence.cpp */; };
0F13912A16771C36009CCB07 /* ProfilerBytecodeSequence.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F13912516771C30009CCB07 /* ProfilerBytecodeSequence.h */; settings = {ATTRIBUTES = (Private, ); }; };
0F13912B16771C3A009CCB07 /* ProfilerProfiledBytecodes.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F13912616771C30009CCB07 /* ProfilerProfiledBytecodes.cpp */; };
......@@ -314,6 +315,9 @@
0FD82E86141F3FF100179C94 /* SpeculatedType.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0FD82E84141F3FDA00179C94 /* SpeculatedType.cpp */; };
0FDB2CC9173DA520007B3C1B /* FTLAbbreviatedTypes.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FDB2CC7173DA51E007B3C1B /* FTLAbbreviatedTypes.h */; settings = {ATTRIBUTES = (Private, ); }; };
0FDB2CCA173DA523007B3C1B /* FTLValueFromBlock.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FDB2CC8173DA51E007B3C1B /* FTLValueFromBlock.h */; settings = {ATTRIBUTES = (Private, ); }; };
0FDB2CE7174830A2007B3C1B /* DFGWorklist.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0FDB2CE5174830A2007B3C1B /* DFGWorklist.cpp */; };
0FDB2CE8174830A2007B3C1B /* DFGWorklist.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FDB2CE6174830A2007B3C1B /* DFGWorklist.h */; settings = {ATTRIBUTES = (Private, ); }; };
0FDB2CEA174896C7007B3C1B /* ConcurrentJITLock.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FDB2CE9174896C7007B3C1B /* ConcurrentJITLock.h */; settings = {ATTRIBUTES = (Private, ); }; };
0FDDBFB51666EED800C55FEF /* DFGVariableAccessDataDump.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0FDDBFB21666EED500C55FEF /* DFGVariableAccessDataDump.cpp */; };
0FDDBFB61666EEDA00C55FEF /* DFGVariableAccessDataDump.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FDDBFB31666EED500C55FEF /* DFGVariableAccessDataDump.h */; settings = {ATTRIBUTES = (Private, ); }; };
0FE228ED1436AB2700196C48 /* Options.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FE228EB1436AB2300196C48 /* Options.h */; settings = {ATTRIBUTES = (Private, ); }; };
......@@ -1081,6 +1085,7 @@
0F0CD4C315F6B6B50032F1C0 /* SparseArrayValueMap.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = SparseArrayValueMap.cpp; sourceTree = "<group>"; };
0F0D85B11723455100338210 /* CodeBlockLock.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = CodeBlockLock.h; sourceTree = "<group>"; };
0F0FC45814BD15F100B81154 /* LLIntCallLinkInfo.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = LLIntCallLinkInfo.h; sourceTree = "<group>"; };
0F136D4B174AD69B0075B354 /* DeferGC.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = DeferGC.h; sourceTree = "<group>"; };
0F13912416771C30009CCB07 /* ProfilerBytecodeSequence.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = ProfilerBytecodeSequence.cpp; path = profiler/ProfilerBytecodeSequence.cpp; sourceTree = "<group>"; };
0F13912516771C30009CCB07 /* ProfilerBytecodeSequence.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = ProfilerBytecodeSequence.h; path = profiler/ProfilerBytecodeSequence.h; sourceTree = "<group>"; };
0F13912616771C30009CCB07 /* ProfilerProfiledBytecodes.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = ProfilerProfiledBytecodes.cpp; path = profiler/ProfilerProfiledBytecodes.cpp; sourceTree = "<group>"; };
......@@ -1334,6 +1339,9 @@
0FD82E84141F3FDA00179C94 /* SpeculatedType.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = SpeculatedType.cpp; sourceTree = "<group>"; };
0FDB2CC7173DA51E007B3C1B /* FTLAbbreviatedTypes.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; name = FTLAbbreviatedTypes.h; path = ftl/FTLAbbreviatedTypes.h; sourceTree = "<group>"; };
0FDB2CC8173DA51E007B3C1B /* FTLValueFromBlock.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; name = FTLValueFromBlock.h; path = ftl/FTLValueFromBlock.h; sourceTree = "<group>"; };
0FDB2CE5174830A2007B3C1B /* DFGWorklist.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = DFGWorklist.cpp; path = dfg/DFGWorklist.cpp; sourceTree = "<group>"; };
0FDB2CE6174830A2007B3C1B /* DFGWorklist.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGWorklist.h; path = dfg/DFGWorklist.h; sourceTree = "<group>"; };
0FDB2CE9174896C7007B3C1B /* ConcurrentJITLock.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ConcurrentJITLock.h; sourceTree = "<group>"; };
0FDDBFB21666EED500C55FEF /* DFGVariableAccessDataDump.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = DFGVariableAccessDataDump.cpp; path = dfg/DFGVariableAccessDataDump.cpp; sourceTree = "<group>"; };
0FDDBFB31666EED500C55FEF /* DFGVariableAccessDataDump.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGVariableAccessDataDump.h; path = dfg/DFGVariableAccessDataDump.h; sourceTree = "<group>"; };
0FE228EA1436AB2300196C48 /* Options.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = Options.cpp; sourceTree = "<group>"; };
......@@ -2374,6 +2382,7 @@
C2239D1316262BDD005AC5FD /* CopyVisitor.h */,
C2239D1416262BDD005AC5FD /* CopyVisitorInlines.h */,
C218D13F1655CFD50062BB81 /* CopyWorkList.h */,
0F136D4B174AD69B0075B354 /* DeferGC.h */,
0F2C556D14738F2E00121E4F /* DFGCodeBlocks.cpp */,
0F2C556E14738F2E00121E4F /* DFGCodeBlocks.h */,
BCBE2CAD14E985AA000593AD /* GCAssertions.h */,
......@@ -2681,6 +2690,7 @@
0F15F15D14B7A73A005DE37D /* CommonSlowPaths.h */,
969A09220ED1E09C00F1F681 /* Completion.cpp */,
F5BB2BC5030F772101FCFE1D /* Completion.h */,
0FDB2CE9174896C7007B3C1B /* ConcurrentJITLock.h */,
BCA62DFF0E2826310004F30D /* ConstructData.cpp */,
BC8F3CCF0DAF17BA00577A80 /* ConstructData.h */,
BCD203450E17135E002C7E82 /* DateConstructor.cpp */,
......@@ -3079,6 +3089,8 @@
0F85A31E16AB76AE0077571E /* DFGVariadicFunction.h */,
0FFFC95314EF909500C72532 /* DFGVirtualRegisterAllocationPhase.cpp */,
0FFFC95414EF909500C72532 /* DFGVirtualRegisterAllocationPhase.h */,
0FDB2CE5174830A2007B3C1B /* DFGWorklist.cpp */,
0FDB2CE6174830A2007B3C1B /* DFGWorklist.h */,
);
name = dfg;
sourceTree = "<group>";
......@@ -3533,6 +3545,7 @@
0F5EF91F16878F7D003E5C25 /* JITThunks.h in Headers */,
A76F54A313B28AAB00EF2BCE /* JITWriteBarrier.h in Headers */,
BC18C4160E16F5CD00B34460 /* JSActivation.h in Headers */,
0FDB2CEA174896C7007B3C1B /* ConcurrentJITLock.h in Headers */,
840480131021A1D9008E7F01 /* JSAPIValueWrapper.h in Headers */,
C2CF39C216E15A8100DD69BE /* JSAPIWrapperObject.h in Headers */,
BC18C4170E16F5CD00B34460 /* JSArray.h in Headers */,
......@@ -3655,6 +3668,7 @@
BC18C43F0E16F5CD00B34460 /* Nodes.h in Headers */,
BC18C4410E16F5CD00B34460 /* NumberConstructor.h in Headers */,
BC18C4420E16F5CD00B34460 /* NumberConstructor.lut.h in Headers */,
0FDB2CE8174830A2007B3C1B /* DFGWorklist.h in Headers */,
BC18C4430E16F5CD00B34460 /* NumberObject.h in Headers */,
BC18C4440E16F5CD00B34460 /* NumberPrototype.h in Headers */,
142D3939103E4560007DCB52 /* NumericStrings.h in Headers */,
......@@ -3804,6 +3818,7 @@
86704B8812DBA33700A9FE7B /* YarrParser.h in Headers */,
86704B8A12DBA33700A9FE7B /* YarrPattern.h in Headers */,
86704B4312DB8A8100A9FE7B /* YarrSyntaxChecker.h in Headers */,
0F136D4D174AD69E0075B354 /* DeferGC.h in Headers */,
);
runOnlyForDeploymentPostprocessing = 0;
};
......@@ -4433,6 +4448,7 @@
14469DE8107EC7E700650446 /* PropertySlot.cpp in Sources */,
ADE39FFF16DD144B0003CD4A /* PropertyTable.cpp in Sources */,
1474C33C16AA2D9B0062F01D /* PrototypeMap.cpp in Sources */,
0FDB2CE7174830A2007B3C1B /* DFGWorklist.cpp in Sources */,
0F9332A314CA7DD70085F3C6 /* PutByIdStatus.cpp in Sources */,
0FF60AC316740F8800029779 /* ReduceWhitespace.cpp in Sources */,
14280841107EC0930013E7B2 /* RegExp.cpp in Sources */,
......
......@@ -130,26 +130,8 @@ class CodeBlock : public ThreadSafeRefCounted<CodeBlock>, public UnconditionalFi
{
return specializationFromIsConstruct(m_isConstructor);
}
#if ENABLE(JIT)
CodeBlock* baselineVersion()
{
if (JITCode::isBaselineCode(getJITType()))
return this;
CodeBlock* result = replacement();
ASSERT(result);
while (result->alternative())
result = result->alternative();
ASSERT(result);
ASSERT(JITCode::isBaselineCode(result->getJITType()));
return result;
}
#else
CodeBlock* baselineVersion()
{
return this;
}
#endif
CodeBlock* baselineVersion();
void visitAggregate(SlotVisitor&);
......@@ -312,12 +294,7 @@ class CodeBlock : public ThreadSafeRefCounted<CodeBlock>, public UnconditionalFi
ASSERT(getJITType() == JITCode::BaselineJIT);
return CompilationNotNeeded;
}
#if ENABLE(JIT)
return jitCompileImpl(exec);
#else
UNUSED_PARAM(exec);
return CompilationFailed;
#endif
}
virtual CodeBlock* replacement() = 0;
......@@ -330,20 +307,7 @@ class CodeBlock : public ThreadSafeRefCounted<CodeBlock>, public UnconditionalFi
}
DFG::CapabilityLevel canCompileWithDFGState() { return m_canCompileWithDFGState; }
bool hasOptimizedReplacement()
{
ASSERT(JITCode::isBaselineCode(getJITType()));
bool result = JITCode::isHigherTier(replacement()->getJITType(), getJITType());
#if !ASSERT_DISABLED
if (result)
ASSERT(JITCode::isOptimizingJIT(replacement()->getJITType()));
else {
ASSERT(JITCode::isBaselineCode(replacement()->getJITType()));
ASSERT(replacement() == this);
}
#endif
return result;
}
bool hasOptimizedReplacement();
#else
JITCode::JITType getJITType() const { return JITCode::BaselineJIT; }
#endif
......@@ -878,7 +842,11 @@ class CodeBlock : public ThreadSafeRefCounted<CodeBlock>, public UnconditionalFi
// to trigger optimization if one of those functions becomes hot
// in the baseline code.
void optimizeSoon();
void forceOptimizationSlowPathConcurrently();
void setOptimizationThresholdBasedOnCompilationResult(CompilationResult);
uint32_t osrExitCounter() const { return m_osrExitCounter; }
void countOSRExit() { m_osrExitCounter++; }
......
......@@ -26,18 +26,12 @@
#ifndef CodeBlockLock_h
#define CodeBlockLock_h
#include <wtf/ByteSpinLock.h>
#include <wtf/NoLock.h>
#include "ConcurrentJITLock.h"
namespace JSC {
#if ENABLE(CONCURRENT_JIT)
typedef ByteSpinLock CodeBlockLock;
typedef ByteSpinLocker CodeBlockLocker;
#else
typedef NoLock CodeBlockLock;
typedef NoLockLocker CodeBlockLocker;
#endif
typedef ConcurrentJITLock CodeBlockLock;
typedef ConcurrentJITLocker CodeBlockLocker;
} // namespace JSC
......
......@@ -37,6 +37,11 @@ ExecutionCounter::ExecutionCounter()
reset();
}
void ExecutionCounter::forceSlowPathConcurrently()
{
m_counter = 0;
}
bool ExecutionCounter::checkIfThresholdCrossedAndSet(CodeBlock* codeBlock)
{
if (hasCrossedThreshold(codeBlock))
......@@ -124,11 +129,11 @@ bool ExecutionCounter::setThreshold(CodeBlock* codeBlock)
return false;
}
ASSERT(!hasCrossedThreshold(codeBlock));
ASSERT(!m_activeThreshold || !hasCrossedThreshold(codeBlock));
// Compute the true total count.
double trueTotalCount = count();
// Correct the threshold for current memory usage.
double threshold = applyMemoryUsageHeuristics(m_activeThreshold, codeBlock);
......
......@@ -38,6 +38,7 @@ class CodeBlock;
class ExecutionCounter {
public:
ExecutionCounter();
void forceSlowPathConcurrently(); // If you use this, checkIfThresholdCrossedAndSet() may still return false.
bool checkIfThresholdCrossedAndSet(CodeBlock*);
void setNewThreshold(int32_t threshold, CodeBlock*);
void deferIndefinitely();
......@@ -74,7 +75,6 @@ private:
void reset();
public:
// NB. These are intentionally public because it will be modified from machine code.
// This counter is incremented by the JIT or LLInt. It starts out negative and is
......
......@@ -118,6 +118,8 @@ void Debugger::recompileAllJSFunctions(VM* vm)
ASSERT(!vm->dynamicGlobalObject);
if (vm->dynamicGlobalObject)
return;
vm->prepareToDiscardCode();
Recompiler recompiler(this);
vm->heap.objectSpace().forEachLiveCell(recompiler);
......
......@@ -264,7 +264,7 @@ private:
{
ASSERT(node->op() == GetLocal);
ASSERT(node->codeOrigin.bytecodeIndex == m_currentIndex);
CodeBlockLock locker(m_inlineStackTop->m_profiledBlock->m_lock);
CodeBlockLocker locker(m_inlineStackTop->m_profiledBlock->m_lock);
LazyOperandValueProfileKey key(m_currentIndex, node->local());
SpeculatedType prediction = m_inlineStackTop->m_lazyOperands.prediction(locker, key);
#if DFG_ENABLE(DEBUG_VERBOSE)
......@@ -839,7 +839,7 @@ private:
ArrayMode getArrayMode(ArrayProfile* profile, Array::Action action)
{
CodeBlockLock locker(m_inlineStackTop->m_profiledBlock->m_lock);
CodeBlockLocker locker(m_inlineStackTop->m_profiledBlock->m_lock);
profile->computeUpdatedPrediction(locker, m_inlineStackTop->m_profiledBlock);
return ArrayMode::fromObserved(locker, profile, action, false);
}
......@@ -851,7 +851,7 @@ private:
ArrayMode getArrayModeAndEmitChecks(ArrayProfile* profile, Array::Action action, Node* base)
{
CodeBlockLock locker(m_inlineStackTop->m_profiledBlock->m_lock);
CodeBlockLocker locker(m_inlineStackTop->m_profiledBlock->m_lock);
profile->computeUpdatedPrediction(locker, m_inlineStackTop->m_profiledBlock);
......
......@@ -136,6 +136,15 @@ inline bool validationEnabled()
#endif
}
inline bool enableConcurrentJIT()
{
#if ENABLE(CONCURRENT_JIT)
return Options::enableConcurrentJIT();
#else
return false;
#endif
}
enum SpillRegistersMode { NeedToSpill, DontSpill };
enum NoResultTag { NoResult };
......
......@@ -35,11 +35,13 @@
#include "DFGJITCode.h"
#include "DFGPlan.h"
#include "DFGThunks.h"
#include "DFGWorklist.h"
#include "FTLThunks.h"
#include "JITCode.h"
#include "Operations.h"
#include "Options.h"
#include "SamplingTool.h"
#include <wtf/Atomics.h>
namespace JSC { namespace DFG {
......@@ -69,19 +71,21 @@ static CompilationResult compile(CompileMode compileMode, ExecState* exec, CodeB
return CompilationFailed;
if (logCompilationChanges())
dataLog("DFG compiling ", *codeBlock, ", number of instructions = ", codeBlock->instructionCount(), "\n");
dataLog("DFG(Driver) compiling ", *codeBlock, ", number of instructions = ", codeBlock->instructionCount(), "\n");
VM& vm = exec->vm();
// Make sure that any stubs that the DFG is going to use are initialized. We want to
// make sure that al JIT code generation does finalization on the main thread.
exec->vm().getCTIStub(osrExitGenerationThunkGenerator);
exec->vm().getCTIStub(throwExceptionFromCallSlowPathGenerator);
exec->vm().getCTIStub(linkCallThunkGenerator);
exec->vm().getCTIStub(linkConstructThunkGenerator);
exec->vm().getCTIStub(linkClosureCallThunkGenerator);
exec->vm().getCTIStub(virtualCallThunkGenerator);
exec->vm().getCTIStub(virtualConstructThunkGenerator);
vm.getCTIStub(osrExitGenerationThunkGenerator);
vm.getCTIStub(throwExceptionFromCallSlowPathGenerator);
vm.getCTIStub(linkCallThunkGenerator);
vm.getCTIStub(linkConstructThunkGenerator);
vm.getCTIStub(linkClosureCallThunkGenerator);
vm.getCTIStub(virtualCallThunkGenerator);
vm.getCTIStub(virtualConstructThunkGenerator);
#if ENABLE(FTL_JIT)
exec->vm().getCTIStub(FTL::osrExitGenerationThunkGenerator);
vm.getCTIStub(FTL::osrExitGenerationThunkGenerator);
#endif
// Derive our set of must-handle values. The compilation must be at least conservative
......@@ -108,7 +112,16 @@ static CompilationResult compile(CompileMode compileMode, ExecState* exec, CodeB
plan->mustHandleValues[i] = exec->uncheckedR(operand).jsValue();
}
plan->compileInThread();
if (enableConcurrentJIT()) {
if (!vm.worklist)
vm.worklist = globalWorklist();
if (logCompilationChanges())
dataLog("Deferring DFG compilation of ", *codeBlock, " with queue length ", vm.worklist->queueLength(), ".\n");
vm.worklist->enqueue(plan);
return CompilationDeferred;
}
plan->compileInThread(*vm.dfgState);
return plan->finalize(jitCode, jitCodeWithArityCheck);
}
......
/*
* Copyright (C) 2011 Apple Inc. All rights reserved.
* Copyright (C) 2011, 2013 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
......@@ -44,12 +44,12 @@ static const char* dfgOpNames[] = {
#undef STRINGIZE_DFG_OP_ENUM
};
Graph::Graph(VM& vm, Plan& plan)
Graph::Graph(VM& vm, Plan& plan, LongLivedState& longLivedState)
: m_vm(vm)
, m_plan(plan)
, m_codeBlock(m_plan.codeBlock.get())
, m_profiledBlock(m_codeBlock->alternative())
, m_allocator(vm.m_dfgState->m_allocator)
, m_allocator(longLivedState.m_allocator)
, m_hasArguments(false)
, m_fixpointState(BeforeFixpoint)
, m_form(LoadStore)
......
......@@ -90,7 +90,7 @@ enum AddSpeculationMode {
// Nodes that are 'dead' remain in the vector with refCount 0.
class Graph {
public:
Graph(VM&, Plan&);
Graph(VM&, Plan&, LongLivedState&);
~Graph();
void changeChild(Edge& edge, Node* newNode)
......
......@@ -45,17 +45,18 @@ void* prepareOSREntry(ExecState* exec, CodeBlock* codeBlock, unsigned bytecodeIn
ASSERT(codeBlock->alternative()->getJITType() == JITCode::BaselineJIT);
ASSERT(!codeBlock->jitCodeMap());
#if ENABLE(JIT_VERBOSE_OSR)
dataLog("OSR in ", *codeBlock->alternative(), " -> ", *codeBlock, " from bc#", bytecodeIndex, "\n");
#endif
if (Options::verboseOSR()) {
dataLog(
"OSR in ", *codeBlock->alternative(), " -> ", *codeBlock,
" from bc#", bytecodeIndex, "\n");
}
VM* vm = &exec->vm();
OSREntryData* entry = codeBlock->getJITCode()->dfg()->osrEntryDataForBytecodeIndex(bytecodeIndex);
if (!entry) {
#if ENABLE(JIT_VERBOSE_OSR)
dataLogF(" OSR failed because the entrypoint was optimized out.\n");
#endif
if (Options::verboseOSR())
dataLogF(" OSR failed because the entrypoint was optimized out.\n");
return 0;
}
......@@ -87,11 +88,11 @@ void* prepareOSREntry(ExecState* exec, CodeBlock* codeBlock, unsigned bytecodeIn
for (size_t argument = 0; argument < entry->m_expectedValues.numberOfArguments(); ++argument) {
if (argument >= exec->argumentCountIncludingThis()) {
#if ENABLE(JIT_VERBOSE_OSR)
dataLogF(" OSR failed because argument %zu was not passed, expected ", argument);
entry->m_expectedValues.argument(argument).dump(WTF::dataFile());
dataLogF(".\n");
#endif
if (Options::verboseOSR()) {
dataLogF(" OSR failed because argument %zu was not passed, expected ", argument);
entry->m_expectedValues.argument(argument).dump(WTF::dataFile());
dataLogF(".\n");
}
return 0;
}
......@@ -102,9 +103,11 @@ void* prepareOSREntry(ExecState* exec, CodeBlock* codeBlock, unsigned bytecodeIn
value = exec->argument(argument - 1);
if (!entry->m_expectedValues.argument(argument).validate(value)) {
#if ENABLE(JIT_VERBOSE_OSR)
dataLog(" OSR failed because argument ", argument, " is ", value, ", expected ", entry->m_expectedValues.argument(argument), ".\n");
#endif
if (Options::verboseOSR()) {
dataLog(
" OSR failed because argument ", argument, " is ", value,
", expected ", entry->m_expectedValues.argument(argument), ".\n");
}
return 0;
}
}
......@@ -112,17 +115,22 @@ void* prepareOSREntry(ExecState* exec, CodeBlock* codeBlock, unsigned bytecodeIn
for (size_t local = 0; local < entry->m_expectedValues.numberOfLocals(); ++local) {
if (entry->m_localsForcedDouble.get(local)) {
if (!exec->registers()[local].jsValue().isNumber()) {
#if ENABLE(JIT_VERBOSE_OSR)
dataLog(" OSR failed because variable ", local, " is ", exec->registers()[local].jsValue(), ", expected number.\n");
#endif
if (Options::verboseOSR()) {
dataLog(
" OSR failed because variable ", local, " is ",
exec->registers()[local].jsValue(), ", expected number.\n");
}
return 0;
}
continue;
}
if (!entry->m_expectedValues.local(local).validate(exec->registers()[local].jsValue())) {
#if ENABLE(JIT_VERBOSE_OSR)
dataLog(" OSR failed because variable ", local, " is ", exec->registers()[local].jsValue(), ", expected ", entry->m_expectedValues.local(local), ".\n");
#endif
if (Options::verboseOSR()) {
dataLog(
" OSR failed because variable ", local, " is ",
exec->registers()[local].jsValue(), ", expected ",
entry->m_expectedValues.local(local), ".\n");
}
return 0;
}
}
......@@ -135,15 +143,13 @@ void* prepareOSREntry(ExecState* exec, CodeBlock* codeBlock, unsigned bytecodeIn
// would have otherwise just kept running albeit less quickly.
if (!vm->interpreter->stack().grow(&exec->registers()[codeBlock->m_numCalleeRegisters])) {
#if ENABLE(JIT_VERBOSE_OSR)
dataLogF(" OSR failed because stack growth failed.\n");
#endif
if (Options::verboseOSR())
dataLogF(" OSR failed because stack growth failed.\n");
return 0;
}
#if ENABLE(JIT_VERBOSE_OSR)
dataLogF(" OSR should succeed.\n");
#endif
if (Options::verboseOSR())
dataLogF(" OSR should succeed.\n");
// 3) Perform data format conversions.
for (size_t local = 0; local < entry->m_expectedValues.numberOfLocals(); ++local) {
......@@ -159,9 +165,8 @@ void* prepareOSREntry(ExecState* exec, CodeBlock* codeBlock, unsigned bytecodeIn
void* result = codeBlock->getJITCode()->executableAddressAtOffset(entry->m_machineCodeOffset);
#if ENABLE(JIT_VERBOSE_OSR)
dataLogF(" OSR returning machine code address %p.\n", result);
#endif
if (Options::verboseOSR())
dataLogF(" OSR returning machine code address %p.\n", result);
return result;
#else // DFG_ENABLE(OSR_ENTRY)
......
......@@ -1722,9 +1722,8 @@ void DFG_OPERATION debugOperationPrintSpeculationFailure(ExecState* exec, void*
extern "C" void DFG_OPERATION triggerReoptimizationNow(CodeBlock* codeBlock)
{
#if ENABLE(JIT_VERBOSE_OSR)
dataLog(*codeBlock, ": Entered reoptimize\n");
#endif
if (Options::verboseOSR())
dataLog(*codeBlock, ": Entered reoptimize\n");
// We must be called with the baseline code block.
ASSERT(JITCode::isBaselineCode(codeBlock->getJITType()));
......
......@@ -78,6 +78,7 @@ Plan::Plan(
, mustHandleValues(codeBlock->numParameters(), numVarsWithValues)
, compilation(codeBlock->vm()->m_perBytecodeProfiler ? adoptRef(new Profiler::Compilation(codeBlock->vm()->m_perBytecodeProfiler->ensureBytecodesFor(codeBlock.get()), Profiler::DFG)) : 0)
, identifiers(codeBlock.get())
, isCompiled(false)
{
}
......@@ -85,12 +86,15 @@ Plan::~Plan()
{
}
void Plan::compileInThread()
void Plan::compileInThread(LongLivedState& longLivedState)
{
SamplingRegion samplingRegion("DFG Compilation (Plan)");
CompilationScope compilationScope;
Graph dfg(vm, *this);
if (logCompilationChanges())
dataLog("DFG(Plan) compiling ", *codeBlock, ", number of instructions = ", codeBlock->instructionCount(), "\n");
Graph dfg(vm, *this, longLivedState);
if (!parse(dfg)) {
finalizer = adoptPtr(new FailedFinalizer(*this));
......@@ -199,6 +203,11 @@ CompilationResult Plan::finalize(RefPtr<JSC::JITCode>& jitCode, MacroAssemblerCo
return CompilationSuccessful;
}
CodeBlock* Plan::key()
{
return codeBlock->alternative();
}
} } // namespace JSC::DFG
#endif // ENABLE(DFG_JIT)
......
......@@ -43,6 +43,8 @@ class CodeBlock;
namespace DFG {
class LongLivedState;
enum CompileMode { CompileFunction, CompileOther };
#if ENABLE(DFG_JIT)
......@@ -53,10 +55,12 @@ struct Plan : public ThreadSafeRefCounted<Plan> {
unsigned osrEntryBytecodeIndex, unsigned numVarsWithValues);
~Plan();
void compileInThread();
void compileInThread(LongLivedState&);
CompilationResult finalize(RefPtr<JSC::JITCode>& jitCode, MacroAssemblerCodePtr* jitCodeWithArityCheck);
CodeBlock* key();
const CompileMode compileMode;
VM& vm;
RefPtr<CodeBlock> codeBlock;
......@@ -71,6 +75,8 @@ struct Plan : public ThreadSafeRefCounted<Plan> {
DesiredWatchpoints watchpoints;
DesiredIdentifiers identifiers;
DesiredStructureChains chains;
bool isCompiled;
private:
bool isStillValid();
......
/*
* Copyright (C) 2013 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include "config.h"
#include "DFGWorklist.h"
#if ENABLE(DFG_JIT)
#include "CodeBlock.h"
#include "DeferGC.h"
#include "DFGLongLivedState.h"
namespace JSC { namespace DFG {
Worklist::Worklist()
{
}
Worklist::~Worklist()
{
m_queue.append(nullptr); // Use null plan to indicate that we want the thread to terminate.
waitForThreadCompletion(m_thread);
}
void Worklist::finishCreation()
{
m_thread = createThread(threadFunction, this, "JSC Compilation Thread");
}
PassRefPtr<Worklist> Worklist::create()
{
RefPtr<Worklist> result = adoptRef(new Worklist());
result->finishCreation();
return result;
}
void Worklist::enqueue(PassRefPtr<Plan> passedPlan)
{
RefPtr<Plan> plan = passedPlan;
MutexLocker locker(m_lock);
if (Options::verboseCompilationQueue()) {
dump(locker, WTF::dataFile());
dataLog(": Enqueueing plan to optimize ", *plan->key(), "\n");
}
ASSERT(m_plans.find(plan->key()) == m_plans.end());
m_plans.add(plan->key(), plan);
m_queue.append(plan);
m_condition.broadcast();
}
Worklist::State Worklist::compilationState(CodeBlock* profiledBlock)
{
MutexLocker locker(m_lock);
PlanMap::iterator iter = m_plans.find(profiledBlock);
if (iter == m_plans.end())
return NotKnown;
return iter->value->isCompiled ? Compiled : Compiling;