Commit 532f1e51 authored by fpizlo@apple.com's avatar fpizlo@apple.com

The DFG should be able to tier-up and OSR enter into the FTL

https://bugs.webkit.org/show_bug.cgi?id=112838

Source/JavaScriptCore: 

Reviewed by Mark Hahnenberg.
        
This adds the ability for the DFG to tier-up into the FTL. This works in both
of the expected tier-up modes:
        
Replacement: frequently called functions eventually have their entrypoint
replaced with one that goes into FTL-compiled code. Note, this will be a
slow-down for now since we don't yet have LLVM calling convention integration.
        
OSR entry: code stuck in hot loops gets OSR'd into the FTL from the DFG.
        
This means that if the DFG detects that a function is an FTL candidate, it
inserts execution counting code similar to the kind that the baseline JIT
would use. If you trip on a loop count in a loop header that is an OSR
candidate (it's not an inlined loop), we do OSR; otherwise we do replacement.
OSR almost always also implies future replacement.
        
OSR entry into the FTL is really cool. It uses a specialized FTL compile of
the code, where early in the DFG pipeline we replace the original root block
with an OSR entrypoint block that jumps to the pre-header of the hot loop.
The OSR entrypoint loads all live state at the loop pre-header using loads
from a scratch buffer, which gets populated by the runtime's OSR entry
preparation code (FTL::prepareOSREntry()). This approach appears to work well
with all of our subsequent optimizations, including prediction propagation,
CFA, and LICM. LLVM seems happy with it, too. Best of all, it works naturally
with concurrent compilation: when we hit the tier-up trigger we spawn a
compilation plan at the bytecode index from which we triggered; once the
compilation finishes the next trigger will try to enter, at that bytecode
index. If it can't - for example because the code has moved on to another
loop - then we just try again. Loops that get hot enough for OSR entry (about
25,000 iterations) will probably still be running when a concurrent compile
finishes, so this doesn't appear to be a big problem.
        
This immediately gives us a 70% speed-up on imaging-gaussian-blur. We could
get a bigger speed-up by adding some more intelligence and tweaking LLVM to
compile code faster. Those things will happen eventually but this is a good
start. Probably this code will see more tuning as we get more coverage in the
FTL JIT, but I'll worry about that in future patches.

* CMakeLists.txt:
* GNUmakefile.list.am:
* JavaScriptCore.xcodeproj/project.pbxproj:
* Target.pri:
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::CodeBlock):
(JSC::CodeBlock::hasOptimizedReplacement):
(JSC::CodeBlock::setOptimizationThresholdBasedOnCompilationResult):
* bytecode/CodeBlock.h:
* dfg/DFGAbstractInterpreterInlines.h:
(JSC::DFG::::executeEffects):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::parseBlock):
(JSC::DFG::ByteCodeParser::parse):
* dfg/DFGCFGSimplificationPhase.cpp:
(JSC::DFG::CFGSimplificationPhase::run):
* dfg/DFGClobberize.h:
(JSC::DFG::clobberize):
* dfg/DFGDriver.cpp:
(JSC::DFG::compileImpl):
(JSC::DFG::compile):
* dfg/DFGDriver.h:
* dfg/DFGFixupPhase.cpp:
(JSC::DFG::FixupPhase::fixupNode):
* dfg/DFGGraph.cpp:
(JSC::DFG::Graph::dump):
(JSC::DFG::Graph::killBlockAndItsContents):
(JSC::DFG::Graph::killUnreachableBlocks):
* dfg/DFGGraph.h:
* dfg/DFGInPlaceAbstractState.cpp:
(JSC::DFG::InPlaceAbstractState::initialize):
* dfg/DFGJITCode.cpp:
(JSC::DFG::JITCode::reconstruct):
(JSC::DFG::JITCode::checkIfOptimizationThresholdReached):
(JSC::DFG::JITCode::optimizeNextInvocation):
(JSC::DFG::JITCode::dontOptimizeAnytimeSoon):
(JSC::DFG::JITCode::optimizeAfterWarmUp):
(JSC::DFG::JITCode::optimizeSoon):
(JSC::DFG::JITCode::forceOptimizationSlowPathConcurrently):
(JSC::DFG::JITCode::setOptimizationThresholdBasedOnCompilationResult):
* dfg/DFGJITCode.h:
* dfg/DFGJITFinalizer.cpp:
(JSC::DFG::JITFinalizer::finalize):
(JSC::DFG::JITFinalizer::finalizeFunction):
(JSC::DFG::JITFinalizer::finalizeCommon):
* dfg/DFGLoopPreHeaderCreationPhase.cpp:
(JSC::DFG::createPreHeader):
(JSC::DFG::LoopPreHeaderCreationPhase::run):
* dfg/DFGLoopPreHeaderCreationPhase.h:
* dfg/DFGNode.h:
(JSC::DFG::Node::hasUnlinkedLocal):
(JSC::DFG::Node::unlinkedLocal):
* dfg/DFGNodeType.h:
* dfg/DFGOSREntry.cpp:
(JSC::DFG::prepareOSREntry):
* dfg/DFGOSREntrypointCreationPhase.cpp: Added.
(JSC::DFG::OSREntrypointCreationPhase::OSREntrypointCreationPhase):
(JSC::DFG::OSREntrypointCreationPhase::run):
(JSC::DFG::performOSREntrypointCreation):
* dfg/DFGOSREntrypointCreationPhase.h: Added.
* dfg/DFGOperations.cpp:
* dfg/DFGOperations.h:
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::Plan):
(JSC::DFG::Plan::compileInThread):
(JSC::DFG::Plan::compileInThreadImpl):
* dfg/DFGPlan.h:
* dfg/DFGPredictionInjectionPhase.cpp:
(JSC::DFG::PredictionInjectionPhase::run):
* dfg/DFGPredictionPropagationPhase.cpp:
(JSC::DFG::PredictionPropagationPhase::propagate):
* dfg/DFGSafeToExecute.h:
(JSC::DFG::safeToExecute):
* dfg/DFGSpeculativeJIT32_64.cpp:
(JSC::DFG::SpeculativeJIT::compile):
* dfg/DFGSpeculativeJIT64.cpp:
(JSC::DFG::SpeculativeJIT::compile):
* dfg/DFGTierUpCheckInjectionPhase.cpp: Added.
(JSC::DFG::TierUpCheckInjectionPhase::TierUpCheckInjectionPhase):
(JSC::DFG::TierUpCheckInjectionPhase::run):
(JSC::DFG::performTierUpCheckInjection):
* dfg/DFGTierUpCheckInjectionPhase.h: Added.
* dfg/DFGToFTLDeferredCompilationCallback.cpp: Added.
(JSC::DFG::ToFTLDeferredCompilationCallback::ToFTLDeferredCompilationCallback):
(JSC::DFG::ToFTLDeferredCompilationCallback::~ToFTLDeferredCompilationCallback):
(JSC::DFG::ToFTLDeferredCompilationCallback::create):
(JSC::DFG::ToFTLDeferredCompilationCallback::compilationDidBecomeReadyAsynchronously):
(JSC::DFG::ToFTLDeferredCompilationCallback::compilationDidComplete):
* dfg/DFGToFTLDeferredCompilationCallback.h: Added.
* dfg/DFGToFTLForOSREntryDeferredCompilationCallback.cpp: Added.
(JSC::DFG::ToFTLForOSREntryDeferredCompilationCallback::ToFTLForOSREntryDeferredCompilationCallback):
(JSC::DFG::ToFTLForOSREntryDeferredCompilationCallback::~ToFTLForOSREntryDeferredCompilationCallback):
(JSC::DFG::ToFTLForOSREntryDeferredCompilationCallback::create):
(JSC::DFG::ToFTLForOSREntryDeferredCompilationCallback::compilationDidBecomeReadyAsynchronously):
(JSC::DFG::ToFTLForOSREntryDeferredCompilationCallback::compilationDidComplete):
* dfg/DFGToFTLForOSREntryDeferredCompilationCallback.h: Added.
* dfg/DFGWorklist.cpp:
(JSC::DFG::globalWorklist):
* dfg/DFGWorklist.h:
* ftl/FTLCapabilities.cpp:
(JSC::FTL::canCompile):
* ftl/FTLCapabilities.h:
* ftl/FTLForOSREntryJITCode.cpp: Added.
(JSC::FTL::ForOSREntryJITCode::ForOSREntryJITCode):
(JSC::FTL::ForOSREntryJITCode::~ForOSREntryJITCode):
(JSC::FTL::ForOSREntryJITCode::ftlForOSREntry):
(JSC::FTL::ForOSREntryJITCode::initializeEntryBuffer):
* ftl/FTLForOSREntryJITCode.h: Added.
(JSC::FTL::ForOSREntryJITCode::entryBuffer):
(JSC::FTL::ForOSREntryJITCode::setBytecodeIndex):
(JSC::FTL::ForOSREntryJITCode::bytecodeIndex):
(JSC::FTL::ForOSREntryJITCode::countEntryFailure):
(JSC::FTL::ForOSREntryJITCode::entryFailureCount):
* ftl/FTLJITFinalizer.cpp:
(JSC::FTL::JITFinalizer::finalizeFunction):
* ftl/FTLLink.cpp:
(JSC::FTL::link):
* ftl/FTLLowerDFGToLLVM.cpp:
(JSC::FTL::LowerDFGToLLVM::compileBlock):
(JSC::FTL::LowerDFGToLLVM::compileNode):
(JSC::FTL::LowerDFGToLLVM::compileExtractOSREntryLocal):
(JSC::FTL::LowerDFGToLLVM::compileGetLocal):
(JSC::FTL::LowerDFGToLLVM::addWeakReference):
* ftl/FTLOSREntry.cpp: Added.
(JSC::FTL::prepareOSREntry):
* ftl/FTLOSREntry.h: Added.
* ftl/FTLOutput.h:
(JSC::FTL::Output::crashNonTerminal):
(JSC::FTL::Output::crash):
* ftl/FTLState.cpp:
(JSC::FTL::State::State):
* interpreter/Register.h:
(JSC::Register::unboxedDouble):
* jit/JIT.cpp:
(JSC::JIT::emitEnterOptimizationCheck):
* jit/JITCode.cpp:
(JSC::JITCode::ftlForOSREntry):
* jit/JITCode.h:
* jit/JITStubs.cpp:
(JSC::DEFINE_STUB_FUNCTION):
* runtime/Executable.cpp:
(JSC::ScriptExecutable::newReplacementCodeBlockFor):
* runtime/Options.h:
* runtime/VM.cpp:
(JSC::VM::ensureWorklist):
* runtime/VM.h:

LayoutTests: 

Reviewed by Mark Hahnenberg.
        
Fix marsaglia to check the result instead of printing, and add a second
version that relies on OSR entry.

* fast/js/regress/marsaglia-osr-entry-expected.txt: Added.
* fast/js/regress/marsaglia-osr-entry.html: Added.
* fast/js/regress/script-tests/marsaglia-osr-entry.js: Added.
(marsaglia):
* fast/js/regress/script-tests/marsaglia.js:



git-svn-id: http://svn.webkit.org/repository/webkit/trunk@155023 268f45cc-cd09-0410-ab3c-d52691b4dbfc
parent be7b8b83
2013-09-03 Filip Pizlo <fpizlo@apple.com>
The DFG should be able to tier-up and OSR enter into the FTL
https://bugs.webkit.org/show_bug.cgi?id=112838
Reviewed by Mark Hahnenberg.
Fix marsaglia to check the result instead of printing, and add a second
version that relies on OSR entry.
* fast/js/regress/marsaglia-osr-entry-expected.txt: Added.
* fast/js/regress/marsaglia-osr-entry.html: Added.
* fast/js/regress/script-tests/marsaglia-osr-entry.js: Added.
(marsaglia):
* fast/js/regress/script-tests/marsaglia.js:
2013-09-03 Chris Fleizach <cfleizach@apple.com>
AX: REGRESSION: @title is exposed as AXDescription when label label from contents already exists.
JSRegress/marsaglia-osr-entry
On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
PASS no exception thrown
PASS successfullyParsed is true
TEST COMPLETE
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
<html>
<head>
<script src="../resources/js-test-pre.js"></script>
</head>
<body>
<script src="resources/regress-pre.js"></script>
<script src="script-tests/marsaglia-osr-entry.js"></script>
<script src="resources/regress-post.js"></script>
<script src="../resources/js-test-post.js"></script>
</body>
</html>
function marsaglia(m_z, m_w, n) {
var result;
for (var i = 0; i < n; ++i) {
m_z = (36969 * (m_z & 65535) + (m_z >> 16)) | 0;
m_w = (18000 * (m_w & 65535) + (m_w >> 16)) | 0;
result = ((m_z << 16) + m_w) | 0;
}
return result;
}
var result = marsaglia(5, 7, 10000000);
if (result != -1047364056)
throw "Error: bad result: " + result;
......@@ -12,4 +12,6 @@ var result = 0;
for (var i = 0; i < 100; ++i)
result += marsaglia(i, i + 1, 1000000);
print(result);
if (result != 8216386243)
throw "Error: bad result: " + result;
......@@ -132,6 +132,7 @@ set(JavaScriptCore_SOURCES
dfg/DFGNodeFlags.cpp
dfg/DFGOSRAvailabilityAnalysisPhase.cpp
dfg/DFGOSREntry.cpp
dfg/DFGOSREntrypointCreationPhase.cpp
dfg/DFGOSRExit.cpp
dfg/DFGOSRExitBase.cpp
dfg/DFGOSRExitCompiler.cpp
......@@ -151,6 +152,7 @@ set(JavaScriptCore_SOURCES
dfg/DFGSpeculativeJIT32_64.cpp
dfg/DFGSpeculativeJIT64.cpp
dfg/DFGThunks.cpp
dfg/DFGTierUpCheckInjectionPhase.cpp
dfg/DFGTypeCheckHoistingPhase.cpp
dfg/DFGUnificationPhase.cpp
dfg/DFGUseKind.cpp
......
2013-09-03 Filip Pizlo <fpizlo@apple.com>
The DFG should be able to tier-up and OSR enter into the FTL
https://bugs.webkit.org/show_bug.cgi?id=112838
Reviewed by Mark Hahnenberg.
This adds the ability for the DFG to tier-up into the FTL. This works in both
of the expected tier-up modes:
Replacement: frequently called functions eventually have their entrypoint
replaced with one that goes into FTL-compiled code. Note, this will be a
slow-down for now since we don't yet have LLVM calling convention integration.
OSR entry: code stuck in hot loops gets OSR'd into the FTL from the DFG.
This means that if the DFG detects that a function is an FTL candidate, it
inserts execution counting code similar to the kind that the baseline JIT
would use. If you trip on a loop count in a loop header that is an OSR
candidate (it's not an inlined loop), we do OSR; otherwise we do replacement.
OSR almost always also implies future replacement.
OSR entry into the FTL is really cool. It uses a specialized FTL compile of
the code, where early in the DFG pipeline we replace the original root block
with an OSR entrypoint block that jumps to the pre-header of the hot loop.
The OSR entrypoint loads all live state at the loop pre-header using loads
from a scratch buffer, which gets populated by the runtime's OSR entry
preparation code (FTL::prepareOSREntry()). This approach appears to work well
with all of our subsequent optimizations, including prediction propagation,
CFA, and LICM. LLVM seems happy with it, too. Best of all, it works naturally
with concurrent compilation: when we hit the tier-up trigger we spawn a
compilation plan at the bytecode index from which we triggered; once the
compilation finishes the next trigger will try to enter, at that bytecode
index. If it can't - for example because the code has moved on to another
loop - then we just try again. Loops that get hot enough for OSR entry (about
25,000 iterations) will probably still be running when a concurrent compile
finishes, so this doesn't appear to be a big problem.
This immediately gives us a 70% speed-up on imaging-gaussian-blur. We could
get a bigger speed-up by adding some more intelligence and tweaking LLVM to
compile code faster. Those things will happen eventually but this is a good
start. Probably this code will see more tuning as we get more coverage in the
FTL JIT, but I'll worry about that in future patches.
* CMakeLists.txt:
* GNUmakefile.list.am:
* JavaScriptCore.xcodeproj/project.pbxproj:
* Target.pri:
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::CodeBlock):
(JSC::CodeBlock::hasOptimizedReplacement):
(JSC::CodeBlock::setOptimizationThresholdBasedOnCompilationResult):
* bytecode/CodeBlock.h:
* dfg/DFGAbstractInterpreterInlines.h:
(JSC::DFG::::executeEffects):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::parseBlock):
(JSC::DFG::ByteCodeParser::parse):
* dfg/DFGCFGSimplificationPhase.cpp:
(JSC::DFG::CFGSimplificationPhase::run):
* dfg/DFGClobberize.h:
(JSC::DFG::clobberize):
* dfg/DFGDriver.cpp:
(JSC::DFG::compileImpl):
(JSC::DFG::compile):
* dfg/DFGDriver.h:
* dfg/DFGFixupPhase.cpp:
(JSC::DFG::FixupPhase::fixupNode):
* dfg/DFGGraph.cpp:
(JSC::DFG::Graph::dump):
(JSC::DFG::Graph::killBlockAndItsContents):
(JSC::DFG::Graph::killUnreachableBlocks):
* dfg/DFGGraph.h:
* dfg/DFGInPlaceAbstractState.cpp:
(JSC::DFG::InPlaceAbstractState::initialize):
* dfg/DFGJITCode.cpp:
(JSC::DFG::JITCode::reconstruct):
(JSC::DFG::JITCode::checkIfOptimizationThresholdReached):
(JSC::DFG::JITCode::optimizeNextInvocation):
(JSC::DFG::JITCode::dontOptimizeAnytimeSoon):
(JSC::DFG::JITCode::optimizeAfterWarmUp):
(JSC::DFG::JITCode::optimizeSoon):
(JSC::DFG::JITCode::forceOptimizationSlowPathConcurrently):
(JSC::DFG::JITCode::setOptimizationThresholdBasedOnCompilationResult):
* dfg/DFGJITCode.h:
* dfg/DFGJITFinalizer.cpp:
(JSC::DFG::JITFinalizer::finalize):
(JSC::DFG::JITFinalizer::finalizeFunction):
(JSC::DFG::JITFinalizer::finalizeCommon):
* dfg/DFGLoopPreHeaderCreationPhase.cpp:
(JSC::DFG::createPreHeader):
(JSC::DFG::LoopPreHeaderCreationPhase::run):
* dfg/DFGLoopPreHeaderCreationPhase.h:
* dfg/DFGNode.h:
(JSC::DFG::Node::hasUnlinkedLocal):
(JSC::DFG::Node::unlinkedLocal):
* dfg/DFGNodeType.h:
* dfg/DFGOSREntry.cpp:
(JSC::DFG::prepareOSREntry):
* dfg/DFGOSREntrypointCreationPhase.cpp: Added.
(JSC::DFG::OSREntrypointCreationPhase::OSREntrypointCreationPhase):
(JSC::DFG::OSREntrypointCreationPhase::run):
(JSC::DFG::performOSREntrypointCreation):
* dfg/DFGOSREntrypointCreationPhase.h: Added.
* dfg/DFGOperations.cpp:
* dfg/DFGOperations.h:
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::Plan):
(JSC::DFG::Plan::compileInThread):
(JSC::DFG::Plan::compileInThreadImpl):
* dfg/DFGPlan.h:
* dfg/DFGPredictionInjectionPhase.cpp:
(JSC::DFG::PredictionInjectionPhase::run):
* dfg/DFGPredictionPropagationPhase.cpp:
(JSC::DFG::PredictionPropagationPhase::propagate):
* dfg/DFGSafeToExecute.h:
(JSC::DFG::safeToExecute):
* dfg/DFGSpeculativeJIT32_64.cpp:
(JSC::DFG::SpeculativeJIT::compile):
* dfg/DFGSpeculativeJIT64.cpp:
(JSC::DFG::SpeculativeJIT::compile):
* dfg/DFGTierUpCheckInjectionPhase.cpp: Added.
(JSC::DFG::TierUpCheckInjectionPhase::TierUpCheckInjectionPhase):
(JSC::DFG::TierUpCheckInjectionPhase::run):
(JSC::DFG::performTierUpCheckInjection):
* dfg/DFGTierUpCheckInjectionPhase.h: Added.
* dfg/DFGToFTLDeferredCompilationCallback.cpp: Added.
(JSC::DFG::ToFTLDeferredCompilationCallback::ToFTLDeferredCompilationCallback):
(JSC::DFG::ToFTLDeferredCompilationCallback::~ToFTLDeferredCompilationCallback):
(JSC::DFG::ToFTLDeferredCompilationCallback::create):
(JSC::DFG::ToFTLDeferredCompilationCallback::compilationDidBecomeReadyAsynchronously):
(JSC::DFG::ToFTLDeferredCompilationCallback::compilationDidComplete):
* dfg/DFGToFTLDeferredCompilationCallback.h: Added.
* dfg/DFGToFTLForOSREntryDeferredCompilationCallback.cpp: Added.
(JSC::DFG::ToFTLForOSREntryDeferredCompilationCallback::ToFTLForOSREntryDeferredCompilationCallback):
(JSC::DFG::ToFTLForOSREntryDeferredCompilationCallback::~ToFTLForOSREntryDeferredCompilationCallback):
(JSC::DFG::ToFTLForOSREntryDeferredCompilationCallback::create):
(JSC::DFG::ToFTLForOSREntryDeferredCompilationCallback::compilationDidBecomeReadyAsynchronously):
(JSC::DFG::ToFTLForOSREntryDeferredCompilationCallback::compilationDidComplete):
* dfg/DFGToFTLForOSREntryDeferredCompilationCallback.h: Added.
* dfg/DFGWorklist.cpp:
(JSC::DFG::globalWorklist):
* dfg/DFGWorklist.h:
* ftl/FTLCapabilities.cpp:
(JSC::FTL::canCompile):
* ftl/FTLCapabilities.h:
* ftl/FTLForOSREntryJITCode.cpp: Added.
(JSC::FTL::ForOSREntryJITCode::ForOSREntryJITCode):
(JSC::FTL::ForOSREntryJITCode::~ForOSREntryJITCode):
(JSC::FTL::ForOSREntryJITCode::ftlForOSREntry):
(JSC::FTL::ForOSREntryJITCode::initializeEntryBuffer):
* ftl/FTLForOSREntryJITCode.h: Added.
(JSC::FTL::ForOSREntryJITCode::entryBuffer):
(JSC::FTL::ForOSREntryJITCode::setBytecodeIndex):
(JSC::FTL::ForOSREntryJITCode::bytecodeIndex):
(JSC::FTL::ForOSREntryJITCode::countEntryFailure):
(JSC::FTL::ForOSREntryJITCode::entryFailureCount):
* ftl/FTLJITFinalizer.cpp:
(JSC::FTL::JITFinalizer::finalizeFunction):
* ftl/FTLLink.cpp:
(JSC::FTL::link):
* ftl/FTLLowerDFGToLLVM.cpp:
(JSC::FTL::LowerDFGToLLVM::compileBlock):
(JSC::FTL::LowerDFGToLLVM::compileNode):
(JSC::FTL::LowerDFGToLLVM::compileExtractOSREntryLocal):
(JSC::FTL::LowerDFGToLLVM::compileGetLocal):
(JSC::FTL::LowerDFGToLLVM::addWeakReference):
* ftl/FTLOSREntry.cpp: Added.
(JSC::FTL::prepareOSREntry):
* ftl/FTLOSREntry.h: Added.
* ftl/FTLOutput.h:
(JSC::FTL::Output::crashNonTerminal):
(JSC::FTL::Output::crash):
* ftl/FTLState.cpp:
(JSC::FTL::State::State):
* interpreter/Register.h:
(JSC::Register::unboxedDouble):
* jit/JIT.cpp:
(JSC::JIT::emitEnterOptimizationCheck):
* jit/JITCode.cpp:
(JSC::JITCode::ftlForOSREntry):
* jit/JITCode.h:
* jit/JITStubs.cpp:
(JSC::DEFINE_STUB_FUNCTION):
* runtime/Executable.cpp:
(JSC::ScriptExecutable::newReplacementCodeBlockFor):
* runtime/Options.h:
* runtime/VM.cpp:
(JSC::VM::ensureWorklist):
* runtime/VM.h:
2013-09-03 Filip Pizlo <fpizlo@apple.com>
CodeBlock memory cost reporting should be rationalized
......
......@@ -316,6 +316,8 @@ javascriptcore_sources += \
Source/JavaScriptCore/dfg/DFGOSRAvailabilityAnalysisPhase.h \
Source/JavaScriptCore/dfg/DFGOSREntry.cpp \
Source/JavaScriptCore/dfg/DFGOSREntry.h \
Source/JavaScriptCore/dfg/DFGOSREntrypointCreationPhase.cpp \
Source/JavaScriptCore/dfg/DFGOSREntrypointCreationPhase.h \
Source/JavaScriptCore/dfg/DFGOSRExitCompilationInfo.h \
Source/JavaScriptCore/dfg/DFGOSRExitCompiler32_64.cpp \
Source/JavaScriptCore/dfg/DFGOSRExitCompiler64.cpp \
......@@ -356,6 +358,8 @@ javascriptcore_sources += \
Source/JavaScriptCore/dfg/DFGSSAConversionPhase.cpp \
Source/JavaScriptCore/dfg/DFGSSAConversionPhase.h \
Source/JavaScriptCore/dfg/DFGStructureAbstractValue.h \
Source/JavaScriptCore/dfg/DFGTierUpCheckInjectionPhase.cpp \
Source/JavaScriptCore/dfg/DFGTierUpCheckInjectionPhase.h \
Source/JavaScriptCore/dfg/DFGThunks.cpp \
Source/JavaScriptCore/dfg/DFGThunks.h \
Source/JavaScriptCore/dfg/DFGTypeCheckHoistingPhase.cpp \
......
......@@ -170,6 +170,7 @@ SOURCES += \
dfg/DFGOperations.cpp \
dfg/DFGOSRAvailabilityAnalysisPhase.cpp \
dfg/DFGOSREntry.cpp \
dfg/DFGOSREntrypointCreationPhase.cpp \
dfg/DFGOSRExit.cpp \
dfg/DFGOSRExitBase.cpp \
dfg/DFGOSRExitCompiler.cpp \
......@@ -189,6 +190,7 @@ SOURCES += \
dfg/DFGSpeculativeJIT64.cpp \
dfg/DFGTypeCheckHoistingPhase.cpp \
dfg/DFGThunks.cpp \
dfg/DFGTierUpCheckInjectionPhase.cpp \
dfg/DFGUnificationPhase.cpp \
dfg/DFGUseKind.cpp \
dfg/DFGValueSource.cpp \
......
......@@ -1479,6 +1479,7 @@ CodeBlock::CodeBlock(CopyParsedBlockTag, CodeBlock& other)
, m_numVars(other.m_numVars)
, m_isConstructor(other.m_isConstructor)
, m_shouldAlwaysBeInlined(true)
, m_didFailFTLCompilation(false)
, m_unlinkedCode(*other.m_vm, other.m_ownerExecutable.get(), other.m_unlinkedCode.get())
, m_ownerExecutable(*other.m_vm, other.m_ownerExecutable.get(), other.m_ownerExecutable.get())
, m_vm(other.m_vm)
......@@ -1529,6 +1530,7 @@ CodeBlock::CodeBlock(ScriptExecutable* ownerExecutable, UnlinkedCodeBlock* unlin
, m_numVars(unlinkedCodeBlock->m_numVars)
, m_isConstructor(unlinkedCodeBlock->isConstructor())
, m_shouldAlwaysBeInlined(true)
, m_didFailFTLCompilation(false)
, m_unlinkedCode(m_globalObject->vm(), ownerExecutable, unlinkedCodeBlock)
, m_ownerExecutable(m_globalObject->vm(), ownerExecutable, ownerExecutable)
, m_vm(unlinkedCodeBlock->vm())
......@@ -1902,6 +1904,18 @@ void EvalCodeCache::visitAggregate(SlotVisitor& visitor)
visitor.append(&ptr->value);
}
CodeBlock* CodeBlock::specialOSREntryBlockOrNull()
{
#if ENABLE(FTL_JIT)
if (jitType() != JITCode::DFGJIT)
return 0;
DFG::JITCode* jitCode = m_jitCode->dfg();
return jitCode->osrEntryBlock.get();
#else // ENABLE(FTL_JIT)
return 0;
#endif // ENABLE(FTL_JIT)
}
void CodeBlock::visitAggregate(SlotVisitor& visitor)
{
#if ENABLE(PARALLEL_GC)
......@@ -1930,6 +1944,9 @@ void CodeBlock::visitAggregate(SlotVisitor& visitor)
if (!!m_alternative)
m_alternative->visitAggregate(visitor);
if (CodeBlock* otherBlock = specialOSREntryBlockOrNull())
otherBlock->visitAggregate(visitor);
visitor.reportExtraMemoryUsage(sizeof(CodeBlock));
if (m_jitCode)
......@@ -2385,17 +2402,14 @@ CodeBlock* CodeBlock::baselineVersion()
}
#if ENABLE(JIT)
bool CodeBlock::hasOptimizedReplacement(JITCode::JITType typeToReplace)
{
return JITCode::isHigherTier(replacement()->jitType(), typeToReplace);
}
bool CodeBlock::hasOptimizedReplacement()
{
ASSERT(JITCode::isBaselineCode(jitType()));
bool result = JITCode::isHigherTier(replacement()->jitType(), jitType());
if (result)
ASSERT(JITCode::isOptimizingJIT(replacement()->jitType()));
else {
ASSERT(JITCode::isBaselineCode(replacement()->jitType()));
ASSERT(replacement() == this);
}
return result;
return hasOptimizedReplacement(jitType());
}
#endif
......@@ -2661,6 +2675,8 @@ void CodeBlock::clearEvalCache()
{
if (!!m_alternative)
m_alternative->clearEvalCache();
if (CodeBlock* otherBlock = specialOSREntryBlockOrNull())
otherBlock->clearEvalCache();
if (!m_rareData)
return;
m_rareData->m_evalCodeCache.clear();
......@@ -3030,10 +3046,10 @@ void CodeBlock::setOptimizationThresholdBasedOnCompilationResult(CompilationResu
case CompilationSuccessful:
RELEASE_ASSERT(JITCode::isOptimizingJIT(replacement()->jitType()));
optimizeNextInvocation();
break;
return;
case CompilationFailed:
dontOptimizeAnytimeSoon();
break;
return;
case CompilationDeferred:
// We'd like to do dontOptimizeAnytimeSoon() but we cannot because
// forceOptimizationSlowPathConcurrently() is inherently racy. It won't
......@@ -3041,16 +3057,14 @@ void CodeBlock::setOptimizationThresholdBasedOnCompilationResult(CompilationResu
// function ends up being a no-op, we still eventually retry and realize
// that we have optimized code ready.
optimizeAfterWarmUp();
break;
return;
case CompilationInvalidated:
// Retry with exponential backoff.
countReoptimization();
optimizeAfterWarmUp();
break;
default:
RELEASE_ASSERT_NOT_REACHED();
break;
return;
}
RELEASE_ASSERT_NOT_REACHED();
}
#endif
......
......@@ -308,7 +308,8 @@ public:
}
DFG::CapabilityLevel capabilityLevelState() { return m_capabilityLevelState; }
bool hasOptimizedReplacement();
bool hasOptimizedReplacement(JITCode::JITType typeToReplace);
bool hasOptimizedReplacement(); // the typeToReplace is my JITType
#endif
ScriptExecutable* ownerExecutable() const { return m_ownerExecutable.get(); }
......@@ -957,6 +958,8 @@ public:
bool m_shouldAlwaysBeInlined;
bool m_allTransitionsHaveBeenMarked; // Initialized and used on every GC.
bool m_didFailFTLCompilation;
protected:
virtual void visitWeakReferences(SlotVisitor&);
virtual void finalizeUnconditionally();
......@@ -970,6 +973,8 @@ protected:
private:
friend class CodeBlockSet;
CodeBlock* specialOSREntryBlockOrNull();
void noticeIncomingCall(ExecState* callerFrame);
double optimizationThresholdScalingFactor();
......
......@@ -150,6 +150,20 @@ bool AbstractInterpreter<AbstractStateType>::executeEffects(unsigned clobberLimi
forNode(node) = value;
break;
}
case ExtractOSREntryLocal: {
forNode(node).makeTop();
if (!operandIsArgument(node->unlinkedLocal())
&& m_graph.m_lazyVars.get(node->unlinkedLocal())) {
// This is kind of pessimistic - we could know in some cases that the
// DFG code at the point of the OSR had already initialized the lazy
// variable. But maybe this is fine, since we're inserting OSR
// entrypoints very early in the pipeline - so any lazy initializations
// ought to be hoisted out anyway.
forNode(node).merge(SpecEmpty);
}
break;
}
case GetLocal: {
VariableAccessData* variableAccessData = node->variableAccessData();
......@@ -1515,6 +1529,14 @@ bool AbstractInterpreter<AbstractStateType>::executeEffects(unsigned clobberLimi
case Phantom:
case InlineStart:
case CountExecution:
case CheckTierUpInLoop:
case CheckTierUpAtReturn:
break;
case CheckTierUpAndOSREnter:
case LoopHint:
// We pretend that it can exit because it may want to get all state.
node->setCanExit(true);
break;
case Unreachable:
......
......@@ -3140,19 +3140,18 @@ bool ByteCodeParser::parseBlock(unsigned limit)
if (!m_inlineStackTop->m_caller)
m_currentBlock->isOSRTarget = true;
addToGraph(LoopHint);
if (m_vm->watchdog.isEnabled())
addToGraph(CheckWatchdogTimer);
else {
// Emit a phantom node to ensure that there is a placeholder
// node for this bytecode op.
addToGraph(Phantom);
}
NEXT_OPCODE(op_loop_hint);
}
case op_init_lazy_reg: {
set(currentInstruction[1].u.operand, getJSConstantForValue(JSValue()));
ASSERT(currentInstruction[1].u.operand >= 0);
m_graph.m_lazyVars.set(currentInstruction[1].u.operand);
NEXT_OPCODE(op_init_lazy_reg);
}
......@@ -3634,7 +3633,7 @@ bool ByteCodeParser::parse()
BasicBlock* block = m_graph.block(blockIndex);
ASSERT(block);
if (!block->isReachable) {
m_graph.killBlock(block);
m_graph.killBlockAndItsContents(block);
continue;
}
......
......@@ -270,16 +270,7 @@ public:
m_graph.invalidateCFG();
m_graph.resetReachability();
for (BlockIndex blockIndex = 0; blockIndex < m_graph.numBlocks(); ++blockIndex) {
BasicBlock* block = m_graph.block(blockIndex);
if (!block)
continue;
if (block->isReachable)
continue;
killUnreachable(block);
}
m_graph.killUnreachableBlocks();
}
if (Options::validateGraphAtEachPhase())
......@@ -320,19 +311,6 @@ private:
}
}
void killUnreachable(BasicBlock* block)
{
ASSERT(block);
ASSERT(!block->isReachable);
for (unsigned phiIndex = block->phis.size(); phiIndex--;)
m_graph.m_allocator.free(block->phis[phiIndex]);
for (unsigned nodeIndex = block->size(); nodeIndex--;)
m_graph.m_allocator.free(block->at(nodeIndex));
m_graph.killBlock(block);
}
void keepOperandAlive(BasicBlock* block, BasicBlock* jettisonedBlock, CodeOrigin codeOrigin, int operand)
{
Node* livenessNode = jettisonedBlock->variablesAtHead.operand(operand);
......
......@@ -111,6 +111,7 @@ void clobberize(Graph& graph, Node* node, ReadFunctor& read, WriteFunctor& write
case IsString:
case LogicalNot:
case Int32ToDouble:
case ExtractOSREntryLocal:
return;
case MovHintAndCheck:
......@@ -133,6 +134,10 @@ void clobberize(Graph& graph, Node* node, ReadFunctor& read, WriteFunctor& write
case ForceOSRExit:
case Return:
case Unreachable:
case CheckTierUpInLoop:
case CheckTierUpAtReturn:
case CheckTierUpAndOSREnter:
case LoopHint:
write(SideState);
return;
......
......@@ -55,9 +55,9 @@ unsigned getNumCompilations()
#if ENABLE(DFG_JIT)
static CompilationResult compileImpl(
ExecState* exec, CodeBlock* codeBlock, CompilationMode mode,
unsigned osrEntryBytecodeIndex, PassRefPtr<DeferredCompilationCallback> callback,
Worklist* worklist)
VM& vm, CodeBlock* codeBlock, CompilationMode mode, unsigned osrEntryBytecodeIndex,
const Operands<JSValue>& mustHandleValues,
PassRefPtr<DeferredCompilationCallback> callback, Worklist* worklist)
{
SamplingRegion samplingRegion("DFG Compilation (Driver)");
......@@ -67,8 +67,6 @@ static CompilationResult compileImpl(
ASSERT(codeBlock->alternative());
ASSERT(codeBlock->alternative()->jitType() == JITCode::BaselineJIT);
ASSERT(osrEntryBytecodeIndex != UINT_MAX);
if (!Options::useDFGJIT() || !MacroAssembler::supportsFloatingPoint())
return CompilationFailed;
......@@ -76,9 +74,7 @@ static CompilationResult compileImpl(
return CompilationFailed;
if (logCompilationChanges())
dataLog("DFG(Driver) compiling ", *codeBlock, ", number of instructions = ", codeBlock->instructionCount(), "\n");
VM& vm = exec->vm();
dataLog("DFG(Driver) compiling ", *codeBlock, " with ", mode, ", number of instructions = ", codeBlock->instructionCount(), "\n");
// Make sure that any stubs that the DFG is going to use are initialized. We want to
// make sure that al JIT code generation does finalization on the main thread.
......@@ -93,29 +89,8 @@ static CompilationResult compileImpl(
vm.getCTIStub(FTL::osrExitGenerationThunkGenerator);
#endif
// Derive our set of must-handle values. The compilation must be at least conservative
// enough to allow for OSR entry with these values.
unsigned numVarsWithValues;
if (osrEntryBytecodeIndex)
numVarsWithValues = codeBlock->m_numVars;
else
numVarsWithValues = 0;
RefPtr<Plan> plan = adoptRef(
new Plan(codeBlock, mode, osrEntryBytecodeIndex, numVarsWithValues));
for (size_t i = 0; i < plan->mustHandleValues.size(); ++i) {
int operand = plan->mustHandleValues.operandForIndex(i);
if (operandIsArgument(operand)
&& !operandToArgument(operand)
&& codeBlock->codeType() == FunctionCode
&& codeBlock->specializationKind() == CodeForConstruct) {
// Ugh. If we're in a constructor, the 'this' argument may hold garbage. It will
// also never be used. It doesn't matter what we put into the value for this,
// but it has to be an actual value that can be grokked by subsequent DFG passes,
// so we sanitize it here by turning it into Undefined.
plan->mustHandleValues[i] = jsUndefined();
} else
plan->mustHandleValues[i] = exec->uncheckedR(operand).jsValue();
}
new Plan(codeBlock, mode, osrEntryBytecodeIndex, mustHandleValues));
if (worklist) {
plan->callback = callback;
......@@ -130,7 +105,7 @@ static CompilationResult compileImpl(
}
#else // ENABLE(DFG_JIT)
static CompilationResult compileImpl(
ExecState*, CodeBlock*, CompilationMode, unsigned,
VM&, CodeBlock*, CompilationMode, unsigned, const Operands<JSValue>&,
PassRefPtr<DeferredCompilationCallback>, Worklist*)
{