Commit a62d4829 authored by fpizlo@apple.com's avatar fpizlo@apple.com

Compress DFG stack layout

https://bugs.webkit.org/show_bug.cgi?id=122024

Reviewed by Oliver Hunt.
        
The DFG needs to be able to store things at a known offset from frame pointer so that
the runtime can read those things. Prior to this patch, the DFG would use the exact
offsets that the bytecode asked for, even in the case of inlining, where it would use
the callsite stack offset to shift all of the inlined function's variables over just as
they would have been if a bytecode interpreter had really made the call.
        
But this won't work once WebKit-LLVM integration is complete. LLVM has no notion of
storing things at a fixed offset from the frame pointer. We could try to hack LLVM to do
that, but it would seriously complicate LLVM's stack layout. But what we might be able
to do is have LLVM tell us (via an addressof intrinsic and a side-channel) where some
alloca landed relative to the frame pointer. Hence if the DFG can put all of its flushed
variables in a contiguous range that can be expressed to LLVM as a struct that we
alloca, then all of this can still work just fine.
        
Previously the flushed variables didn't fit in a contiguous range, but this patch makes
them contiguous by allowing the stack layout to be compressed.
        
What this really means is that there is now a distinction between where the DFG saw a
variable stored in bytecode and where it will actually store it in the resulting machine
code. Henceforth when the DFG says "local" or "virtual register" it means the variable
according to bytecode (with the stack offsetting for inlined code as before), but when
it says "machine local" or "machine virtual register" it means the actual place where it
will store things in the resulting machine code. All of the OSR exit, inlined arguments,
captured variables, and various stack unwinding machine now knows about all of this.
        
Note that the DFG's abstract interpretation still uses bytecode variables rather than
machine variables. Same for CSE and abstract heaps. This makes sense since it means that
we don't have to decide on machine variable allocation just to do those optimizations.
        
The decision of what a local's machine location becomes is deferred to very late in
compilation. We only need to assign machine locations to variables that must be stored
to the stack. It's now mandatory to run some kind of "stack layout phase" that makes the
decision and updates all data structures.
        
So far the way that this is being used is just to compress the DFG stack layout, which
is something that we should have done anyway, a long time ago. And the compression isn't
even that good - the current StackLayoutPhase just identifies local indices that are
unused in machine code and slides all other variables towards zero. This doesn't achieve
particularly good compression but it is better than nothing. Note that this phase makes
it seem like the bytecode-machine mapping is based on bytecode local indices; for
example if bytecode local 4 is mapped to machine local 3 then it always will be. That's
true for the current StackLayoutPhase but it _will not_ be true for all possible stack
layout phases and it would be incorrect to assume that it should be true. This is why
the current data structures have each VariableAccessData hold its own copy of the
machine virtual register, and also have each InlineCallFrame report their own machine
virtual registers for the various things. The DFG backend is likely to always use the
dumb StackLayoutPhase since it is very cheap to run, but the FTL backend is likely to
eventually get a better one, where we do some kind of constraint-based coloring: we
institute constraints where some VariableAccessData's must have the same indices as some
other ones, and also must be right next to some other ones; then we process all
VariableAccessData's and attempt to assign them machine locals while preserving those
constraints. This could lead to two VariableAccessDatas for the same bytecode local
ending up with different machine locals.

* CMakeLists.txt:
* GNUmakefile.list.am:
* JavaScriptCore.xcodeproj/project.pbxproj:
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::CodeBlock):
(JSC::CodeBlock::isCaptured):
(JSC::CodeBlock::framePointerOffsetToGetActivationRegisters):
(JSC::CodeBlock::machineSlowArguments):
* bytecode/CodeBlock.h:
(JSC::CodeBlock::hasSlowArguments):
* bytecode/CodeOrigin.cpp:
(JSC::CodeOrigin::dump):
(JSC::InlineCallFrame::calleeForCallFrame):
(JSC::InlineCallFrame::dumpInContext):
* bytecode/CodeOrigin.h:
(JSC::InlineCallFrame::InlineCallFrame):
(JSC::InlineCallFrame::calleeConstant):
* bytecode/Operands.h:
(JSC::Operands::indexForOperand):
* dfg/DFGBasicBlock.cpp:
(JSC::DFG::BasicBlock::SSAData::SSAData):
* dfg/DFGBasicBlock.h:
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::ByteCodeParser):
(JSC::DFG::ByteCodeParser::get):
(JSC::DFG::ByteCodeParser::getLocal):
(JSC::DFG::ByteCodeParser::flushDirect):
(JSC::DFG::ByteCodeParser::flush):
(JSC::DFG::ByteCodeParser::handleInlining):
(JSC::DFG::ByteCodeParser::InlineStackEntry::InlineStackEntry):
(JSC::DFG::ByteCodeParser::parse):
* dfg/DFGCommon.h:
* dfg/DFGCommonData.h:
(JSC::DFG::CommonData::CommonData):
* dfg/DFGDesiredWriteBarriers.cpp:
(JSC::DFG::DesiredWriteBarrier::trigger):
* dfg/DFGDesiredWriteBarriers.h:
* dfg/DFGFlushLivenessAnalysisPhase.cpp:
(JSC::DFG::FlushLivenessAnalysisPhase::run):
(JSC::DFG::FlushLivenessAnalysisPhase::process):
(JSC::DFG::FlushLivenessAnalysisPhase::reportError):
* dfg/DFGFlushedAt.cpp: Added.
(JSC::DFG::FlushedAt::dump):
(JSC::DFG::FlushedAt::dumpInContext):
* dfg/DFGFlushedAt.h: Added.
(JSC::DFG::FlushedAt::FlushedAt):
(JSC::DFG::FlushedAt::operator!):
(JSC::DFG::FlushedAt::format):
(JSC::DFG::FlushedAt::virtualRegister):
(JSC::DFG::FlushedAt::operator==):
(JSC::DFG::FlushedAt::operator!=):
* dfg/DFGGraph.cpp:
(JSC::DFG::Graph::Graph):
(JSC::DFG::Graph::dump):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::bytecodeRegisterForArgument):
(JSC::DFG::Graph::argumentsRegisterFor):
(JSC::DFG::Graph::machineArgumentsRegisterFor):
(JSC::DFG::Graph::uncheckedArgumentsRegisterFor):
(JSC::DFG::Graph::activationRegister):
(JSC::DFG::Graph::uncheckedActivationRegister):
(JSC::DFG::Graph::machineActivationRegister):
(JSC::DFG::Graph::uncheckedMachineActivationRegister):
* dfg/DFGJITCompiler.cpp:
(JSC::DFG::JITCompiler::link):
* dfg/DFGJITCompiler.h:
(JSC::DFG::JITCompiler::noticeOSREntry):
* dfg/DFGNode.h:
(JSC::DFG::Node::convertToGetLocalUnlinked):
(JSC::DFG::Node::convertToGetLocal):
(JSC::DFG::Node::machineLocal):
(JSC::DFG::Node::hasUnlinkedMachineLocal):
(JSC::DFG::Node::setUnlinkedMachineLocal):
(JSC::DFG::Node::unlinkedMachineLocal):
(JSC::DFG::Node::hasInlineStartData):
(JSC::DFG::Node::inlineStartData):
* dfg/DFGNodeFlags.cpp:
(JSC::DFG::dumpNodeFlags):
* dfg/DFGOSREntry.cpp:
(JSC::DFG::prepareOSREntry):
* dfg/DFGOSREntry.h:
(JSC::DFG::OSREntryReshuffling::OSREntryReshuffling):
* dfg/DFGOSRExitCompiler64.cpp:
(JSC::DFG::OSRExitCompiler::compileExit):
* dfg/DFGOSRExitCompilerCommon.cpp:
(JSC::DFG::reifyInlinedCallFrames):
* dfg/DFGOperations.cpp:
* dfg/DFGOperations.h:
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::compileInThreadImpl):
* dfg/DFGScoreBoard.h:
(JSC::DFG::ScoreBoard::ScoreBoard):
* dfg/DFGSpeculativeJIT.cpp:
(JSC::DFG::SpeculativeJIT::compileInlineStart):
(JSC::DFG::SpeculativeJIT::compileCurrentBlock):
(JSC::DFG::SpeculativeJIT::createOSREntries):
(JSC::DFG::SpeculativeJIT::compileGetByValOnArguments):
* dfg/DFGSpeculativeJIT.h:
(JSC::DFG::SpeculativeJIT::calleeFrameOffset):
(JSC::DFG::SpeculativeJIT::callFrameSlot):
(JSC::DFG::SpeculativeJIT::argumentSlot):
(JSC::DFG::SpeculativeJIT::callFrameTagSlot):
(JSC::DFG::SpeculativeJIT::callFramePayloadSlot):
(JSC::DFG::SpeculativeJIT::argumentTagSlot):
(JSC::DFG::SpeculativeJIT::argumentPayloadSlot):
(JSC::DFG::SpeculativeJIT::framePointerOffsetToGetActivationRegisters):
(JSC::DFG::SpeculativeJIT::callOperation):
(JSC::DFG::SpeculativeJIT::recordSetLocal):
* dfg/DFGSpeculativeJIT32_64.cpp:
(JSC::DFG::SpeculativeJIT::emitCall):
(JSC::DFG::SpeculativeJIT::compile):
* dfg/DFGSpeculativeJIT64.cpp:
(JSC::DFG::SpeculativeJIT::emitCall):
(JSC::DFG::SpeculativeJIT::compile):
* dfg/DFGStackLayoutPhase.cpp: Added.
(JSC::DFG::StackLayoutPhase::StackLayoutPhase):
(JSC::DFG::StackLayoutPhase::run):
(JSC::DFG::performStackLayout):
* dfg/DFGStackLayoutPhase.h: Added.
* dfg/DFGValidate.cpp:
(JSC::DFG::Validate::validate):
* dfg/DFGVariableAccessData.h:
(JSC::DFG::VariableAccessData::machineLocal):
(JSC::DFG::VariableAccessData::flushedAt):
* dfg/DFGVirtualRegisterAllocationPhase.cpp:
(JSC::DFG::VirtualRegisterAllocationPhase::run):
* ftl/FTLExitValue.h:
(JSC::FTL::ExitValue::inJSStack):
(JSC::FTL::ExitValue::inJSStackAsInt32):
(JSC::FTL::ExitValue::inJSStackAsInt52):
(JSC::FTL::ExitValue::inJSStackAsDouble):
(JSC::FTL::ExitValue::virtualRegister):
* ftl/FTLLowerDFGToLLVM.cpp:
(JSC::FTL::LowerDFGToLLVM::compileGetArgument):
(JSC::FTL::LowerDFGToLLVM::compileGetLocal):
(JSC::FTL::LowerDFGToLLVM::compileSetLocal):
(JSC::FTL::LowerDFGToLLVM::initializeOSRExitStateForBlock):
(JSC::FTL::LowerDFGToLLVM::emitOSRExitCall):
* ftl/FTLOSRExitCompiler.cpp:
(JSC::FTL::compileStub):
* ftl/FTLValueSource.cpp:
(JSC::FTL::ValueSource::dump):
* ftl/FTLValueSource.h:
(JSC::FTL::ValueSource::ValueSource):
(JSC::FTL::ValueSource::kind):
(JSC::FTL::ValueSource::operator!):
(JSC::FTL::ValueSource::node):
(JSC::FTL::ValueSource::virtualRegister):
* interpreter/Interpreter.cpp:
(JSC::unwindCallFrame):
* interpreter/StackVisitor.cpp:
(JSC::StackVisitor::readInlinedFrame):
(JSC::StackVisitor::Frame::createArguments):
(JSC::StackVisitor::Frame::existingArguments):
* interpreter/StackVisitor.h:
* jit/AssemblyHelpers.h:
(JSC::AssemblyHelpers::addressFor):
(JSC::AssemblyHelpers::tagFor):
(JSC::AssemblyHelpers::payloadFor):
(JSC::AssemblyHelpers::offsetOfArgumentsIncludingThis):
* runtime/Arguments.cpp:
(JSC::Arguments::tearOff):
* runtime/Arguments.h:
(JSC::Arguments::allocateSlowArguments):
(JSC::Arguments::tryDeleteArgument):
(JSC::Arguments::isDeletedArgument):
(JSC::Arguments::isArgument):
(JSC::Arguments::argument):
(JSC::Arguments::finishCreation):
* runtime/JSActivation.h:
(JSC::JSActivation::create):
(JSC::JSActivation::JSActivation):
* runtime/JSFunction.cpp:
(JSC::RetrieveArgumentsFunctor::operator()):



git-svn-id: http://svn.webkit.org/repository/webkit/trunk@156984 268f45cc-cd09-0410-ab3c-d52691b4dbfc
parent 1172d37e
......@@ -121,6 +121,7 @@ set(JavaScriptCore_SOURCES
dfg/DFGFixupPhase.cpp
dfg/DFGFlushFormat.cpp
dfg/DFGFlushLivenessAnalysisPhase.cpp
dfg/DFGFlushedAt.cpp
dfg/DFGGraph.cpp
dfg/DFGInPlaceAbstractState.cpp
dfg/DFGJITCode.cpp
......@@ -155,6 +156,7 @@ set(JavaScriptCore_SOURCES
dfg/DFGSpeculativeJIT.cpp
dfg/DFGSpeculativeJIT32_64.cpp
dfg/DFGSpeculativeJIT64.cpp
dfg/DFGStackLayoutPhase.cpp
dfg/DFGThunks.cpp
dfg/DFGTierUpCheckInjectionPhase.cpp
dfg/DFGTypeCheckHoistingPhase.cpp
......
This diff is collapsed.
......@@ -272,6 +272,8 @@ javascriptcore_sources += \
Source/JavaScriptCore/dfg/DFGFlushFormat.h \
Source/JavaScriptCore/dfg/DFGFlushLivenessAnalysisPhase.cpp \
Source/JavaScriptCore/dfg/DFGFlushLivenessAnalysisPhase.h \
Source/JavaScriptCore/dfg/DFGFlushedAt.cpp \
Source/JavaScriptCore/dfg/DFGFlushedAt.h \
Source/JavaScriptCore/dfg/DFGGenerationInfo.h \
Source/JavaScriptCore/dfg/DFGGenerationInfo.h \
Source/JavaScriptCore/dfg/DFGGraph.cpp \
......@@ -351,6 +353,8 @@ javascriptcore_sources += \
Source/JavaScriptCore/dfg/DFGSpeculativeJIT.h \
Source/JavaScriptCore/dfg/DFGSSAConversionPhase.cpp \
Source/JavaScriptCore/dfg/DFGSSAConversionPhase.h \
Source/JavaScriptCore/dfg/DFGStackLayoutPhase.cpp \
Source/JavaScriptCore/dfg/DFGStackLayoutPhase.h \
Source/JavaScriptCore/dfg/DFGStructureAbstractValue.h \
Source/JavaScriptCore/dfg/DFGTierUpCheckInjectionPhase.cpp \
Source/JavaScriptCore/dfg/DFGTierUpCheckInjectionPhase.h \
......
......@@ -312,6 +312,10 @@
0F98206016BFE38100240D02 /* PreciseJumpTargets.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F98205D16BFE37F00240D02 /* PreciseJumpTargets.cpp */; };
0F98206116BFE38300240D02 /* PreciseJumpTargets.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F98205E16BFE37F00240D02 /* PreciseJumpTargets.h */; settings = {ATTRIBUTES = (Private, ); }; };
0F9D3370165DBB90005AD387 /* Disassembler.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F9D336E165DBB8D005AD387 /* Disassembler.cpp */; };
0F9D339617FFC4E60073C2BC /* DFGFlushedAt.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F9D339417FFC4E60073C2BC /* DFGFlushedAt.cpp */; };
0F9D339717FFC4E60073C2BC /* DFGFlushedAt.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F9D339517FFC4E60073C2BC /* DFGFlushedAt.h */; settings = {ATTRIBUTES = (Private, ); }; };
0F9FB4F417FCB91700CB67F8 /* DFGStackLayoutPhase.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F9FB4F217FCB91700CB67F8 /* DFGStackLayoutPhase.cpp */; };
0F9FB4F517FCB91700CB67F8 /* DFGStackLayoutPhase.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F9FB4F317FCB91700CB67F8 /* DFGStackLayoutPhase.h */; settings = {ATTRIBUTES = (Private, ); }; };
0F9FC8C314E1B5FE00D52AE0 /* PolymorphicPutByIdList.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F9FC8BF14E1B5FB00D52AE0 /* PolymorphicPutByIdList.cpp */; };
0F9FC8C414E1B60000D52AE0 /* PolymorphicPutByIdList.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F9FC8C014E1B5FB00D52AE0 /* PolymorphicPutByIdList.h */; settings = {ATTRIBUTES = (Private, ); }; };
0F9FC8C514E1B60400D52AE0 /* PutKind.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F9FC8C114E1B5FB00D52AE0 /* PutKind.h */; settings = {ATTRIBUTES = (Private, ); }; };
......@@ -1522,6 +1526,10 @@
0F98205D16BFE37F00240D02 /* PreciseJumpTargets.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = PreciseJumpTargets.cpp; sourceTree = "<group>"; };
0F98205E16BFE37F00240D02 /* PreciseJumpTargets.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = PreciseJumpTargets.h; sourceTree = "<group>"; };
0F9D336E165DBB8D005AD387 /* Disassembler.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = Disassembler.cpp; path = disassembler/Disassembler.cpp; sourceTree = "<group>"; };
0F9D339417FFC4E60073C2BC /* DFGFlushedAt.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = DFGFlushedAt.cpp; path = dfg/DFGFlushedAt.cpp; sourceTree = "<group>"; };
0F9D339517FFC4E60073C2BC /* DFGFlushedAt.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGFlushedAt.h; path = dfg/DFGFlushedAt.h; sourceTree = "<group>"; };
0F9FB4F217FCB91700CB67F8 /* DFGStackLayoutPhase.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = DFGStackLayoutPhase.cpp; path = dfg/DFGStackLayoutPhase.cpp; sourceTree = "<group>"; };
0F9FB4F317FCB91700CB67F8 /* DFGStackLayoutPhase.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGStackLayoutPhase.h; path = dfg/DFGStackLayoutPhase.h; sourceTree = "<group>"; };
0F9FC8BF14E1B5FB00D52AE0 /* PolymorphicPutByIdList.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = PolymorphicPutByIdList.cpp; sourceTree = "<group>"; };
0F9FC8C014E1B5FB00D52AE0 /* PolymorphicPutByIdList.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = PolymorphicPutByIdList.h; sourceTree = "<group>"; };
0F9FC8C114E1B5FB00D52AE0 /* PutKind.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = PutKind.h; sourceTree = "<group>"; };
......@@ -3489,6 +3497,8 @@
86EC9DB31328DF44002B2AD7 /* dfg */ = {
isa = PBXGroup;
children = (
0F9D339417FFC4E60073C2BC /* DFGFlushedAt.cpp */,
0F9D339517FFC4E60073C2BC /* DFGFlushedAt.h */,
A77A423617A0BBFD00A8DB81 /* DFGAbstractHeap.cpp */,
A77A423717A0BBFD00A8DB81 /* DFGAbstractHeap.h */,
A704D8FE17A0BAA8006BA554 /* DFGAbstractInterpreter.h */,
......@@ -3659,6 +3669,8 @@
86880F4C14353B2100B08D42 /* DFGSpeculativeJIT64.cpp */,
A7D89CF017A0B8CC00773AD8 /* DFGSSAConversionPhase.cpp */,
A7D89CF117A0B8CC00773AD8 /* DFGSSAConversionPhase.h */,
0F9FB4F217FCB91700CB67F8 /* DFGStackLayoutPhase.cpp */,
0F9FB4F317FCB91700CB67F8 /* DFGStackLayoutPhase.h */,
0F63947615DCE347006A597C /* DFGStructureAbstractValue.h */,
0FC0979F146B28C700CF2442 /* DFGThunks.cpp */,
0FC097A0146B28C700CF2442 /* DFGThunks.h */,
......@@ -4347,6 +4359,7 @@
86AE64A9135E5E1C00963012 /* MacroAssemblerSH4.h in Headers */,
860161E40F3A83C100F84710 /* MacroAssemblerX86.h in Headers */,
860161E50F3A83C100F84710 /* MacroAssemblerX86_64.h in Headers */,
0F9FB4F517FCB91700CB67F8 /* DFGStackLayoutPhase.h in Headers */,
860161E60F3A83C100F84710 /* MacroAssemblerX86Common.h in Headers */,
A700873A17CBE85300C3E643 /* MapConstructor.h in Headers */,
A78507D717CBC6FD0011F6E7 /* MapData.h in Headers */,
......@@ -4415,6 +4428,7 @@
0FB1058E1675483A00F8AB6E /* ProfilerOSRExitSite.h in Headers */,
0F13912C16771C3D009CCB07 /* ProfilerProfiledBytecodes.h in Headers */,
0F24E54217EA9F5900ABB217 /* CCallHelpers.h in Headers */,
0F9D339717FFC4E60073C2BC /* DFGFlushedAt.h in Headers */,
A7FB61001040C38B0017A286 /* PropertyDescriptor.h in Headers */,
BC95437D0EBA70FD0072B6D3 /* PropertyMapHashTable.h in Headers */,
86158AB3155C8B4000B45C9C /* PropertyName.h in Headers */,
......@@ -4940,6 +4954,7 @@
A7E5A3A71797432D00E893C0 /* CompilationResult.cpp in Sources */,
147F39C2107EC37600427A48 /* Completion.cpp in Sources */,
146B16D812EB5B59001BEC1B /* ConservativeRoots.cpp in Sources */,
0F9D339617FFC4E60073C2BC /* DFGFlushedAt.cpp in Sources */,
1428082E107EC0570013E7B2 /* ConstructData.cpp in Sources */,
C240305514B404E60079EB64 /* CopiedSpace.cpp in Sources */,
C2239D1716262BDD005AC5FD /* CopyVisitor.cpp in Sources */,
......@@ -5171,6 +5186,7 @@
1482B74E0A43032800517CFC /* JSStringRef.cpp in Sources */,
146AAB380B66A94400E55F16 /* JSStringRefCF.cpp in Sources */,
0F919D0C157EE09F004A4E7D /* JSSymbolTableObject.cpp in Sources */,
0F9FB4F417FCB91700CB67F8 /* DFGStackLayoutPhase.cpp in Sources */,
0F2B66FA17B6B5AB00A7AE3F /* JSTypedArrayConstructors.cpp in Sources */,
0F2B66FC17B6B5AB00A7AE3F /* JSTypedArrayPrototypes.cpp in Sources */,
0F2B66FE17B6B5AB00A7AE3F /* JSTypedArrays.cpp in Sources */,
......
......@@ -1887,6 +1887,7 @@ CodeBlock::CodeBlock(ScriptExecutable* ownerExecutable, UnlinkedCodeBlock* unlin
if (Options::showDisassembly()
|| Options::showDFGDisassembly()
|| Options::dumpBytecodeAtDFGTime()
|| Options::dumpGraphAtEachPhase()
|| Options::verboseCompilation()
|| Options::logCompilationChanges()
|| Options::validateGraph()
......@@ -2483,6 +2484,66 @@ bool CodeBlock::hasOptimizedReplacement()
}
#endif
bool CodeBlock::isCaptured(VirtualRegister operand, InlineCallFrame* inlineCallFrame) const
{
if (operand.isArgument())
return operand.toArgument() && usesArguments();
if (inlineCallFrame)
return inlineCallFrame->capturedVars.get(operand.toLocal());
// The activation object isn't in the captured region, but it's "captured"
// in the sense that stores to its location can be observed indirectly.
if (needsActivation() && operand == activationRegister())
return true;
// Ditto for the arguments object.
if (usesArguments() && operand == argumentsRegister())
return true;
// Ditto for the arguments object.
if (usesArguments() && operand == unmodifiedArgumentsRegister(argumentsRegister()))
return true;
// We're in global code so there are no locals to capture
if (!symbolTable())
return false;
return operand.offset() <= symbolTable()->captureStart()
&& operand.offset() > symbolTable()->captureEnd();
}
int CodeBlock::framePointerOffsetToGetActivationRegisters(int machineCaptureStart)
{
// We'll be adding this to the stack pointer to get a registers pointer that looks
// like it would have looked in the baseline engine. For example, if bytecode would
// have put the first captured variable at offset -5 but we put it at offset -1, then
// we'll have an offset of 4.
int32_t offset = 0;
// Compute where we put the captured variables. This offset will point the registers
// pointer directly at the first captured var.
offset += machineCaptureStart;
// Now compute the offset needed to make the runtime see the captured variables at the
// same offset that the bytecode would have used.
offset -= symbolTable()->captureStart();
return offset;
}
int CodeBlock::framePointerOffsetToGetActivationRegisters()
{
if (!JITCode::isOptimizingJIT(jitType()))
return 0;
#if ENABLE(DFG_JIT)
return framePointerOffsetToGetActivationRegisters(jitCode()->dfgCommon()->machineCaptureStart);
#else
RELEASE_ASSERT_NOT_REACHED();
return 0;
#endif
}
HandlerInfo* CodeBlock::handlerForBytecodeOffset(unsigned bytecodeOffset)
{
RELEASE_ASSERT(bytecodeOffset < instructions().size());
......@@ -2683,6 +2744,18 @@ PassRefPtr<CodeBlock> CodeBlock::newReplacement()
return ownerExecutable()->newReplacementCodeBlockFor(specializationKind());
}
const SlowArgument* CodeBlock::machineSlowArguments()
{
if (!JITCode::isOptimizingJIT(jitType()))
return symbolTable()->slowArguments();
#if ENABLE(DFG_JIT)
return jitCode()->dfgCommon()->slowArguments.get();
#else // ENABLE(DFG_JIT)
return 0;
#endif // ENABLE(DFG_JIT)
}
#if ENABLE(JIT)
void CodeBlock::reoptimize()
{
......
......@@ -245,6 +245,9 @@ public:
unsigned instructionCount() { return m_instructions.size(); }
int argumentIndexAfterCapture(size_t argument);
bool hasSlowArguments();
const SlowArgument* machineSlowArguments();
// Exactly equivalent to codeBlock->ownerExecutable()->installCode(codeBlock);
void install();
......@@ -347,34 +350,10 @@ public:
return m_needsActivation;
}
bool isCaptured(VirtualRegister operand, InlineCallFrame* inlineCallFrame = 0) const
{
if (operand.isArgument())
return operand.toArgument() && usesArguments();
if (inlineCallFrame)
return inlineCallFrame->capturedVars.get(operand.toLocal());
// The activation object isn't in the captured region, but it's "captured"
// in the sense that stores to its location can be observed indirectly.
if (needsActivation() && operand == activationRegister())
return true;
// Ditto for the arguments object.
if (usesArguments() && operand == argumentsRegister())
return true;
// Ditto for the arguments object.
if (usesArguments() && operand == unmodifiedArgumentsRegister(argumentsRegister()))
return true;
// We're in global code so there are no locals to capture
if (!symbolTable())
return false;
return operand.offset() <= symbolTable()->captureStart()
&& operand.offset() > symbolTable()->captureEnd();
}
bool isCaptured(VirtualRegister operand, InlineCallFrame* = 0) const;
int framePointerOffsetToGetActivationRegisters(int machineCaptureStart);
int framePointerOffsetToGetActivationRegisters();
CodeType codeType() const { return m_unlinkedCode->codeType(); }
PutPropertySlot::Context putByIdContext() const
......@@ -1225,6 +1204,11 @@ inline int CodeBlock::argumentIndexAfterCapture(size_t argument)
return slowArguments[argument].index;
}
inline bool CodeBlock::hasSlowArguments()
{
return !!symbolTable()->slowArguments();
}
inline Register& ExecState::r(int index)
{
CodeBlock* codeBlock = this->codeBlock();
......
......@@ -66,7 +66,7 @@ void CodeOrigin::dump(PrintStream& out) const
if (InlineCallFrame* frame = stack[i].inlineCallFrame) {
out.print(frame->briefFunctionInformation(), ":<", RawPointer(frame->executable.get()), "> ");
if (frame->isClosureCall())
if (frame->isClosureCall)
out.print("(closure) ");
}
......@@ -81,10 +81,7 @@ void CodeOrigin::dumpInContext(PrintStream& out, DumpContext*) const
JSFunction* InlineCallFrame::calleeForCallFrame(ExecState* exec) const
{
if (!isClosureCall())
return callee.get();
return jsCast<JSFunction*>((exec + stackOffset)->callee());
return jsCast<JSFunction*>(calleeRecovery.recover(exec));
}
CodeBlockHash InlineCallFrame::hash() const
......@@ -114,10 +111,10 @@ void InlineCallFrame::dumpInContext(PrintStream& out, DumpContext* context) cons
if (executable->isStrictMode())
out.print(" (StrictMode)");
out.print(", bc#", caller.bytecodeIndex, ", ", specializationKind());
if (callee)
out.print(", known callee: ", inContext(JSValue(callee.get()), context));
else
if (isClosureCall)
out.print(", closure call");
else
out.print(", known callee: ", inContext(calleeRecovery.constant(), context));
out.print(", numArgs+this = ", arguments.size());
out.print(", stack >= r", stackOffset);
out.print(">");
......
......@@ -28,6 +28,7 @@
#include "CodeBlockHash.h"
#include "CodeSpecializationKind.h"
#include "JSFunction.h"
#include "ValueRecovery.h"
#include "WriteBarrier.h"
#include <wtf/BitVector.h>
......@@ -94,11 +95,13 @@ struct CodeOrigin {
struct InlineCallFrame {
Vector<ValueRecovery> arguments;
WriteBarrier<ScriptExecutable> executable;
WriteBarrier<JSFunction> callee; // This may be null, indicating that this is a closure call and that the JSFunction and JSScope are already on the stack.
ValueRecovery calleeRecovery;
CodeOrigin caller;
BitVector capturedVars; // Indexed by the machine call frame's variable numbering.
signed stackOffset : 31;
signed stackOffset : 30;
bool isCall : 1;
bool isClosureCall : 1; // If false then we know that callee/scope are constants and the DFG won't treat them as variables, i.e. they have to be recovered manually.
VirtualRegister argumentsRegister; // This is only set if the code uses arguments. The unmodified arguments register follows the unmodifiedArgumentsRegister() convention (see CodeBlock.h).
// There is really no good notion of a "default" set of values for
// InlineCallFrame's fields. This constructor is here just to reduce confusion if
......@@ -106,12 +109,18 @@ struct InlineCallFrame {
InlineCallFrame()
: stackOffset(0)
, isCall(false)
, isClosureCall(false)
{
}
CodeSpecializationKind specializationKind() const { return specializationFromIsCall(isCall); }
bool isClosureCall() const { return !callee; }
JSFunction* calleeConstant() const
{
if (calleeRecovery.isConstant())
return jsCast<JSFunction*>(calleeRecovery.constant());
return 0;
}
// Get the callee given a machine call frame to which this InlineCallFrame belongs.
JSFunction* calleeForCallFrame(ExecState*) const;
......
......@@ -209,6 +209,16 @@ public:
return virtualRegisterForArgument(index).offset();
return virtualRegisterForLocal(index - numberOfArguments()).offset();
}
size_t indexForOperand(int operand) const
{
if (operandIsArgument(operand))
return static_cast<size_t>(VirtualRegister(operand).toArgument());
return static_cast<size_t>(VirtualRegister(operand).toLocal()) + numberOfArguments();
}
size_t indexForOperand(VirtualRegister reg) const
{
return indexForOperand(reg.offset());
}
void setOperandFirstTime(int operand, const T& value)
{
......
......@@ -109,8 +109,8 @@ void BasicBlock::dump(PrintStream& out) const
}
BasicBlock::SSAData::SSAData(BasicBlock* block)
: flushFormatAtHead(OperandsLike, block->variablesAtHead)
, flushFormatAtTail(OperandsLike, block->variablesAtHead)
: flushAtHead(OperandsLike, block->variablesAtHead)
, flushAtTail(OperandsLike, block->variablesAtHead)
, availabilityAtHead(OperandsLike, block->variablesAtHead)
, availabilityAtTail(OperandsLike, block->variablesAtHead)
{
......
......@@ -30,6 +30,7 @@
#include "DFGAbstractValue.h"
#include "DFGBranchDirection.h"
#include "DFGFlushedAt.h"
#include "DFGNode.h"
#include "DFGVariadicFunction.h"
#include "Operands.h"
......@@ -137,8 +138,8 @@ struct BasicBlock : RefCounted<BasicBlock> {
unsigned innerMostLoopIndices[numberOfInnerMostLoopIndices];
struct SSAData {
Operands<FlushFormat> flushFormatAtHead;
Operands<FlushFormat> flushFormatAtTail;
Operands<FlushedAt> flushAtHead;
Operands<FlushedAt> flushAtTail;
Operands<Node*> availabilityAtHead;
Operands<Node*> availabilityAtTail;
HashSet<Node*> liveAtHead;
......
......@@ -136,7 +136,6 @@ public:
, m_constants(m_codeBlock->numberOfConstantRegisters())
, m_numArguments(m_codeBlock->numParameters())
, m_numLocals(m_codeBlock->m_numCalleeRegisters)
, m_preservedVars(m_codeBlock->m_numVars)
, m_parameterSlots(0)
, m_numPassedVarArgs(0)
, m_inlineStackTop(0)
......@@ -145,9 +144,6 @@ public:
, m_currentInstruction(0)
{
ASSERT(m_profiledBlock);
for (int i = 0; i < m_codeBlock->m_numVars; ++i)
m_preservedVars.set(i);
}
// Parse a full CodeBlock of bytecode.
......@@ -220,7 +216,8 @@ private:
Node* get(VirtualRegister operand)
{
if (inlineCallFrame()) {
if (JSFunction* callee = inlineCallFrame()->callee.get()) {
if (!inlineCallFrame()->isClosureCall) {
JSFunction* callee = inlineCallFrame()->calleeConstant();
if (operand.offset() == JSStack::Callee)
return cellConstant(callee);
if (operand.offset() == JSStack::ScopeChain)
......@@ -293,10 +290,8 @@ private:
break;
}
}
} else {
m_preservedVars.set(local);
} else
variable = newVariableAccessData(operand, isCaptured);
}
node = injectLazyOperandSpeculation(addToGraph(GetLocal, OpInfo(variable)));
m_currentBlock->variablesAtTail.local(local) = node;
......@@ -441,9 +436,6 @@ private:
ASSERT(!operand.isConstant());
if (operand.isLocal())
m_preservedVars.set(operand.toLocal());
Node* node = m_currentBlock->variablesAtTail.operand(operand);
VariableAccessData* variable;
......@@ -465,7 +457,7 @@ private:
int numArguments;
if (InlineCallFrame* inlineCallFrame = inlineStackEntry->m_inlineCallFrame) {
numArguments = inlineCallFrame->arguments.size();
if (!inlineCallFrame->callee) {
if (inlineCallFrame->isClosureCall) {
flushDirect(inlineStackEntry->remapOperand(VirtualRegister(JSStack::Callee)));
flushDirect(inlineStackEntry->remapOperand(VirtualRegister(JSStack::ScopeChain)));
}
......@@ -1005,10 +997,6 @@ private:
unsigned m_numArguments;
// The number of locals (vars + temporaries) used in the function.
unsigned m_numLocals;
// The set of registers we need to preserve across BasicBlock boundaries;
// typically equal to the set of vars, but we expand this to cover all
// temporaries that persist across blocks (dues to ?:, &&, ||, etc).
BitVector m_preservedVars;
// The number of slots (in units of sizeof(Register)) that we need to
// preallocate for calls emanating from this frame. This includes the
// size of the CallFrame, only if this is not a leaf function. (I.e.
......@@ -1295,10 +1283,6 @@ bool ByteCodeParser::handleInlining(Node* callTargetNode, int resultOperand, con
int inlineCallFrameStart = m_inlineStackTop->remapOperand(VirtualRegister(registerOffset)).offset() + JSStack::CallFrameHeaderSize;
// Make sure that the area used by the call frame is reserved.
for (int arg = VirtualRegister(inlineCallFrameStart).toLocal() + JSStack::CallFrameHeaderSize + codeBlock->m_numVars; arg-- > VirtualRegister(inlineCallFrameStart).toLocal();)
m_preservedVars.set(arg);
// Make sure that we have enough locals.
unsigned newNumLocals = VirtualRegister(inlineCallFrameStart).toLocal() + JSStack::CallFrameHeaderSize + codeBlock->m_numCalleeRegisters;
if (newNumLocals > m_numLocals) {
......@@ -1318,7 +1302,14 @@ bool ByteCodeParser::handleInlining(Node* callTargetNode, int resultOperand, con
unsigned oldIndex = m_currentIndex;
m_currentIndex = 0;
addToGraph(InlineStart, OpInfo(argumentPositionStart));
InlineStartData* inlineStartData = &m_graph.m_inlineStartData.alloc();
inlineStartData->argumentPositionStart = argumentPositionStart;
inlineStartData->calleeVariable = 0;
addToGraph(InlineStart, OpInfo(inlineStartData));
RELEASE_ASSERT(
m_inlineStackTop->m_inlineCallFrame->isClosureCall
== callLinkStatus.isClosureCall());
if (callLinkStatus.isClosureCall()) {
VariableAccessData* calleeVariable =
set(VirtualRegister(JSStack::Callee), callTargetNode)->variableAccessData();
......@@ -1327,6 +1318,8 @@ bool ByteCodeParser::handleInlining(Node* callTargetNode, int resultOperand, con
calleeVariable->mergeShouldNeverUnbox(true);
scopeVariable->mergeShouldNeverUnbox(true);
inlineStartData->calleeVariable = calleeVariable;
}
parseCodeBlock();
......@@ -3376,14 +3369,10 @@ ByteCodeParser::InlineStackEntry::InlineStackEntry(
codeBlock->ownerExecutable());
m_inlineCallFrame->stackOffset = inlineCallFrameStart.offset() - JSStack::CallFrameHeaderSize;
if (callee) {
initializeLazyWriteBarrierForInlineCallFrameCallee(
byteCodeParser->m_graph.m_plan.writeBarriers,
m_inlineCallFrame->callee,
byteCodeParser->m_codeBlock,
m_inlineCallFrame,
byteCodeParser->m_codeBlock->ownerExecutable(),
callee);
}
m_inlineCallFrame->calleeRecovery = ValueRecovery::constant(callee);
m_inlineCallFrame->isClosureCall = false;
} else
m_inlineCallFrame->isClosureCall = true;
m_inlineCallFrame->caller = byteCodeParser->currentCodeOrigin();
m_inlineCallFrame->arguments.resize(argumentCountIncludingThis); // Set the number of arguments including this, but don't configure the value recoveries, yet.
m_inlineCallFrame->isCall = isCall(kind);
......@@ -3630,29 +3619,18 @@ bool ByteCodeParser::parse()
linkBlocks(inlineStackEntry.m_unlinkedBlocks, inlineStackEntry.m_blockLinkingTargets);
m_graph.determineReachability();
m_graph.killUnreachableBlocks();
ASSERT(m_preservedVars.size());
size_t numberOfLocals = 0;
for (size_t i = m_preservedVars.size(); i--;) {
if (m_preservedVars.quickGet(i)) {
numberOfLocals = i + 1;
break;
}
}
for (BlockIndex blockIndex = 0; blockIndex < m_graph.numBlocks(); ++blockIndex) {
for (BlockIndex blockIndex = m_graph.numBlocks(); blockIndex--;) {
BasicBlock* block = m_graph.block(blockIndex);
ASSERT(block);
if (!block->isReachable) {
m_graph.killBlockAndItsContents(block);
if (!block)
continue;
}
block->variablesAtHead.ensureLocals(numberOfLocals);
block->variablesAtTail.ensureLocals(numberOfLocals);
ASSERT(block->variablesAtHead.numberOfLocals() == m_graph.block(0)->variablesAtHead.numberOfLocals());
ASSERT(block->variablesAtHead.numberOfArguments() == m_graph.block(0)->variablesAtHead.numberOfArguments());
ASSERT(block->variablesAtTail.numberOfLocals() == m_graph.block(0)->variablesAtHead.numberOfLocals());
ASSERT(block->variablesAtTail.numberOfArguments() == m_graph.block(0)->variablesAtHead.numberOfArguments());
}
m_graph.m_preservedVars = m_preservedVars;
m_graph.m_localVars = m_numLocals;
m_graph.m_parameterSlots = m_parameterSlots;
......
......@@ -64,8 +64,6 @@
#define DFG_ENABLE_JIT_BREAK_ON_SPECULATION_FAILURE 0
// Disable the DFG JIT without having to touch Platform.h
#define DFG_DEBUG_LOCAL_DISBALE 0
// Enable OSR entry from baseline JIT.
#define DFG_ENABLE_OSR_ENTRY ENABLE(DFG_JIT)
// Generate stats on how successful we were in making use of the DFG jit, and remaining on the hot path.
#define DFG_ENABLE_SUCCESS_STATS 0
......
......@@ -33,6 +33,7 @@
#include "InlineCallFrameSet.h"
#include "JSCell.h"
#include "ProfilerCompilation.h"
#include "SymbolTable.h"
#include <wtf/Noncopyable.h>
namespace JSC {
......@@ -67,7 +68,9 @@ struct WeakReferenceTransition {
class CommonData {
WTF_MAKE_NONCOPYABLE(CommonData);
public:
CommonData() { }
CommonData()
: machineCaptureStart(std::numeric_limits<int>::max())
{ }
void notifyCompilingStructureTransition(Plan&, CodeBlock*, Node*);
unsigned addCodeOrigin(CodeOrigin codeOrigin);
......@@ -84,6 +87,9 @@ public:
RefPtr<Profiler::Compilation> compilation;
bool livenessHasBeenProved; // Initialized and used on every GC.
bool allTransitionsHaveBeenMarked; // Initialized and used on every GC.
int machineCaptureStart;
std::unique_ptr<SlowArgument[]> slowArguments;
};
} } // namespace JSC::DFG
......
......@@ -64,14 +64,6 @@ void DesiredWriteBarrier::trigger(VM& vm)
WriteBarrier<ScriptExecutable>& executable = inlineCallFrame->executable;
executable.set(vm, m_owner, executable.get());
return;
}
case InlineCallFrameCalleeType: {
InlineCallFrame* inlineCallFrame = m_which.inlineCallFrame;
ASSERT(!!inlineCallFrame->callee);
WriteBarrier<JSFunction>& callee = inlineCallFrame->callee;
callee.set(vm, m_owner, callee.get());
return;
} }
RELEASE_ASSERT_NOT_REACHED();
}
......
......@@ -42,7 +42,10 @@ namespace DFG {
class DesiredWriteBarrier {
public:
enum Type { ConstantType, InlineCallFrameExecutableType, InlineCallFrameCalleeType };
enum Type {
ConstantType,
InlineCallFrameExecutableType,
};
DesiredWriteBarrier(Type, CodeBlock*, unsigned index, JSCell* owner);
DesiredWriteBarrier(Type, CodeBlock*, InlineCallFrame*, JSCell* owner);
......@@ -86,12 +89,6 @@ inline void initializeLazyWriteBarrierForInlineCallFrameExecutable(DesiredWriteB
barrier = WriteBarrier<ScriptExecutable>(desiredBarrier, value);
}
inline void initializeLazyWriteBarrierForInlineCallFrameCallee(DesiredWriteBarriers& barriers, WriteBarrier<JSFunction>& barrier, CodeBlock* codeBlock, InlineCallFrame* inlineCallFrame, JSCell* owner, JSFunction* value)
{
DesiredWriteBarrier& desiredBarrier = barriers.add(DesiredWriteBarrier::InlineCallFrameCalleeType, codeBlock, inlineCallFrame, owner);
barrier = WriteBarrier<JSFunction>(desiredBarrier, value);
}
inline void initializeLazyWriteBarrierForConstant(DesiredWriteBarriers& barriers, WriteBarrier<Unknown>& barrier, CodeBlock* codeBlock, unsigned index, JSCell* owner, JSValue value)
{