Commit ea92c209 authored by fpizlo@apple.com's avatar fpizlo@apple.com

FTL should be able to do simple OSR exits using llvm.webkit.stackmap

https://bugs.webkit.org/show_bug.cgi?id=122538

Reviewed by Oliver Hunt.
        
This gives the FTL the ability to OSR exit using the llvm.webkit.stackmap intrinsic.
        
- The FTL compiles all OSR exit calls as calls to llvm.webkit.stackmap with a unique
  ID, passing a requested size that is big enough for own jump replacement.
        
- After LLVM compilation, we parse the new LLVM stackmap section.
        
- For all llvm.webkit.stackmaps that we used for OSR exits, we do a jumpReplacement,
  which targets exit thunks that we generate.
        
- If an exit thunk fires, it causes JSC to compile an exit off-ramp that uses a
  combination of the JSC-internal OSR exit accounting (FTL::ExitValue and friends) and
  LLVM stackmap's accounting of where data actually ended up (register, indirect,
  constant) to reconstruct bytecode state.
        
This still has shortcomings; for example it cannot handle XMM or YMM registers. Handling
YMM registers will require adding some basic YMM support to our assemblers - really we
just need the ability to move a YMM's value into a GPR.
        
This patch preserves all of the old, intrinsic-less, FTL OSR exit support. Hence it
manages to pass all existing FTL tests even despite its incompleteness. I think that's
the right way to go since this is already a big patch, and anyway it would be great to
keep the intrinsic-less FTL OSR exit support so long as the LLVM side of this hasn't
landed.

* JavaScriptCore.xcodeproj/project.pbxproj:
* assembler/AbstractMacroAssembler.h:
(JSC::AbstractMacroAssembler::firstRegister):
(JSC::AbstractMacroAssembler::lastRegister):
* assembler/MacroAssembler.h:
(JSC::MacroAssembler::isStackRelated):
(JSC::MacroAssembler::firstRealRegister):
(JSC::MacroAssembler::nextRegister):
(JSC::MacroAssembler::secondRealRegister):
* assembler/MacroAssemblerX86Common.h:
* assembler/X86Assembler.h:
(JSC::X86Assembler::firstRegister):
(JSC::X86Assembler::lastRegister):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::compileInThreadImpl):
* ftl/FTLCArgumentGetter.cpp:
(JSC::FTL::CArgumentGetter::loadNextAndBox):
* ftl/FTLCArgumentGetter.h:
(JSC::FTL::CArgumentGetter::loadNextDoubleIntoGPR):
* ftl/FTLCompile.cpp:
(JSC::FTL::mmAllocateCodeSection):
(JSC::FTL::mmAllocateDataSection):
(JSC::FTL::dumpDataSection):
(JSC::FTL::fixFunctionBasedOnStackMaps):
(JSC::FTL::compile):
* ftl/FTLExitThunkGenerator.cpp:
(JSC::FTL::ExitThunkGenerator::emitThunk):
(JSC::FTL::ExitThunkGenerator::emitThunks):
* ftl/FTLExitThunkGenerator.h:
* ftl/FTLExitValue.h:
(JSC::FTL::ExitValue::isInJSStackSomehow):
(JSC::FTL::ExitValue::valueFormat):
* ftl/FTLFail.cpp:
(JSC::FTL::fail):
* ftl/FTLIntrinsicRepository.h:
* ftl/FTLJITCode.h:
* ftl/FTLLowerDFGToLLVM.cpp:
(JSC::FTL::generateExitThunks):
(JSC::FTL::LowerDFGToLLVM::LowerDFGToLLVM):
(JSC::FTL::LowerDFGToLLVM::appendOSRExit):
(JSC::FTL::LowerDFGToLLVM::emitOSRExitCall):
(JSC::FTL::LowerDFGToLLVM::linkOSRExitsAndCompleteInitializationBlocks):
* ftl/FTLOSRExit.h:
* ftl/FTLOSRExitCompilationInfo.h:
(JSC::FTL::OSRExitCompilationInfo::OSRExitCompilationInfo):
* ftl/FTLOSRExitCompiler.cpp:
(JSC::FTL::compileStubWithOSRExitStackmap):
(JSC::FTL::compileStubWithoutOSRExitStackmap):
(JSC::FTL::compileFTLOSRExit):
* ftl/FTLSaveRestore.cpp: Added.
(JSC::FTL::bytesForGPRs):
(JSC::FTL::requiredScratchMemorySizeInBytes):
(JSC::FTL::offsetOfGPR):
(JSC::FTL::saveAllRegisters):
(JSC::FTL::restoreAllRegisters):
* ftl/FTLSaveRestore.h: Added.
* ftl/FTLStackMaps.cpp: Added.
(JSC::FTL::readObject):
(JSC::FTL::StackMaps::Constant::parse):
(JSC::FTL::StackMaps::Constant::dump):
(JSC::FTL::StackMaps::Location::parse):
(JSC::FTL::StackMaps::Location::dump):
(JSC::FTL::StackMaps::Location::involvesGPR):
(JSC::FTL::StackMaps::Location::isGPR):
(JSC::FTL::StackMaps::Location::gpr):
(JSC::FTL::StackMaps::Location::restoreInto):
(JSC::FTL::StackMaps::Record::parse):
(JSC::FTL::StackMaps::Record::dump):
(JSC::FTL::StackMaps::parse):
(JSC::FTL::StackMaps::dump):
(JSC::FTL::StackMaps::dumpMultiline):
(JSC::FTL::StackMaps::getRecordMap):
(WTF::printInternal):
* ftl/FTLStackMaps.h: Added.
* ftl/FTLState.h:
* ftl/FTLThunks.cpp:
(JSC::FTL::osrExitGenerationThunkGenerator):
* ftl/FTLValueFormat.cpp:
(JSC::FTL::reboxAccordingToFormat):
* ftl/FTLValueFormat.h:
* runtime/DataView.cpp:
(JSC::DataView::create):
* runtime/DataView.h:
(JSC::DataView::read):
* runtime/Options.h:



git-svn-id: http://svn.webkit.org/repository/webkit/trunk@157209 268f45cc-cd09-0410-ab3c-d52691b4dbfc
parent d071a8d5
2013-10-08 Filip Pizlo <fpizlo@apple.com>
FTL should be able to do simple OSR exits using llvm.webkit.stackmap
https://bugs.webkit.org/show_bug.cgi?id=122538
Reviewed by Oliver Hunt.
This gives the FTL the ability to OSR exit using the llvm.webkit.stackmap intrinsic.
- The FTL compiles all OSR exit calls as calls to llvm.webkit.stackmap with a unique
ID, passing a requested size that is big enough for own jump replacement.
- After LLVM compilation, we parse the new LLVM stackmap section.
- For all llvm.webkit.stackmaps that we used for OSR exits, we do a jumpReplacement,
which targets exit thunks that we generate.
- If an exit thunk fires, it causes JSC to compile an exit off-ramp that uses a
combination of the JSC-internal OSR exit accounting (FTL::ExitValue and friends) and
LLVM stackmap's accounting of where data actually ended up (register, indirect,
constant) to reconstruct bytecode state.
This still has shortcomings; for example it cannot handle XMM or YMM registers. Handling
YMM registers will require adding some basic YMM support to our assemblers - really we
just need the ability to move a YMM's value into a GPR.
This patch preserves all of the old, intrinsic-less, FTL OSR exit support. Hence it
manages to pass all existing FTL tests even despite its incompleteness. I think that's
the right way to go since this is already a big patch, and anyway it would be great to
keep the intrinsic-less FTL OSR exit support so long as the LLVM side of this hasn't
landed.
* JavaScriptCore.xcodeproj/project.pbxproj:
* assembler/AbstractMacroAssembler.h:
(JSC::AbstractMacroAssembler::firstRegister):
(JSC::AbstractMacroAssembler::lastRegister):
* assembler/MacroAssembler.h:
(JSC::MacroAssembler::isStackRelated):
(JSC::MacroAssembler::firstRealRegister):
(JSC::MacroAssembler::nextRegister):
(JSC::MacroAssembler::secondRealRegister):
* assembler/MacroAssemblerX86Common.h:
* assembler/X86Assembler.h:
(JSC::X86Assembler::firstRegister):
(JSC::X86Assembler::lastRegister):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::compileInThreadImpl):
* ftl/FTLCArgumentGetter.cpp:
(JSC::FTL::CArgumentGetter::loadNextAndBox):
* ftl/FTLCArgumentGetter.h:
(JSC::FTL::CArgumentGetter::loadNextDoubleIntoGPR):
* ftl/FTLCompile.cpp:
(JSC::FTL::mmAllocateCodeSection):
(JSC::FTL::mmAllocateDataSection):
(JSC::FTL::dumpDataSection):
(JSC::FTL::fixFunctionBasedOnStackMaps):
(JSC::FTL::compile):
* ftl/FTLExitThunkGenerator.cpp:
(JSC::FTL::ExitThunkGenerator::emitThunk):
(JSC::FTL::ExitThunkGenerator::emitThunks):
* ftl/FTLExitThunkGenerator.h:
* ftl/FTLExitValue.h:
(JSC::FTL::ExitValue::isInJSStackSomehow):
(JSC::FTL::ExitValue::valueFormat):
* ftl/FTLFail.cpp:
(JSC::FTL::fail):
* ftl/FTLIntrinsicRepository.h:
* ftl/FTLJITCode.h:
* ftl/FTLLowerDFGToLLVM.cpp:
(JSC::FTL::generateExitThunks):
(JSC::FTL::LowerDFGToLLVM::LowerDFGToLLVM):
(JSC::FTL::LowerDFGToLLVM::appendOSRExit):
(JSC::FTL::LowerDFGToLLVM::emitOSRExitCall):
(JSC::FTL::LowerDFGToLLVM::linkOSRExitsAndCompleteInitializationBlocks):
* ftl/FTLOSRExit.h:
* ftl/FTLOSRExitCompilationInfo.h:
(JSC::FTL::OSRExitCompilationInfo::OSRExitCompilationInfo):
* ftl/FTLOSRExitCompiler.cpp:
(JSC::FTL::compileStubWithOSRExitStackmap):
(JSC::FTL::compileStubWithoutOSRExitStackmap):
(JSC::FTL::compileFTLOSRExit):
* ftl/FTLSaveRestore.cpp: Added.
(JSC::FTL::bytesForGPRs):
(JSC::FTL::requiredScratchMemorySizeInBytes):
(JSC::FTL::offsetOfGPR):
(JSC::FTL::saveAllRegisters):
(JSC::FTL::restoreAllRegisters):
* ftl/FTLSaveRestore.h: Added.
* ftl/FTLStackMaps.cpp: Added.
(JSC::FTL::readObject):
(JSC::FTL::StackMaps::Constant::parse):
(JSC::FTL::StackMaps::Constant::dump):
(JSC::FTL::StackMaps::Location::parse):
(JSC::FTL::StackMaps::Location::dump):
(JSC::FTL::StackMaps::Location::involvesGPR):
(JSC::FTL::StackMaps::Location::isGPR):
(JSC::FTL::StackMaps::Location::gpr):
(JSC::FTL::StackMaps::Location::restoreInto):
(JSC::FTL::StackMaps::Record::parse):
(JSC::FTL::StackMaps::Record::dump):
(JSC::FTL::StackMaps::parse):
(JSC::FTL::StackMaps::dump):
(JSC::FTL::StackMaps::dumpMultiline):
(JSC::FTL::StackMaps::getRecordMap):
(WTF::printInternal):
* ftl/FTLStackMaps.h: Added.
* ftl/FTLState.h:
* ftl/FTLThunks.cpp:
(JSC::FTL::osrExitGenerationThunkGenerator):
* ftl/FTLValueFormat.cpp:
(JSC::FTL::reboxAccordingToFormat):
* ftl/FTLValueFormat.h:
* runtime/DataView.cpp:
(JSC::DataView::create):
* runtime/DataView.h:
(JSC::DataView::read):
* runtime/Options.h:
2013-10-09 Filip Pizlo <fpizlo@apple.com>
Minor clean-ups in the JSC Xcode project.
......@@ -314,6 +314,8 @@
0F9D3370165DBB90005AD387 /* Disassembler.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F9D336E165DBB8D005AD387 /* Disassembler.cpp */; };
0F9D339617FFC4E60073C2BC /* DFGFlushedAt.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F9D339417FFC4E60073C2BC /* DFGFlushedAt.cpp */; };
0F9D339717FFC4E60073C2BC /* DFGFlushedAt.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F9D339517FFC4E60073C2BC /* DFGFlushedAt.h */; settings = {ATTRIBUTES = (Private, ); }; };
0F9D339A1803ADB70073C2BC /* FTLStackMaps.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F9D33981803ADB70073C2BC /* FTLStackMaps.cpp */; };
0F9D339B1803ADB70073C2BC /* FTLStackMaps.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F9D33991803ADB70073C2BC /* FTLStackMaps.h */; settings = {ATTRIBUTES = (Private, ); }; };
0F9FB4F417FCB91700CB67F8 /* DFGStackLayoutPhase.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F9FB4F217FCB91700CB67F8 /* DFGStackLayoutPhase.cpp */; };
0F9FB4F517FCB91700CB67F8 /* DFGStackLayoutPhase.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F9FB4F317FCB91700CB67F8 /* DFGStackLayoutPhase.h */; settings = {ATTRIBUTES = (Private, ); }; };
0F9FC8C314E1B5FE00D52AE0 /* PolymorphicPutByIdList.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F9FC8BF14E1B5FB00D52AE0 /* PolymorphicPutByIdList.cpp */; };
......@@ -372,6 +374,8 @@
0FC815151405119B00CFA603 /* VTableSpectrum.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FC815141405118D00CFA603 /* VTableSpectrum.h */; settings = {ATTRIBUTES = (Private, ); }; };
0FC81516140511B500CFA603 /* VTableSpectrum.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0FC815121405118600CFA603 /* VTableSpectrum.cpp */; };
0FCCAE4516D0CF7400D0C65B /* ParserError.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FCCAE4316D0CF6E00D0C65B /* ParserError.h */; settings = {ATTRIBUTES = (Private, ); }; };
0FCEFAAB1804C13E00472CE4 /* FTLSaveRestore.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0FCEFAA91804C13E00472CE4 /* FTLSaveRestore.cpp */; };
0FCEFAAC1804C13E00472CE4 /* FTLSaveRestore.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FCEFAAA1804C13E00472CE4 /* FTLSaveRestore.h */; settings = {ATTRIBUTES = (Private, ); }; };
0FD2C92416D01EE900C7803F /* StructureInlines.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FD2C92316D01EE900C7803F /* StructureInlines.h */; settings = {ATTRIBUTES = (Private, ); }; };
0FD3C82614115D4000FD81CB /* DFGDriver.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0FD3C82014115CF800FD81CB /* DFGDriver.cpp */; };
0FD3C82814115D4F00FD81CB /* DFGDriver.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FD3C82214115D0E00FD81CB /* DFGDriver.h */; };
......@@ -1528,6 +1532,8 @@
0F9D336E165DBB8D005AD387 /* Disassembler.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = Disassembler.cpp; path = disassembler/Disassembler.cpp; sourceTree = "<group>"; };
0F9D339417FFC4E60073C2BC /* DFGFlushedAt.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = DFGFlushedAt.cpp; path = dfg/DFGFlushedAt.cpp; sourceTree = "<group>"; };
0F9D339517FFC4E60073C2BC /* DFGFlushedAt.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGFlushedAt.h; path = dfg/DFGFlushedAt.h; sourceTree = "<group>"; };
0F9D33981803ADB70073C2BC /* FTLStackMaps.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = FTLStackMaps.cpp; path = ftl/FTLStackMaps.cpp; sourceTree = "<group>"; };
0F9D33991803ADB70073C2BC /* FTLStackMaps.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = FTLStackMaps.h; path = ftl/FTLStackMaps.h; sourceTree = "<group>"; };
0F9FB4F217FCB91700CB67F8 /* DFGStackLayoutPhase.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = DFGStackLayoutPhase.cpp; path = dfg/DFGStackLayoutPhase.cpp; sourceTree = "<group>"; };
0F9FB4F317FCB91700CB67F8 /* DFGStackLayoutPhase.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGStackLayoutPhase.h; path = dfg/DFGStackLayoutPhase.h; sourceTree = "<group>"; };
0F9FC8BF14E1B5FB00D52AE0 /* PolymorphicPutByIdList.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = PolymorphicPutByIdList.cpp; sourceTree = "<group>"; };
......@@ -1598,6 +1604,8 @@
0FC815141405118D00CFA603 /* VTableSpectrum.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = VTableSpectrum.h; sourceTree = "<group>"; };
0FCB408515C0A3C30048932B /* SlotVisitorInlines.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = SlotVisitorInlines.h; sourceTree = "<group>"; };
0FCCAE4316D0CF6E00D0C65B /* ParserError.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ParserError.h; sourceTree = "<group>"; };
0FCEFAA91804C13E00472CE4 /* FTLSaveRestore.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = FTLSaveRestore.cpp; path = ftl/FTLSaveRestore.cpp; sourceTree = "<group>"; };
0FCEFAAA1804C13E00472CE4 /* FTLSaveRestore.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = FTLSaveRestore.h; path = ftl/FTLSaveRestore.h; sourceTree = "<group>"; };
0FD2C92316D01EE900C7803F /* StructureInlines.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = StructureInlines.h; sourceTree = "<group>"; };
0FD3C82014115CF800FD81CB /* DFGDriver.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = DFGDriver.cpp; path = dfg/DFGDriver.cpp; sourceTree = "<group>"; };
0FD3C82214115D0E00FD81CB /* DFGDriver.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGDriver.h; path = dfg/DFGDriver.h; sourceTree = "<group>"; };
......@@ -2640,6 +2648,10 @@
0F235BCA17178E1C00690C7F /* FTLOSRExitCompiler.h */,
0FEA0A291709629600BB722C /* FTLOutput.cpp */,
0FEA0A06170513DB00BB722C /* FTLOutput.h */,
0FCEFAA91804C13E00472CE4 /* FTLSaveRestore.cpp */,
0FCEFAAA1804C13E00472CE4 /* FTLSaveRestore.h */,
0F9D33981803ADB70073C2BC /* FTLStackMaps.cpp */,
0F9D33991803ADB70073C2BC /* FTLStackMaps.h */,
0FEA0A151706BB9000BB722C /* FTLState.cpp */,
0FEA0A07170513DB00BB722C /* FTLState.h */,
A7FCC26C17A0B6AA00786D1A /* FTLSwitchCase.h */,
......@@ -4304,6 +4316,7 @@
BC18C42A0E16F5CD00B34460 /* JSType.h in Headers */,
0F2B66FB17B6B5AB00A7AE3F /* JSTypedArrayConstructors.h in Headers */,
0F2B66FD17B6B5AB00A7AE3F /* JSTypedArrayPrototypes.h in Headers */,
0FCEFAAC1804C13E00472CE4 /* FTLSaveRestore.h in Headers */,
0F2B66FF17B6B5AB00A7AE3F /* JSTypedArrays.h in Headers */,
6507D29E0E871E5E00D7D896 /* JSTypeInfo.h in Headers */,
0F2B670217B6B5AB00A7AE3F /* JSUint16Array.h in Headers */,
......@@ -4499,6 +4512,7 @@
BC9041480EB9250900FE26FA /* StructureTransitionTable.h in Headers */,
C2DF44301707AC0100A5CA96 /* SuperRegion.h in Headers */,
BC18C46B0E16F5CD00B34460 /* SymbolTable.h in Headers */,
0F9D339B1803ADB70073C2BC /* FTLStackMaps.h in Headers */,
A784A26411D16622005776AC /* SyntaxChecker.h in Headers */,
0F572D4F16879FDD00E57FBD /* ThunkGenerator.h in Headers */,
0FD8A32A17D51F5700CA2C40 /* DFGToFTLDeferredCompilationCallback.h in Headers */,
......@@ -5249,6 +5263,7 @@
969A079A0ED1D3AE00F1F681 /* Opcode.cpp in Sources */,
14280850107EC0D70013E7B2 /* Operations.cpp in Sources */,
A7BDAEC817F4EA1400F6140C /* ArrayIteratorPrototype.cpp in Sources */,
0FCEFAAB1804C13E00472CE4 /* FTLSaveRestore.cpp in Sources */,
0FE228EE1436AB2C00196C48 /* Options.cpp in Sources */,
148F21BC107EC54D0042EC2C /* Parser.cpp in Sources */,
93052C340FB792190048FDC3 /* ParserArena.cpp in Sources */,
......@@ -5346,6 +5361,7 @@
86704B8412DBA33700A9FE7B /* YarrInterpreter.cpp in Sources */,
86704B8612DBA33700A9FE7B /* YarrJIT.cpp in Sources */,
86704B8912DBA33700A9FE7B /* YarrPattern.cpp in Sources */,
0F9D339A1803ADB70073C2BC /* FTLStackMaps.cpp in Sources */,
86704B4212DB8A8100A9FE7B /* YarrSyntaxChecker.cpp in Sources */,
);
runOnlyForDeploymentPostprocessing = 0;
......
......@@ -74,6 +74,9 @@ public:
class Jump;
typedef typename AssemblerType::RegisterID RegisterID;
static RegisterID firstRegister() { return AssemblerType::firstRegister(); }
static RegisterID lastRegister() { return AssemblerType::lastRegister(); }
// Section 1: MacroAssembler operand types
//
......
......@@ -67,6 +67,32 @@ namespace JSC {
class MacroAssembler : public MacroAssemblerBase {
public:
static bool isStackRelated(RegisterID reg)
{
return reg == stackPointerRegister || reg == framePointerRegister;
}
static RegisterID firstRealRegister()
{
RegisterID firstRegister = MacroAssembler::firstRegister();
while (MacroAssembler::isStackRelated(firstRegister))
firstRegister = static_cast<RegisterID>(firstRegister + 1);
return firstRegister;
}
static RegisterID nextRegister(RegisterID reg)
{
RegisterID result = static_cast<RegisterID>(reg + 1);
while (MacroAssembler::isStackRelated(result))
result = static_cast<RegisterID>(result + 1);
return result;
}
static RegisterID secondRealRegister()
{
return nextRegister(firstRealRegister());
}
using MacroAssemblerBase::pop;
using MacroAssemblerBase::jump;
using MacroAssemblerBase::branch32;
......
......@@ -97,7 +97,7 @@ public:
static const RegisterID stackPointerRegister = X86Registers::esp;
static const RegisterID framePointerRegister = X86Registers::ebp;
static bool shouldBlindForSpecificArch(uint32_t value) { return value >= 0x00ffffff; }
#if CPU(X86_64)
static bool shouldBlindForSpecificArch(uint64_t value) { return value >= 0x00ffffff; }
......
......@@ -126,6 +126,16 @@ namespace X86Registers {
class X86Assembler {
public:
typedef X86Registers::RegisterID RegisterID;
static RegisterID firstRegister() { return X86Registers::eax; }
static RegisterID lastRegister()
{
#if CPU(X86_64)
return X86Registers::r15;
#else
return X86Registers::edi;
#endif
}
typedef X86Registers::XMMRegisterID XMMRegisterID;
typedef XMMRegisterID FPRegisterID;
......
......@@ -264,12 +264,18 @@ Plan::CompilationPath Plan::compileInThreadImpl(LongLivedState& longLivedState)
if (Options::reportCompileTimes())
beforeFTL = currentTimeMS();
if (Options::llvmAlwaysFails()) {
if (Options::llvmAlwaysFailsBeforeCompile()) {
FTL::fail(state);
return FTLPath;
}
FTL::compile(state);
if (Options::llvmAlwaysFailsBeforeLink()) {
FTL::fail(state);
return FTLPath;
}
FTL::link(state);
return FTLPath;
#else
......
......@@ -52,60 +52,31 @@ void CArgumentGetter::loadNextAndBox(
}
switch (format) {
case ValueFormatInt32: {
case ValueFormatInt32:
case ValueFormatUInt32:
loadNext32(destination);
m_jit.or64(GPRInfo::tagTypeNumberRegister, destination);
break;
}
case ValueFormatUInt32: {
loadNext32(destination);
m_jit.moveDoubleTo64(FPRInfo::fpRegT0, scratch2);
m_jit.boxInt52(destination, destination, scratch1, FPRInfo::fpRegT0);
m_jit.move64ToDouble(scratch2, FPRInfo::fpRegT0);
break;
}
case ValueFormatInt52: {
case ValueFormatInt52:
case ValueFormatStrictInt52:
case ValueFormatJSValue:
loadNext64(destination);
m_jit.rshift64(AssemblyHelpers::TrustedImm32(JSValue::int52ShiftAmount), destination);
m_jit.moveDoubleTo64(FPRInfo::fpRegT0, scratch2);
m_jit.boxInt52(destination, destination, scratch1, FPRInfo::fpRegT0);
m_jit.move64ToDouble(scratch2, FPRInfo::fpRegT0);
break;
}
case ValueFormatStrictInt52: {
loadNext64(destination);
m_jit.moveDoubleTo64(FPRInfo::fpRegT0, scratch2);
m_jit.boxInt52(destination, destination, scratch1, FPRInfo::fpRegT0);
m_jit.move64ToDouble(scratch2, FPRInfo::fpRegT0);
break;
}
case ValueFormatBoolean: {
case ValueFormatBoolean:
loadNext8(destination);
m_jit.or32(MacroAssembler::TrustedImm32(ValueFalse), destination);
break;
}
case ValueFormatJSValue: {
loadNext64(destination);
break;
}
case ValueFormatDouble: {
m_jit.moveDoubleTo64(FPRInfo::fpRegT0, scratch1);
loadNextDouble(FPRInfo::fpRegT0);
m_jit.boxDouble(FPRInfo::fpRegT0, destination);
m_jit.move64ToDouble(scratch1, FPRInfo::fpRegT0);
case ValueFormatDouble:
loadNextDoubleIntoGPR(destination);
break;
}
default:
RELEASE_ASSERT_NOT_REACHED();
break;
}
reboxAccordingToFormat(format, m_jit, destination, scratch1, scratch2);
}
} } // namespace JSC::FTL
......
......@@ -100,6 +100,16 @@ public:
loadNext64(destination);
}
void loadNextDoubleIntoGPR(GPRReg destination)
{
if (m_fprArgumentIndex < FPRInfo::numberOfArgumentRegisters) {
m_jit.moveDoubleTo64(FPRInfo::toArgumentRegister(m_fprArgumentIndex++), destination);
return;
}
m_jit.load64(nextAddress(), destination);
}
void loadNextDouble(FPRReg destination)
{
ASSERT(
......
......@@ -31,10 +31,14 @@
#include "CodeBlockWithJITType.h"
#include "CCallHelpers.h"
#include "DFGCommon.h"
#include "DataView.h"
#include "Disassembler.h"
#include "FTLExitThunkGenerator.h"
#include "FTLJITCode.h"
#include "FTLThunks.h"
#include "JITStubs.h"
#include "LinkBuffer.h"
#include "RepatchBuffer.h"
#include <wtf/LLVMHeaders.h>
namespace JSC { namespace FTL {
......@@ -42,11 +46,8 @@ namespace JSC { namespace FTL {
using namespace DFG;
static uint8_t* mmAllocateCodeSection(
void* opaqueState, uintptr_t size, unsigned alignment, unsigned sectionID,
const char* sectionName)
void* opaqueState, uintptr_t size, unsigned alignment, unsigned, const char* sectionName)
{
UNUSED_PARAM(sectionID);
UNUSED_PARAM(sectionName);
State& state = *static_cast<State*>(opaqueState);
......@@ -57,6 +58,7 @@ static uint8_t* mmAllocateCodeSection(
state.graph.m_vm, size, state.graph.m_codeBlock, JITCompilationMustSucceed);
state.jitCode->addHandle(result);
state.codeSectionNames.append(sectionName);
return static_cast<uint8_t*>(result->start());
}
......@@ -66,7 +68,6 @@ static uint8_t* mmAllocateDataSection(
const char* sectionName, LLVMBool isReadOnly)
{
UNUSED_PARAM(sectionID);
UNUSED_PARAM(sectionName);
UNUSED_PARAM(isReadOnly);
State& state = *static_cast<State*>(opaqueState);
......@@ -76,7 +77,12 @@ static uint8_t* mmAllocateDataSection(
RefCountedArray<LSectionWord> section(
(size + sizeof(LSectionWord) - 1) / sizeof(LSectionWord));
state.jitCode->addDataSection(section);
if (!strcmp(sectionName, "__js_stackmaps"))
state.stackmapsSection = section;
else {
state.jitCode->addDataSection(section);
state.dataSectionNames.append(sectionName);
}
return bitwise_cast<uint8_t*>(section.data());
}
......@@ -90,6 +96,60 @@ static void mmDestroy(void*)
{
}
static void dumpDataSection(RefCountedArray<LSectionWord> section, const char* prefix)
{
for (unsigned j = 0; j < section.size(); ++j) {
char buf[32];
snprintf(buf, sizeof(buf), "0x%lx", static_cast<unsigned long>(bitwise_cast<uintptr_t>(section.data() + j)));
dataLogF("%s%16s: 0x%016llx\n", prefix, buf, static_cast<long long>(section[j]));
}
}
static void fixFunctionBasedOnStackMaps(
State& state, CodeBlock* codeBlock, JITCode* jitCode, GeneratedFunction generatedFunction,
StackMaps::RecordMap& recordMap)
{
VM& vm = state.graph.m_vm;
MacroAssemblerCodeRef osrExitThunk =
vm.getCTIStub(osrExitGenerationThunkGenerator);
CodeLocationLabel target = CodeLocationLabel(osrExitThunk.code());
ExitThunkGenerator exitThunkGenerator(state);
exitThunkGenerator.emitThunks();
if (exitThunkGenerator.didThings()) {
OwnPtr<LinkBuffer> linkBuffer = adoptPtr(new LinkBuffer(
vm, &exitThunkGenerator, codeBlock, JITCompilationMustSucceed));
ASSERT(state.osrExit.size() == state.jitCode->osrExit.size());
for (unsigned i = 0; i < state.osrExit.size(); ++i) {
OSRExitCompilationInfo& info = state.osrExit[i];
OSRExit& exit = jitCode->osrExit[i];
linkBuffer->link(info.m_thunkJump, target);
info.m_thunkAddress = linkBuffer->locationOf(info.m_thunkLabel);
exit.m_patchableCodeOffset = linkBuffer->offsetOf(info.m_thunkJump);
}
state.finalizer->initializeExitThunksLinkBuffer(linkBuffer.release());
}
RepatchBuffer repatchBuffer(codeBlock);
for (unsigned exitIndex = jitCode->osrExit.size(); exitIndex--;) {
OSRExitCompilationInfo& info = state.osrExit[exitIndex];
OSRExit& exit = jitCode->osrExit[exitIndex];
StackMaps::Record& record = recordMap.find(exit.m_stackmapID)->value;
repatchBuffer.replaceWithJump(
CodeLocationLabel(
bitwise_cast<char*>(generatedFunction) + record.instructionOffset),
info.m_thunkAddress);
}
}
void compile(State& state)
{
char* error = 0;
......@@ -143,15 +203,12 @@ void compile(State& state)
LLVMDisposeExecutionEngine(engine);
if (shouldShowDisassembly()) {
// FIXME: fourthTier: FTL memory allocator should be able to tell us which of
// these things is actually code or data.
// https://bugs.webkit.org/show_bug.cgi?id=116189
for (unsigned i = 0; i < state.jitCode->handles().size(); ++i) {
ExecutableMemoryHandle* handle = state.jitCode->handles()[i].get();
dataLog(
"Generated LLVM code for ",
CodeBlockWithJITType(state.graph.m_codeBlock, JITCode::DFGJIT),
" #", i, ":\n");
" #", i, ", ", state.codeSectionNames[i], ":\n");
disassemble(
MacroAssemblerCodePtr(handle->start()), handle->sizeInBytes(),
" ", WTF::dataFile(), LLVMSubset);
......@@ -162,14 +219,52 @@ void compile(State& state)
dataLog(
"Generated LLVM data section for ",
CodeBlockWithJITType(state.graph.m_codeBlock, JITCode::DFGJIT),
" #", i, ":\n");
for (unsigned j = 0; j < section.size(); ++j) {
char buf[32];
snprintf(buf, sizeof(buf), "0x%lx", static_cast<unsigned long>(bitwise_cast<uintptr_t>(section.data() + j)));
dataLogF(" %16s: 0x%016llx\n", buf, static_cast<long long>(section[j]));
" #", i, ", ", state.dataSectionNames[i], ":\n");
dumpDataSection(section, " ");
}
}
if (state.stackmapsSection.size()) {
if (shouldShowDisassembly()) {
dataLog(
"Generated LLVM stackmaps section for ",
CodeBlockWithJITType(state.graph.m_codeBlock, JITCode::DFGJIT), ":\n");
dataLog(" Raw data:\n");
dumpDataSection(state.stackmapsSection, " ");
}
RefPtr<DataView> stackmapsData = DataView::create(
ArrayBuffer::create(state.stackmapsSection.data(), state.stackmapsSection.byteSize()));
state.jitCode->stackmaps.parse(stackmapsData.get());
if (shouldShowDisassembly()) {
dataLog(" Structured data:\n");
state.jitCode->stackmaps.dumpMultiline(WTF::dataFile(), " ");
}
StackMaps::RecordMap recordMap = state.jitCode->stackmaps.getRecordMap();
fixFunctionBasedOnStackMaps(
state, state.graph.m_codeBlock, state.jitCode.get(), state.generatedFunction,
recordMap);
if (shouldShowDisassembly()) {
for (unsigned i = 0; i < state.jitCode->handles().size(); ++i) {
if (state.codeSectionNames[i] != "__text")
continue;
ExecutableMemoryHandle* handle = state.jitCode->handles()[i].get();
dataLog(
"Generated LLVM code after stackmap-based fix-up for ",
CodeBlockWithJITType(state.graph.m_codeBlock, JITCode::DFGJIT),
" #", i, ", ", state.codeSectionNames[i], ":\n");
disassemble(
MacroAssemblerCodePtr(handle->start()), handle->sizeInBytes(),
" ", WTF::dataFile(), LLVMSubset);
}
}
}
state.module = 0; // We no longer own the module.
}
} } // namespace JSC::FTL
......
......@@ -51,12 +51,21 @@ void ExitThunkGenerator::emitThunk(unsigned index)
OSRExitCompilationInfo& info = m_state.osrExit[index];
info.m_thunkLabel = label();
move(TrustedImm32(index), GPRInfo::nonArgGPR0);
if (Options::ftlOSRExitUsesStackmap())
push(TrustedImm32(index));
else
move(TrustedImm32(index), GPRInfo::nonArgGPR0);
info.m_thunkJump = patchableJump();
m_didThings = true;
}
void ExitThunkGenerator::emitThunks()
{
for (unsigned i = 0; i < m_state.osrExit.size(); ++i)
emitThunk(i);
}
} } // namespace JSC::FTL
#endif // ENABLE(FTL_JIT)
......
......@@ -43,6 +43,7 @@ public:
~ExitThunkGenerator();
void emitThunk(unsigned index);
void emitThunks();
bool didThings() const { return m_didThings; }
......
......@@ -126,6 +126,7 @@ public:
switch (kind()) {
case ExitValueInJSStack:
case ExitValueInJSStackAsInt32:
case ExitValueInJSStackAsInt52:
case ExitValueInJSStackAsDouble:
return true;
default:
......@@ -152,7 +153,40 @@ public:
ASSERT(isInJSStackSomehow());
return VirtualRegister(u.virtualRegister);
}
// If it's in the JSStack somehow, this will tell you what format it's in, in a manner
// that is compatible with exitArgument().format(). If it's a constant or it's dead, it
// will claim to be a JSValue. If it's an argument then it will tell you the argument's
// format.
ValueFormat valueFormat() const
{
switch (kind()) {
case InvalidExitValue:
RELEASE_ASSERT_NOT_REACHED();
return InvalidValueFormat;
case ExitValueDead:
case ExitValueConstant:
case ExitValueInJSStack:
return ValueFormatJSValue;
case ExitValueArgument:
return exitArgument().format();
case ExitValueInJSStackAsInt32:
return ValueFormatInt32;