Commit d68b1f84 authored by fpizlo@apple.com's avatar fpizlo@apple.com

Inline property storage should not be wasted when it is exhausted

https://bugs.webkit.org/show_bug.cgi?id=90347

Reviewed by Gavin Barraclough.
        
Previously, if we switched an object from using inline storage to out-of-line
storage, we would abandon the inline storage. This would have two main implications:
(i) all accesses to the object, even for properties that were previously in inline
storage, must now take an extra indirection; and (ii) we waste a non-trivial amount
of space since we must allocate additional out-of-line storage to hold properties
that would have fit in the inline storage. There's also the copying cost when
switching to out-of-line storage - we must copy all inline properties into ouf-of-line
storage.
        
This patch changes the way that object property storage works so that we can use both
inline and out-of-line storage concurrently. This is accomplished by introducing a
new notion of property offset. This PropertyOffset is a 32-bit signed integer and it
behaves as follows:
        
offset == -1: invalid offset, indicating a property that does not exist.
        
0 <= offset <= inlineStorageCapacity: offset into inline storage.
        
inlineStorageCapacity < offset: offset into out-of-line storage.
        
Because non-final objects don't have inline storage, the only valid PropertyOffsets
for those objects' properties are -1 or > inlineStorageCapacity.
        
This now means that the decision to use inline or out-of-line storage for an access is
made based on the offset, rather than the structure. It also means that any access
where the offset is a variable must have an extra branch, unless the type of the
object is also known (if it's known to be a non-final object then we can just assert
that the offset is >= inlineStorageCapacity).
        
This looks like a big Kraken speed-up and a slight V8 speed-up.

* GNUmakefile.list.am:
* JavaScriptCore.vcproj/JavaScriptCore/JavaScriptCore.def:
* JavaScriptCore.vcproj/JavaScriptCore/JavaScriptCore.vcproj:
* JavaScriptCore.xcodeproj/project.pbxproj:
* assembler/ARMv7Assembler.h:
(ARMv7Assembler):
(JSC::ARMv7Assembler::ldrWide8BitImmediate):
(JSC::ARMv7Assembler::replaceWithLoad):
(JSC::ARMv7Assembler::replaceWithAddressComputation):
* assembler/AbstractMacroAssembler.h:
(AbstractMacroAssembler):
(ConvertibleLoadLabel):
(JSC::AbstractMacroAssembler::ConvertibleLoadLabel::ConvertibleLoadLabel):
(JSC::AbstractMacroAssembler::ConvertibleLoadLabel::isSet):
(JSC::AbstractMacroAssembler::labelIgnoringWatchpoints):
(JSC::AbstractMacroAssembler::replaceWithLoad):
(JSC::AbstractMacroAssembler::replaceWithAddressComputation):
* assembler/CodeLocation.h:
(JSC):
(CodeLocationCommon):
(CodeLocationConvertibleLoad):
(JSC::CodeLocationConvertibleLoad::CodeLocationConvertibleLoad):
(JSC::CodeLocationCommon::convertibleLoadAtOffset):
* assembler/LinkBuffer.cpp:
(JSC::LinkBuffer::finalizeCodeWithDisassembly):
* assembler/LinkBuffer.h:
(LinkBuffer):
(JSC::LinkBuffer::locationOf):
* assembler/MacroAssemblerARMv7.h:
(MacroAssemblerARMv7):
(JSC::MacroAssemblerARMv7::convertibleLoadPtr):
* assembler/MacroAssemblerX86.h:
(JSC::MacroAssemblerX86::convertibleLoadPtr):
(MacroAssemblerX86):
* assembler/MacroAssemblerX86_64.h:
(JSC::MacroAssemblerX86_64::convertibleLoadPtr):
(MacroAssemblerX86_64):
* assembler/RepatchBuffer.h:
(RepatchBuffer):
(JSC::RepatchBuffer::replaceWithLoad):
(JSC::RepatchBuffer::replaceWithAddressComputation):
(JSC::RepatchBuffer::setLoadInstructionIsActive):
* assembler/X86Assembler.h:
(JSC::X86Assembler::replaceWithLoad):
(X86Assembler):
(JSC::X86Assembler::replaceWithAddressComputation):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::printGetByIdOp):
(JSC::CodeBlock::dump):
(JSC::CodeBlock::finalizeUnconditionally):
* bytecode/GetByIdStatus.cpp:
(JSC::GetByIdStatus::computeFromLLInt):
(JSC::GetByIdStatus::computeForChain):
(JSC::GetByIdStatus::computeFor):
* bytecode/GetByIdStatus.h:
(JSC::GetByIdStatus::GetByIdStatus):
(JSC::GetByIdStatus::offset):
(GetByIdStatus):
* bytecode/Opcode.h:
(JSC):
(JSC::padOpcodeName):
* bytecode/PutByIdStatus.cpp:
(JSC::PutByIdStatus::computeFromLLInt):
(JSC::PutByIdStatus::computeFor):
* bytecode/PutByIdStatus.h:
(JSC::PutByIdStatus::PutByIdStatus):
(JSC::PutByIdStatus::offset):
(PutByIdStatus):
* bytecode/ResolveGlobalStatus.cpp:
(JSC):
(JSC::computeForStructure):
* bytecode/ResolveGlobalStatus.h:
(JSC::ResolveGlobalStatus::ResolveGlobalStatus):
(JSC::ResolveGlobalStatus::offset):
(ResolveGlobalStatus):
* bytecode/StructureSet.h:
(StructureSet):
* bytecode/StructureStubInfo.h:
* dfg/DFGByteCodeParser.cpp:
(ByteCodeParser):
(JSC::DFG::ByteCodeParser::handleGetByOffset):
(JSC::DFG::ByteCodeParser::handleGetById):
(JSC::DFG::ByteCodeParser::parseBlock):
* dfg/DFGCapabilities.h:
(JSC::DFG::canCompileOpcode):
* dfg/DFGJITCompiler.cpp:
(JSC::DFG::JITCompiler::link):
* dfg/DFGJITCompiler.h:
(JSC::DFG::PropertyAccessRecord::PropertyAccessRecord):
(PropertyAccessRecord):
* dfg/DFGRepatch.cpp:
(JSC::DFG::dfgRepatchByIdSelfAccess):
(JSC::DFG::generateProtoChainAccessStub):
(JSC::DFG::tryCacheGetByID):
(JSC::DFG::tryBuildGetByIDList):
(JSC::DFG::tryBuildGetByIDProtoList):
(JSC::DFG::emitPutReplaceStub):
(JSC::DFG::emitPutTransitionStub):
(JSC::DFG::tryCachePutByID):
(JSC::DFG::tryBuildPutByIdList):
* dfg/DFGSpeculativeJIT.h:
(JSC::DFG::SpeculativeJIT::emitAllocateBasicJSObject):
* dfg/DFGSpeculativeJIT32_64.cpp:
(JSC::DFG::SpeculativeJIT::cachedGetById):
(JSC::DFG::SpeculativeJIT::cachedPutById):
(JSC::DFG::SpeculativeJIT::compile):
* dfg/DFGSpeculativeJIT64.cpp:
(JSC::DFG::SpeculativeJIT::cachedGetById):
(JSC::DFG::SpeculativeJIT::cachedPutById):
(JSC::DFG::SpeculativeJIT::compile):
* heap/MarkStack.cpp:
(JSC::visitChildren):
* interpreter/Interpreter.cpp:
(JSC::Interpreter::tryCacheGetByID):
(JSC::Interpreter::privateExecute):
* jit/JIT.cpp:
(JSC::JIT::privateCompileMainPass):
(JSC::JIT::privateCompileSlowCases):
(JSC::PropertyStubCompilationInfo::copyToStubInfo):
* jit/JIT.h:
(JSC::PropertyStubCompilationInfo::PropertyStubCompilationInfo):
(JSC::JIT::compileGetByIdProto):
(JSC::JIT::compileGetByIdSelfList):
(JSC::JIT::compileGetByIdProtoList):
(JSC::JIT::compileGetByIdChainList):
(JSC::JIT::compileGetByIdChain):
(JSC::JIT::compilePutByIdTransition):
(JIT):
* jit/JITInlineMethods.h:
(JSC::JIT::emitAllocateBasicJSObject):
* jit/JITOpcodes.cpp:
(JSC::JIT::emit_op_resolve_global):
* jit/JITOpcodes32_64.cpp:
(JSC::JIT::emit_op_resolve_global):
* jit/JITPropertyAccess.cpp:
(JSC::JIT::compileGetDirectOffset):
(JSC::JIT::emit_op_method_check):
(JSC::JIT::compileGetByIdHotPath):
(JSC::JIT::emit_op_put_by_id):
(JSC::JIT::compilePutDirectOffset):
(JSC::JIT::privateCompilePutByIdTransition):
(JSC::JIT::patchGetByIdSelf):
(JSC::JIT::patchPutByIdReplace):
(JSC::JIT::privateCompileGetByIdProto):
(JSC::JIT::privateCompileGetByIdSelfList):
(JSC::JIT::privateCompileGetByIdProtoList):
(JSC::JIT::privateCompileGetByIdChainList):
(JSC::JIT::privateCompileGetByIdChain):
* jit/JITPropertyAccess32_64.cpp:
(JSC::JIT::emit_op_method_check):
(JSC::JIT::compileGetByIdHotPath):
(JSC::JIT::emit_op_put_by_id):
(JSC::JIT::compilePutDirectOffset):
(JSC::JIT::compileGetDirectOffset):
(JSC::JIT::privateCompilePutByIdTransition):
(JSC::JIT::patchGetByIdSelf):
(JSC::JIT::patchPutByIdReplace):
(JSC::JIT::privateCompileGetByIdProto):
(JSC::JIT::privateCompileGetByIdSelfList):
(JSC::JIT::privateCompileGetByIdProtoList):
(JSC::JIT::privateCompileGetByIdChainList):
(JSC::JIT::privateCompileGetByIdChain):
(JSC::JIT::emit_op_get_by_pname):
* jit/JITStubs.cpp:
(JSC::JITThunks::tryCacheGetByID):
(JSC::DEFINE_STUB_FUNCTION):
* llint/LLIntSlowPaths.cpp:
(JSC::LLInt::LLINT_SLOW_PATH_DECL):
* llint/LowLevelInterpreter.asm:
* llint/LowLevelInterpreter32_64.asm:
* llint/LowLevelInterpreter64.asm:
* offlineasm/x86.rb:
* runtime/JSGlobalObject.h:
(JSGlobalObject):
(JSC::JSGlobalObject::functionNameOffset):
* runtime/JSObject.cpp:
(JSC::JSObject::visitChildren):
(JSC):
(JSC::JSFinalObject::visitChildren):
(JSC::JSObject::put):
(JSC::JSObject::deleteProperty):
(JSC::JSObject::getPropertySpecificValue):
(JSC::JSObject::removeDirect):
(JSC::JSObject::growOutOfLineStorage):
(JSC::JSObject::getOwnPropertyDescriptor):
* runtime/JSObject.h:
(JSObject):
(JSC::JSObject::getDirect):
(JSC::JSObject::getDirectLocation):
(JSC::JSObject::hasInlineStorage):
(JSC::JSObject::inlineStorageUnsafe):
(JSC::JSObject::inlineStorage):
(JSC::JSObject::outOfLineStorage):
(JSC::JSObject::locationForOffset):
(JSC::JSObject::offsetForLocation):
(JSC::JSObject::getDirectOffset):
(JSC::JSObject::putDirectOffset):
(JSC::JSObject::putUndefinedAtDirectOffset):
(JSC::JSObject::addressOfOutOfLineStorage):
(JSC::JSObject::finishCreation):
(JSC::JSNonFinalObject::JSNonFinalObject):
(JSC::JSNonFinalObject::finishCreation):
(JSFinalObject):
(JSC::JSFinalObject::finishCreation):
(JSC::JSFinalObject::JSFinalObject):
(JSC::JSObject::offsetOfOutOfLineStorage):
(JSC::JSObject::setOutOfLineStorage):
(JSC::JSObject::JSObject):
(JSC):
(JSC::JSCell::fastGetOwnProperty):
(JSC::JSObject::putDirectInternal):
(JSC::JSObject::setStructureAndReallocateStorageIfNecessary):
(JSC::JSObject::putDirectWithoutTransition):
(JSC::offsetRelativeToPatchedStorage):
(JSC::indexRelativeToBase):
(JSC::offsetRelativeToBase):
* runtime/JSPropertyNameIterator.cpp:
(JSC::JSPropertyNameIterator::create):
* runtime/JSPropertyNameIterator.h:
(JSPropertyNameIterator):
(JSC::JSPropertyNameIterator::getOffset):
(JSC::JSPropertyNameIterator::finishCreation):
* runtime/JSValue.cpp:
(JSC::JSValue::putToPrimitive):
* runtime/Operations.h:
(JSC::normalizePrototypeChain):
* runtime/Options.cpp:
(JSC):
(JSC::Options::initialize):
* runtime/PropertyMapHashTable.h:
(PropertyMapEntry):
(JSC::PropertyMapEntry::PropertyMapEntry):
(PropertyTable):
(JSC::PropertyTable::PropertyTable):
(JSC::PropertyTable::getDeletedOffset):
(JSC::PropertyTable::addDeletedOffset):
(JSC::PropertyTable::nextOffset):
(JSC):
(JSC::PropertyTable::sizeInMemory):
* runtime/PropertyOffset.h: Added.
(JSC):
(JSC::checkOffset):
(JSC::validateOffset):
(JSC::isValidOffset):
(JSC::isInlineOffset):
(JSC::isOutOfLineOffset):
(JSC::offsetInInlineStorage):
(JSC::offsetInOutOfLineStorage):
(JSC::offsetInRespectiveStorage):
(JSC::numberOfOutOfLineSlotsForLastOffset):
(JSC::numberOfSlotsForLastOffset):
(JSC::nextPropertyOffsetFor):
(JSC::firstPropertyOffsetFor):
* runtime/PropertySlot.h:
(JSC::PropertySlot::cachedOffset):
(JSC::PropertySlot::setValue):
(JSC::PropertySlot::setCacheableGetterSlot):
(JSC::PropertySlot::clearOffset):
* runtime/PutPropertySlot.h:
(JSC::PutPropertySlot::setExistingProperty):
(JSC::PutPropertySlot::setNewProperty):
(JSC::PutPropertySlot::cachedOffset):
(PutPropertySlot):
* runtime/Structure.cpp:
(JSC::Structure::Structure):
(JSC::Structure::materializePropertyMap):
(JSC::nextOutOfLineStorageCapacity):
(JSC::Structure::growOutOfLineCapacity):
(JSC::Structure::suggestedNewOutOfLineStorageCapacity):
(JSC::Structure::addPropertyTransitionToExistingStructure):
(JSC::Structure::addPropertyTransition):
(JSC::Structure::removePropertyTransition):
(JSC::Structure::flattenDictionaryStructure):
(JSC::Structure::addPropertyWithoutTransition):
(JSC::Structure::removePropertyWithoutTransition):
(JSC::Structure::copyPropertyTableForPinning):
(JSC::Structure::get):
(JSC::Structure::putSpecificValue):
(JSC::Structure::remove):
* runtime/Structure.h:
(Structure):
(JSC::Structure::putWillGrowOutOfLineStorage):
(JSC::Structure::previousID):
(JSC::Structure::outOfLineCapacity):
(JSC::Structure::outOfLineSizeForKnownFinalObject):
(JSC::Structure::outOfLineSizeForKnownNonFinalObject):
(JSC::Structure::outOfLineSize):
(JSC::Structure::hasInlineStorage):
(JSC::Structure::inlineCapacity):
(JSC::Structure::inlineSizeForKnownFinalObject):
(JSC::Structure::inlineSize):
(JSC::Structure::totalStorageSize):
(JSC::Structure::totalStorageCapacity):
(JSC::Structure::firstValidOffset):
(JSC::Structure::lastValidOffset):
(JSC::Structure::isValidOffset):
(JSC::Structure::isEmpty):
(JSC::Structure::transitionCount):
(JSC::Structure::get):



git-svn-id: http://svn.webkit.org/repository/webkit/trunk@121925 268f45cc-cd09-0410-ab3c-d52691b4dbfc
parent 5635b54d
This diff is collapsed.
......@@ -582,6 +582,7 @@ javascriptcore_sources += \
Source/JavaScriptCore/runtime/PropertyName.h \
Source/JavaScriptCore/runtime/PropertyNameArray.cpp \
Source/JavaScriptCore/runtime/PropertyNameArray.h \
Source/JavaScriptCore/runtime/PropertyOffset.h \
Source/JavaScriptCore/runtime/PropertySlot.cpp \
Source/JavaScriptCore/runtime/PropertySlot.h \
Source/JavaScriptCore/runtime/Protect.h \
......
......@@ -61,9 +61,9 @@ EXPORTS
?addBytes@SHA1@WTF@@QAEXPBEI@Z
?addCurrentThread@MachineThreads@JSC@@QAEXXZ
?addFinalizer@Heap@JSC@@QAEXPAVJSCell@2@P6AX0@Z@Z
?addPropertyTransition@Structure@JSC@@SAPAV12@AAVJSGlobalData@2@PAV12@VPropertyName@2@IPAVJSCell@2@AAI@Z
?addPropertyTransitionToExistingStructure@Structure@JSC@@SAPAV12@PAV12@VPropertyName@2@IPAVJSCell@2@AAI@Z
?addPropertyWithoutTransition@Structure@JSC@@QAEIAAVJSGlobalData@2@VPropertyName@2@IPAVJSCell@2@@Z
?addPropertyTransition@Structure@JSC@@SAPAV12@AAVJSGlobalData@2@PAV12@VPropertyName@2@IPAVJSCell@2@AAH@Z
?addPropertyTransitionToExistingStructure@Structure@JSC@@SAPAV12@PAV12@VPropertyName@2@IPAVJSCell@2@AAH@Z
?addPropertyWithoutTransition@Structure@JSC@@QAEHAAVJSGlobalData@2@VPropertyName@2@IPAVJSCell@2@@Z
?addSlowCase@Identifier@JSC@@CA?AV?$PassRefPtr@VStringImpl@WTF@@@WTF@@PAVExecState@2@PAVStringImpl@4@@Z
?addSlowCase@Identifier@JSC@@CA?AV?$PassRefPtr@VStringImpl@WTF@@@WTF@@PAVJSGlobalData@2@PAVStringImpl@4@@Z
?addStaticGlobals@JSGlobalObject@JSC@@IAEXPAUGlobalPropertyInfo@12@H@Z
......@@ -192,7 +192,6 @@ EXPORTS
?from@Identifier@JSC@@SA?AV12@PAVExecState@2@I@Z
?functionGetter@PropertySlot@JSC@@ABE?AVJSValue@2@PAVExecState@2@@Z
?functionName@DebuggerCallFrame@JSC@@QBEPBVUString@2@XZ
?get@Structure@JSC@@QAEIAAVJSGlobalData@2@VPropertyName@2@AAIAAPAVJSCell@2@@Z
?getCalculatedDisplayName@JSC@@YA?AVUString@1@PAVExecState@1@PAVJSObject@1@@Z
?getCallData@JSCell@JSC@@SA?AW4CallType@2@PAV12@AATCallData@2@@Z
?getCallableObjectSlow@JSC@@YAPAVJSCell@1@PAV21@@Z
......@@ -211,11 +210,12 @@ EXPORTS
?getStackTrace@Interpreter@JSC@@SAXPAVJSGlobalData@2@AAV?$Vector@UStackFrame@JSC@@$0A@@WTF@@@Z
?getString@JSCell@JSC@@QBE?AVUString@2@PAVExecState@2@@Z
?getString@JSCell@JSC@@QBE_NPAVExecState@2@AAVUString@2@@Z
?get@Structure@JSC@@QAEHAAVJSGlobalData@2@VPropertyName@2@AAIAAPAVJSCell@2@@Z
?getter@PropertyDescriptor@JSC@@QBE?AVJSValue@2@XZ
?globalExec@JSGlobalObject@JSC@@QAEPAVExecState@2@XZ
?globalObjectCount@Heap@JSC@@QAEIXZ
?growOutOfLineStorage@JSObject@JSC@@QAEPAV?$WriteBarrierBase@W4Unknown@JSC@@@2@AAVJSGlobalData@2@II@Z
?grow@HandleSet@JSC@@AAEXXZ
?growPropertyStorage@JSObject@JSC@@QAEPAV?$WriteBarrierBase@W4Unknown@JSC@@@2@AAVJSGlobalData@2@II@Z
?hasInstance@JSObject@JSC@@SA_NPAV12@PAVExecState@2@VJSValue@2@2@Z
?hasProperty@JSObject@JSC@@QBE_NPAVExecState@2@I@Z
?hasProperty@JSObject@JSC@@QBE_NPAVExecState@2@VPropertyName@2@@Z
......@@ -328,7 +328,7 @@ EXPORTS
?stopProfiling@Profiler@JSC@@QAE?AV?$PassRefPtr@VProfile@JSC@@@WTF@@PAVExecState@2@ABVUString@2@@Z
?stopSampling@JSGlobalData@JSC@@QAEXXZ
?substringSharingImpl@UString@JSC@@QBE?AV12@II@Z
?suggestedNewPropertyStorageSize@Structure@JSC@@QAEIXZ
?suggestedNewOutOfLineStorageCapacity@Structure@JSC@@QAEIXZ
?sweeper@Heap@JSC@@QAEPAVIncrementalSweeper@2@XZ
?synthesizePrototype@JSValue@JSC@@QBEPAVJSObject@2@PAVExecState@2@@Z
?thisObject@DebuggerCallFrame@JSC@@QBEPAVJSObject@2@XZ
......
......@@ -1093,6 +1093,10 @@
RelativePath="..\..\runtime\PropertyNameArray.h"
>
</File>
<File
RelativePath="..\..\runtime\PropertyOffset.h"
>
</File>
<File
RelativePath="..\..\runtime\PropertySlot.cpp"
>
......
......@@ -223,6 +223,7 @@
0FF427651591A1CE004CB9FF /* DFGDisassembler.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FF427621591A1C9004CB9FF /* DFGDisassembler.h */; settings = {ATTRIBUTES = (Private, ); }; };
0FF42771159275D5004CB9FF /* ResolveGlobalStatus.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0FF4276E159275D2004CB9FF /* ResolveGlobalStatus.cpp */; };
0FF42772159275D8004CB9FF /* ResolveGlobalStatus.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FF4276F159275D2004CB9FF /* ResolveGlobalStatus.h */; settings = {ATTRIBUTES = (Private, ); }; };
0FF7168C15A3B235008F5DAA /* PropertyOffset.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FF7168A15A3B231008F5DAA /* PropertyOffset.h */; settings = {ATTRIBUTES = (Private, ); }; };
0FF922D414F46B410041A24E /* LLIntOffsetsExtractor.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F4680A114BA7F8200BFE272 /* LLIntOffsetsExtractor.cpp */; };
0FFFC95714EF90A000C72532 /* DFGCFAPhase.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0FFFC94B14EF909500C72532 /* DFGCFAPhase.cpp */; };
0FFFC95814EF90A200C72532 /* DFGCFAPhase.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FFFC94C14EF909500C72532 /* DFGCFAPhase.h */; settings = {ATTRIBUTES = (Private, ); }; };
......@@ -957,6 +958,7 @@
0FF427621591A1C9004CB9FF /* DFGDisassembler.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGDisassembler.h; path = dfg/DFGDisassembler.h; sourceTree = "<group>"; };
0FF4276E159275D2004CB9FF /* ResolveGlobalStatus.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = ResolveGlobalStatus.cpp; sourceTree = "<group>"; };
0FF4276F159275D2004CB9FF /* ResolveGlobalStatus.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ResolveGlobalStatus.h; sourceTree = "<group>"; };
0FF7168A15A3B231008F5DAA /* PropertyOffset.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = PropertyOffset.h; sourceTree = "<group>"; };
0FF922CF14F46B130041A24E /* JSCLLIntOffsetsExtractor */ = {isa = PBXFileReference; explicitFileType = "compiled.mach-o.executable"; includeInIndex = 0; path = JSCLLIntOffsetsExtractor; sourceTree = BUILT_PRODUCTS_DIR; };
0FFFC94B14EF909500C72532 /* DFGCFAPhase.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = DFGCFAPhase.cpp; path = dfg/DFGCFAPhase.cpp; sourceTree = "<group>"; };
0FFFC94C14EF909500C72532 /* DFGCFAPhase.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGCFAPhase.h; path = dfg/DFGCFAPhase.h; sourceTree = "<group>"; };
......@@ -2107,6 +2109,7 @@
86158AB2155C8B3F00B45C9C /* PropertyName.h */,
65400C0F0A69BAF200509887 /* PropertyNameArray.cpp */,
65400C100A69BAF200509887 /* PropertyNameArray.h */,
0FF7168A15A3B231008F5DAA /* PropertyOffset.h */,
65621E6B089E859700760F35 /* PropertySlot.cpp */,
65621E6C089E859700760F35 /* PropertySlot.h */,
65C02FBB0637462A003E7EE6 /* Protect.h */,
......@@ -2811,6 +2814,7 @@
0FF4274B158EBE91004CB9FF /* udis86.h in Headers */,
0FF427651591A1CE004CB9FF /* DFGDisassembler.h in Headers */,
0FF42772159275D8004CB9FF /* ResolveGlobalStatus.h in Headers */,
0FF7168C15A3B235008F5DAA /* PropertyOffset.h in Headers */,
);
runOnlyForDeploymentPostprocessing = 0;
};
......
......@@ -1018,6 +1018,12 @@ public:
else
m_formatter.twoWordOp12Reg4Reg4Imm12(OP_LDR_imm_T3, rn, rt, imm.getUInt12());
}
ALWAYS_INLINE void ldrWide8BitImmediate(RegisterID rt, RegisterID rn, uint8_t immediate)
{
ASSERT(rn != ARMRegisters::pc);
m_formatter.twoWordOp12Reg4Reg4Imm12(OP_LDR_imm_T3, rn, rt, immediate);
}
ALWAYS_INLINE void ldrCompact(RegisterID rt, RegisterID rn, ARMThumbImmediate imm)
{
......@@ -2117,6 +2123,46 @@ public:
{
return 6;
}
static void replaceWithLoad(void* instructionStart)
{
ASSERT(!(bitwise_cast<uintptr_t>(instructionStart) & 1));
uint16_t* ptr = reinterpret_cast<uint16_t*>(instructionStart);
switch (ptr[0] & 0xFFF0) {
case OP_LDR_imm_T3:
break;
case OP_ADD_imm_T3:
ASSERT(!(ptr[1] & 0xF000));
ptr[0] &= 0x000F;
ptr[0] |= OP_LDR_imm_T3;
ptr[1] |= (ptr[1] & 0x0F00) << 4;
ptr[1] &= 0xF0FF;
break;
default:
ASSERT_NOT_REACHED();
}
cacheFlush(ptr, sizeof(uint16_t) * 2);
}
static void replaceWithAddressComputation(void* instructionStart)
{
ASSERT(!(bitwise_cast<uintptr_t>(instructionStart) & 1));
uint16_t* ptr = reinterpret_cast<uint16_t*>(instructionStart);
switch (ptr[0] & 0xFFF0) {
case OP_LDR_imm_T3:
ASSERT(!(ptr[1] & 0x0F00));
ptr[0] &= 0x000F;
ptr[0] |= OP_ADD_imm_T3;
ptr[1] |= (ptr[1] & 0xF000) >> 4;
ptr[1] &= 0x0FFF;
break;
case OP_ADD_imm_T3:
break;
default:
ASSERT_NOT_REACHED();
}
cacheFlush(ptr, sizeof(uint16_t) * 2);
}
unsigned debugOffset() { return m_formatter.debugOffset(); }
......
......@@ -295,6 +295,36 @@ public:
private:
AssemblerLabel m_label;
};
// ConvertibleLoadLabel:
//
// A ConvertibleLoadLabel records a loadPtr instruction that can be patched to an addPtr
// so that:
//
// loadPtr(Address(a, i), b)
//
// becomes:
//
// addPtr(TrustedImmPtr(i), a, b)
class ConvertibleLoadLabel {
template<class TemplateAssemblerType>
friend class AbstractMacroAssembler;
friend class LinkBuffer;
public:
ConvertibleLoadLabel()
{
}
ConvertibleLoadLabel(AbstractMacroAssembler<AssemblerType>* masm)
: m_label(masm->m_assembler.labelIgnoringWatchpoints())
{
}
bool isSet() const { return m_label.isSet(); }
private:
AssemblerLabel m_label;
};
// DataLabelPtr:
//
......@@ -562,6 +592,11 @@ public:
result.m_label = m_assembler.labelIgnoringWatchpoints();
return result;
}
#else
Label labelIgnoringWatchpoints()
{
return label();
}
#endif
Label label()
......@@ -672,6 +707,16 @@ protected:
{
return AssemblerType::readPointer(dataLabelPtr.dataLocation());
}
static void replaceWithLoad(CodeLocationConvertibleLoad label)
{
AssemblerType::replaceWithLoad(label.dataLocation());
}
static void replaceWithAddressComputation(CodeLocationConvertibleLoad label)
{
AssemblerType::replaceWithAddressComputation(label.dataLocation());
}
};
} // namespace JSC
......
......@@ -40,6 +40,7 @@ class CodeLocationNearCall;
class CodeLocationDataLabelCompact;
class CodeLocationDataLabel32;
class CodeLocationDataLabelPtr;
class CodeLocationConvertibleLoad;
// The CodeLocation* types are all pretty much do-nothing wrappers around
// CodePtr (or MacroAssemblerCodePtr, to give it its full name). These
......@@ -62,6 +63,7 @@ public:
CodeLocationDataLabelPtr dataLabelPtrAtOffset(int offset);
CodeLocationDataLabel32 dataLabel32AtOffset(int offset);
CodeLocationDataLabelCompact dataLabelCompactAtOffset(int offset);
CodeLocationConvertibleLoad convertibleLoadAtOffset(int offset);
protected:
CodeLocationCommon()
......@@ -146,6 +148,15 @@ public:
: CodeLocationCommon(MacroAssemblerCodePtr(location)) {}
};
class CodeLocationConvertibleLoad : public CodeLocationCommon {
public:
CodeLocationConvertibleLoad() { }
explicit CodeLocationConvertibleLoad(MacroAssemblerCodePtr location)
: CodeLocationCommon(location) { }
explicit CodeLocationConvertibleLoad(void* location)
: CodeLocationCommon(MacroAssemblerCodePtr(location)) { }
};
inline CodeLocationInstruction CodeLocationCommon::instructionAtOffset(int offset)
{
ASSERT_VALID_CODE_OFFSET(offset);
......@@ -194,6 +205,12 @@ inline CodeLocationDataLabelCompact CodeLocationCommon::dataLabelCompactAtOffset
return CodeLocationDataLabelCompact(reinterpret_cast<char*>(dataLocation()) + offset);
}
inline CodeLocationConvertibleLoad CodeLocationCommon::convertibleLoadAtOffset(int offset)
{
ASSERT_VALID_CODE_OFFSET(offset);
return CodeLocationConvertibleLoad(reinterpret_cast<char*>(dataLocation()) + offset);
}
} // namespace JSC
#endif // ENABLE(ASSEMBLER)
......
......@@ -54,7 +54,7 @@ LinkBuffer::CodeRef LinkBuffer::finalizeCodeWithDisassembly(const char* format,
dataLog(" Code at [%p, %p):\n", result.code().executableAddress(), static_cast<char*>(result.code().executableAddress()) + result.size());
if (!tryToDisassemble(result.code(), m_size, " ", WTF::dataFile()))
dataLog(" <no disassembly available>");
dataLog(" <no disassembly available>\n");
return result;
}
......
......@@ -69,6 +69,7 @@ class LinkBuffer {
typedef MacroAssembler::DataLabelCompact DataLabelCompact;
typedef MacroAssembler::DataLabel32 DataLabel32;
typedef MacroAssembler::DataLabelPtr DataLabelPtr;
typedef MacroAssembler::ConvertibleLoadLabel ConvertibleLoadLabel;
#if ENABLE(BRANCH_COMPACTION)
typedef MacroAssembler::LinkRecord LinkRecord;
typedef MacroAssembler::JumpLinkType JumpLinkType;
......@@ -180,6 +181,11 @@ public:
return CodeLocationDataLabelCompact(MacroAssembler::getLinkerAddress(code(), applyOffset(label.m_label)));
}
CodeLocationConvertibleLoad locationOf(ConvertibleLoadLabel label)
{
return CodeLocationConvertibleLoad(MacroAssembler::getLinkerAddress(code(), applyOffset(label.m_label)));
}
// This method obtains the return address of the call, given as an offset from
// the start of the code.
unsigned returnAddressOffset(Call call)
......
......@@ -600,6 +600,14 @@ public:
move(TrustedImmPtr(address), addressTempRegister);
m_assembler.ldr(dest, addressTempRegister, ARMThumbImmediate::makeUInt16(0));
}
ConvertibleLoadLabel convertibleLoadPtr(Address address, RegisterID dest)
{
ConvertibleLoadLabel result(this);
ASSERT(address.offset >=0 && address.offset <= 255);
m_assembler.ldrWide8BitImmediate(dest, address.base, address.offset);
return result;
}
void load8(ImplicitAddress address, RegisterID dest)
{
......
......@@ -89,6 +89,13 @@ public:
m_assembler.movl_mr(address, dest);
}
ConvertibleLoadLabel convertibleLoadPtr(Address address, RegisterID dest)
{
ConvertibleLoadLabel result = ConvertibleLoadLabel(this);
m_assembler.movl_mr(address.offset, address.base, dest);
return result;
}
void addDouble(AbsoluteAddress address, FPRegisterID dest)
{
m_assembler.addsd_mr(address.m_ptr, dest);
......
......@@ -258,6 +258,13 @@ public:
m_assembler.movq_mr(address.offset, address.base, dest);
}
ConvertibleLoadLabel convertibleLoadPtr(Address address, RegisterID dest)
{
ConvertibleLoadLabel result = ConvertibleLoadLabel(this);
m_assembler.movq_mr(address.offset, address.base, dest);
return result;
}
void loadPtr(BaseIndex address, RegisterID dest)
{
m_assembler.movq_mr(address.offset, address.base, address.index, address.scale, dest);
......
......@@ -122,6 +122,24 @@ public:
{
relinkNearCallerToTrampoline(returnAddress, CodeLocationLabel(newCalleeFunction));
}
void replaceWithLoad(CodeLocationConvertibleLoad label)
{
MacroAssembler::replaceWithLoad(label);
}
void replaceWithAddressComputation(CodeLocationConvertibleLoad label)
{
MacroAssembler::replaceWithAddressComputation(label);
}
void setLoadInstructionIsActive(CodeLocationConvertibleLoad label, bool isActive)
{
if (isActive)
replaceWithLoad(label);
else
replaceWithAddressComputation(label);
}
private:
void* m_start;
......
......@@ -1839,6 +1839,42 @@ public:
return 5;
}
static void replaceWithLoad(void* instructionStart)
{
uint8_t* ptr = reinterpret_cast<uint8_t*>(instructionStart);
#if CPU(X86_64)
if ((*ptr & ~15) == PRE_REX)
ptr++;
#endif
switch (*ptr) {
case OP_MOV_GvEv:
break;
case OP_LEA:
*ptr = OP_MOV_GvEv;
break;
default:
ASSERT_NOT_REACHED();
}
}
static void replaceWithAddressComputation(void* instructionStart)
{
uint8_t* ptr = reinterpret_cast<uint8_t*>(instructionStart);
#if CPU(X86_64)
if ((*ptr & ~15) == PRE_REX)
ptr++;
#endif
switch (*ptr) {
case OP_MOV_GvEv:
*ptr = OP_LEA;
break;
case OP_LEA:
break;
default:
ASSERT_NOT_REACHED();
}
}
static unsigned getCallReturnOffset(AssemblerLabel call)
{
ASSERT(call.isSet());
......
......@@ -197,6 +197,9 @@ void CodeBlock::printGetByIdOp(ExecState* exec, int location, Vector<Instruction
case op_get_by_id:
op = "get_by_id";
break;
case op_get_by_id_out_of_line:
op = "get_by_id_out_of_line";
break;
case op_get_by_id_self:
op = "get_by_id_self";
break;
......@@ -1033,6 +1036,7 @@ void CodeBlock::dump(ExecState* exec, const Vector<Instruction>::const_iterator&
break;
}
case op_get_by_id:
case op_get_by_id_out_of_line:
case op_get_by_id_self:
case op_get_by_id_proto:
case op_get_by_id_chain:
......@@ -1059,6 +1063,10 @@ void CodeBlock::dump(ExecState* exec, const Vector<Instruction>::const_iterator&
printPutByIdOp(exec, location, it, "put_by_id");
break;
}
case op_put_by_id_out_of_line: {
printPutByIdOp(exec, location, it, "put_by_id_out_of_line");
break;
}
case op_put_by_id_replace: {
printPutByIdOp(exec, location, it, "put_by_id_replace");
break;
......@@ -1071,10 +1079,18 @@ void CodeBlock::dump(ExecState* exec, const Vector<Instruction>::const_iterator&
printPutByIdOp(exec, location, it, "put_by_id_transition_direct");
break;
}
case op_put_by_id_transition_direct_out_of_line: {
printPutByIdOp(exec, location, it, "put_by_id_transition_direct_out_of_line");
break;
}
case op_put_by_id_transition_normal: {
printPutByIdOp(exec, location, it, "put_by_id_transition_normal");
break;
}
case op_put_by_id_transition_normal_out_of_line: {
printPutByIdOp(exec, location, it, "put_by_id_transition_normal_out_of_line");
break;
}
case op_put_by_id_generic: {
printPutByIdOp(exec, location, it, "put_by_id_generic");
break;
......@@ -2029,7 +2045,9 @@ void CodeBlock::finalizeUnconditionally()
Instruction* curInstruction = &instructions()[m_propertyAccessInstructions[i]];
switch (interpreter->getOpcodeID(curInstruction[0].u.opcode)) {
case op_get_by_id:
case op_get_by_id_out_of_line:
case op_put_by_id:
case op_put_by_id_out_of_line:
if (!curInstruction[4].u.structure || Heap::isMarked(curInstruction[4].u.structure.get()))
break;
if (verboseUnlinking)
......@@ -2039,6 +2057,8 @@ void CodeBlock::finalizeUnconditionally()
break;
case op_put_by_id_transition_direct:
case op_put_by_id_transition_normal:
case op_put_by_id_transition_direct_out_of_line:
case op_put_by_id_transition_normal_out_of_line:
if (Heap::isMarked(curInstruction[4].u.structure.get())
&& Heap::isMarked(curInstruction[6].u.structure.get())
&& Heap::isMarked(curInstruction[7].u.structureChain.get()))
......
......@@ -48,9 +48,9 @@ GetByIdStatus GetByIdStatus::computeFromLLInt(CodeBlock* profiledBlock, unsigned
unsigned attributesIgnored;
JSCell* specificValue;
size_t offset = structure->get(
PropertyOffset offset = structure->get(
*profiledBlock->globalData(), ident, attributesIgnored, specificValue);
if (offset == notFound)
if (!isValidOffset(offset))
return GetByIdStatus(NoInformation, false);
return GetByIdStatus(Simple, false, StructureSet(structure), offset, specificValue);
......@@ -88,7 +88,7 @@ void GetByIdStatus::computeForChain(GetByIdStatus& result, CodeBlock* profiledBl
result.m_offset = currentStructure->get(
*profiledBlock->globalData(), ident, attributesIgnored, specificValue);
if (result.m_offset == notFound)
if (!isValidOffset(result.m_offset))
return;
result.m_structureSet.add(structure);
......@@ -156,12 +156,12 @@ GetByIdStatus GetByIdStatus::computeFor(CodeBlock* profiledBlock, unsigned bytec
result.m_offset = structure->get(
*profiledBlock->globalData(), ident, attributesIgnored, specificValue);
if (result.m_offset != notFound) {
if (isValidOffset(result.m_offset)) {
result.m_structureSet.add(structure);
result.m_specificValue = JSValue(specificValue);
}
if (result.m_offset != notFound)
if (isValidOffset(result.m_offset))
ASSERT(result.m_structureSet.size());
break;
}
......@@ -176,11 +176,11 @@ GetByIdStatus GetByIdStatus::computeFor(CodeBlock* profiledBlock, unsigned bytec
unsigned attributesIgnored;
JSCell* specificValue;
size_t myOffset = structure->get(
PropertyOffset myOffset = structure->get(
*profiledBlock->globalData(), ident, attributesIgnored, specificValue);
if (myOffset == notFound) {
result.m_offset = notFound;
if (!isValidOffset(myOffset)) {
result.m_offset = invalidOffset;
break;
}
......@@ -188,7 +188,7 @@ GetByIdStatus GetByIdStatus::computeFor(CodeBlock* profiledBlock, unsigned bytec
result.m_offset = myOffset;
result.m_specificValue = JSValue(specificValue);
} else if (result.m_offset != myOffset) {
result.m_offset = notFound;
result.m_offset = invalidOffset;
break;
} else if (result.m_specificValue != JSValue(specificValue))
result.m_specificValue = JSValue();
......@@ -196,7 +196,7 @@ GetByIdStatus GetByIdStatus::computeFor(CodeBlock* profiledBlock, unsigned bytec
result.m_structureSet.add(structure);
}
if (result.m_offset != notFound)
if (isValidOffset(result.m_offset))
ASSERT(result.m_structureSet.size());
break;
}
......@@ -223,11 +223,11 @@ GetByIdStatus GetByIdStatus::computeFor(CodeBlock* profiledBlock, unsigned bytec
}
default:
ASSERT(result.m_offset == notFound);
ASSERT(!isValidOffset(result.m_offset));
break;
}
if (result.m_offset == notFound) {
if (!isValidOffset(result.m_offset)) {
result.m_state = TakesSlowPath;
result.m_structureSet.clear();
result.m_chain.clear();
......
......@@ -26,6 +26,7 @@
#ifndef GetByIdStatus_h
#define GetByIdStatus_h
#include "PropertyOffset.h"
#include "StructureSet.h"
#include <wtf/NotFound.h>
......@@ -46,13 +47,13 @@ public:
GetByIdStatus()
: m_state(NoInformation)
, m_offset(notFound)
, m_offset(invalidOffset)
{
}
GetByIdStatus(
State state, bool wasSeenInJIT, const StructureSet& structureSet = StructureSet(),
size_t offset = notFound, JSValue specificValue = JSValue(), Vector<Structure*> chain = Vector<Structure*>())
size_t offset = invalidOffset, JSValue specificValue = JSValue(), Vector<Structure*> chain = Vector<Structure*>())
: m_state(state)
, m_structureSet(structureSet)
, m_chain(chain)
......@@ -76,7 +77,7 @@ public:
const StructureSet& structureSet() const { return m_structureSet; }
const Vector<Structure*>& chain() const { return m_chain; } // Returns empty vector if this is a direct access.
JSValue specificValue() const { return m_specificValue; } // Returns JSValue() if there is no specific value.
size_t offset() const { return m_offset; }
PropertyOffset offset() const { return m_offset; }
bool wasSeenInJIT() const { return m_wasSeenInJIT; }
......@@ -88,7 +89,7 @@ private:
StructureSet m_structureSet;
Vector<Structure*> m_chain;
JSValue m_specificValue;
size_t m_offset;
PropertyOffset m_offset;
bool m_wasSeenInJIT;
};
......
......@@ -108,6 +108,7 @@ namespace JSC {
macro(op_resolve_with_base, 5) /* has value profiling */ \
macro(op_resolve_with_this, 5) /* has value profiling */ \
macro(op_get_by_id, 9) /* has value profiling */ \
macro(op_get_by_id_out_of_line, 9) /* has value profiling */ \
macro(op_get_by_id_self, 9) /* has value profiling */ \
macro(op_get_by_id_proto, 9) /* has value profiling */ \
macro(op_get_by_id_chain, 9) /* has value profiling */ \
......@@ -122,9 +123,12 @@ namespace JSC {
macro(op_get_string_length, 9) /* has value profiling */ \
macro(op_get_arguments_length, 4) \
macro(op_put_by_id, 9) \
macro(op_put_by_id_out_of_line, 9) \
macro(op_put_by_id_transition, 9) \
macro(op_put_by_id_transition_direct, 9) \
macro(op_put_by_id_transition_direct_out_of_line, 9) \
macro(op_put_by_id_transition_normal, 9) \
macro(op_put_by_id_transition_normal_out_of_line, 9) \
macro(op_put_by_id_replace, 9) \
macro(op_put_by_id_generic, 9) \
macro(op_del_by_id, 4) \
......
......@@ -43,12 +43,13 @@ PutByIdStatus PutByIdStatus::computeFromLLInt(CodeBlock* profiledBlock, unsigned
Structure* structure = instruction[4].u.structure.get();