Commit 2e342af4 authored by barraclough@apple.com's avatar barraclough@apple.com

2009-08-06 Gavin Barraclough <barraclough@apple.com>

        Reviewed by Oliver Hunt.

        Make get_by_id/put_by_id/method_check/call defer optimization using a data flag rather than a code modification.
        ( https://bugs.webkit.org/show_bug.cgi?id=27635 )

        This improves performance of ENABLE(ASSEMBLER_WX_EXCLUSIVE) builds by 2-2.5%, reducing the overhead to about 2.5%.
        (No performance impact with ASSEMBLER_WX_EXCLUSIVE disabled).

        * bytecode/CodeBlock.cpp:
        (JSC::printStructureStubInfo):
            - Make StructureStubInfo store the type as an integer, rather than an OpcodeID.

        * bytecode/CodeBlock.h:
        (JSC::):
        (JSC::CallLinkInfo::seenOnce):
        (JSC::CallLinkInfo::setSeen):
        (JSC::MethodCallLinkInfo::seenOnce):
        (JSC::MethodCallLinkInfo::setSeen):
            - Change a pointer in CallLinkInfo/MethodCallLinkInfo to use a PtrAndFlags, use a flag to track when an op has been executed once.

        * bytecode/StructureStubInfo.cpp:
        (JSC::StructureStubInfo::deref):
            - Make StructureStubInfo store the type as an integer, rather than an OpcodeID.

        * bytecode/StructureStubInfo.h:
        (JSC::StructureStubInfo::StructureStubInfo):
        (JSC::StructureStubInfo::initGetByIdSelf):
        (JSC::StructureStubInfo::initGetByIdProto):
        (JSC::StructureStubInfo::initGetByIdChain):
        (JSC::StructureStubInfo::initGetByIdSelfList):
        (JSC::StructureStubInfo::initGetByIdProtoList):
        (JSC::StructureStubInfo::initPutByIdTransition):
        (JSC::StructureStubInfo::initPutByIdReplace):
        (JSC::StructureStubInfo::seenOnce):
        (JSC::StructureStubInfo::setSeen):
            - Make StructureStubInfo store the type as an integer, rather than an OpcodeID, add a flag to track when an op has been executed once.

        * bytecompiler/BytecodeGenerator.cpp:
        (JSC::BytecodeGenerator::emitGetById):
        (JSC::BytecodeGenerator::emitPutById):
            - Make StructureStubInfo store the type as an integer, rather than an OpcodeID.

        * jit/JIT.cpp:
        (JSC::JIT::privateCompileCTIMachineTrampolines):
        (JSC::JIT::unlinkCall):
            - Remove the "don't lazy link" stage of calls.

        * jit/JIT.h:
        (JSC::JIT::compileCTIMachineTrampolines):
            - Remove the "don't lazy link" stage of calls.

        * jit/JITCall.cpp:
        (JSC::JIT::compileOpCallSlowCase):
            - Remove the "don't lazy link" stage of calls.

        * jit/JITStubs.cpp:
        (JSC::JITThunks::JITThunks):
        (JSC::JITThunks::tryCachePutByID):
        (JSC::JITThunks::tryCacheGetByID):
        (JSC::JITStubs::DEFINE_STUB_FUNCTION):
        (JSC::JITStubs::getPolymorphicAccessStructureListSlot):
            - Remove the "don't lazy link" stage of calls, and the "_second" stage of get_by_id/put_by_id/method_check.

        * jit/JITStubs.h:
        (JSC::JITThunks::ctiStringLengthTrampoline):
        (JSC::JITStubs::):
            - Remove the "don't lazy link" stage of calls, and the "_second" stage of get_by_id/put_by_id/method_check.

        * wtf/PtrAndFlags.h:
        (WTF::PtrAndFlags::PtrAndFlags):
        (WTF::PtrAndFlags::operator!):
        (WTF::PtrAndFlags::operator->):
            - Add ! and -> operators, add constuctor with pointer argument.



git-svn-id: http://svn.webkit.org/repository/webkit/trunk@46879 268f45cc-cd09-0410-ab3c-d52691b4dbfc
parent 92d485ea
2009-08-06 Gavin Barraclough <barraclough@apple.com>
Reviewed by Oliver Hunt.
Make get_by_id/put_by_id/method_check/call defer optimization using a data flag rather than a code modification.
( https://bugs.webkit.org/show_bug.cgi?id=27635 )
This improves performance of ENABLE(ASSEMBLER_WX_EXCLUSIVE) builds by 2-2.5%, reducing the overhead to about 2.5%.
(No performance impact with ASSEMBLER_WX_EXCLUSIVE disabled).
* bytecode/CodeBlock.cpp:
(JSC::printStructureStubInfo):
- Make StructureStubInfo store the type as an integer, rather than an OpcodeID.
* bytecode/CodeBlock.h:
(JSC::):
(JSC::CallLinkInfo::seenOnce):
(JSC::CallLinkInfo::setSeen):
(JSC::MethodCallLinkInfo::seenOnce):
(JSC::MethodCallLinkInfo::setSeen):
- Change a pointer in CallLinkInfo/MethodCallLinkInfo to use a PtrAndFlags, use a flag to track when an op has been executed once.
* bytecode/StructureStubInfo.cpp:
(JSC::StructureStubInfo::deref):
- Make StructureStubInfo store the type as an integer, rather than an OpcodeID.
* bytecode/StructureStubInfo.h:
(JSC::StructureStubInfo::StructureStubInfo):
(JSC::StructureStubInfo::initGetByIdSelf):
(JSC::StructureStubInfo::initGetByIdProto):
(JSC::StructureStubInfo::initGetByIdChain):
(JSC::StructureStubInfo::initGetByIdSelfList):
(JSC::StructureStubInfo::initGetByIdProtoList):
(JSC::StructureStubInfo::initPutByIdTransition):
(JSC::StructureStubInfo::initPutByIdReplace):
(JSC::StructureStubInfo::seenOnce):
(JSC::StructureStubInfo::setSeen):
- Make StructureStubInfo store the type as an integer, rather than an OpcodeID, add a flag to track when an op has been executed once.
* bytecompiler/BytecodeGenerator.cpp:
(JSC::BytecodeGenerator::emitGetById):
(JSC::BytecodeGenerator::emitPutById):
- Make StructureStubInfo store the type as an integer, rather than an OpcodeID.
* jit/JIT.cpp:
(JSC::JIT::privateCompileCTIMachineTrampolines):
(JSC::JIT::unlinkCall):
- Remove the "don't lazy link" stage of calls.
* jit/JIT.h:
(JSC::JIT::compileCTIMachineTrampolines):
- Remove the "don't lazy link" stage of calls.
* jit/JITCall.cpp:
(JSC::JIT::compileOpCallSlowCase):
- Remove the "don't lazy link" stage of calls.
* jit/JITStubs.cpp:
(JSC::JITThunks::JITThunks):
(JSC::JITThunks::tryCachePutByID):
(JSC::JITThunks::tryCacheGetByID):
(JSC::JITStubs::DEFINE_STUB_FUNCTION):
(JSC::JITStubs::getPolymorphicAccessStructureListSlot):
- Remove the "don't lazy link" stage of calls, and the "_second" stage of get_by_id/put_by_id/method_check.
* jit/JITStubs.h:
(JSC::JITThunks::ctiStringLengthTrampoline):
(JSC::JITStubs::):
- Remove the "don't lazy link" stage of calls, and the "_second" stage of get_by_id/put_by_id/method_check.
* wtf/PtrAndFlags.h:
(WTF::PtrAndFlags::PtrAndFlags):
(WTF::PtrAndFlags::operator!):
(WTF::PtrAndFlags::operator->):
- Add ! and -> operators, add constuctor with pointer argument.
2009-08-06 Zoltan Horvath <hzoltan@inf.u-szeged.hu>
Reviewed by Adam Barth.
......
......@@ -230,44 +230,44 @@ static void printGlobalResolveInfo(const GlobalResolveInfo& resolveInfo, unsigne
static void printStructureStubInfo(const StructureStubInfo& stubInfo, unsigned instructionOffset)
{
switch (stubInfo.opcodeID) {
case op_get_by_id_self:
switch (stubInfo.accessType) {
case access_get_by_id_self:
printf(" [%4d] %s: %s\n", instructionOffset, "get_by_id_self", pointerToSourceString(stubInfo.u.getByIdSelf.baseObjectStructure).UTF8String().c_str());
return;
case op_get_by_id_proto:
case access_get_by_id_proto:
printf(" [%4d] %s: %s, %s\n", instructionOffset, "get_by_id_proto", pointerToSourceString(stubInfo.u.getByIdProto.baseObjectStructure).UTF8String().c_str(), pointerToSourceString(stubInfo.u.getByIdProto.prototypeStructure).UTF8String().c_str());
return;
case op_get_by_id_chain:
case access_get_by_id_chain:
printf(" [%4d] %s: %s, %s\n", instructionOffset, "get_by_id_chain", pointerToSourceString(stubInfo.u.getByIdChain.baseObjectStructure).UTF8String().c_str(), pointerToSourceString(stubInfo.u.getByIdChain.chain).UTF8String().c_str());
return;
case op_get_by_id_self_list:
case access_get_by_id_self_list:
printf(" [%4d] %s: %s (%d)\n", instructionOffset, "op_get_by_id_self_list", pointerToSourceString(stubInfo.u.getByIdSelfList.structureList).UTF8String().c_str(), stubInfo.u.getByIdSelfList.listSize);
return;
case op_get_by_id_proto_list:
case access_get_by_id_proto_list:
printf(" [%4d] %s: %s (%d)\n", instructionOffset, "op_get_by_id_proto_list", pointerToSourceString(stubInfo.u.getByIdProtoList.structureList).UTF8String().c_str(), stubInfo.u.getByIdProtoList.listSize);
return;
case op_put_by_id_transition:
case access_put_by_id_transition:
printf(" [%4d] %s: %s, %s, %s\n", instructionOffset, "put_by_id_transition", pointerToSourceString(stubInfo.u.putByIdTransition.previousStructure).UTF8String().c_str(), pointerToSourceString(stubInfo.u.putByIdTransition.structure).UTF8String().c_str(), pointerToSourceString(stubInfo.u.putByIdTransition.chain).UTF8String().c_str());
return;
case op_put_by_id_replace:
case access_put_by_id_replace:
printf(" [%4d] %s: %s\n", instructionOffset, "put_by_id_replace", pointerToSourceString(stubInfo.u.putByIdReplace.baseObjectStructure).UTF8String().c_str());
return;
case op_get_by_id:
case access_get_by_id:
printf(" [%4d] %s\n", instructionOffset, "get_by_id");
return;
case op_put_by_id:
case access_put_by_id:
printf(" [%4d] %s\n", instructionOffset, "put_by_id");
return;
case op_get_by_id_generic:
case access_get_by_id_generic:
printf(" [%4d] %s\n", instructionOffset, "op_get_by_id_generic");
return;
case op_put_by_id_generic:
case access_put_by_id_generic:
printf(" [%4d] %s\n", instructionOffset, "op_put_by_id_generic");
return;
case op_get_array_length:
case access_get_array_length:
printf(" [%4d] %s\n", instructionOffset, "op_get_array_length");
return;
case op_get_string_length:
case access_get_string_length:
printf(" [%4d] %s\n", instructionOffset, "op_get_string_length");
return;
default:
......@@ -1319,7 +1319,7 @@ CodeBlock::~CodeBlock()
if (Structure* structure = m_methodCallLinkInfos[i].cachedStructure) {
structure->deref();
// Both members must be filled at the same time
ASSERT(m_methodCallLinkInfos[i].cachedPrototypeStructure);
ASSERT(!!m_methodCallLinkInfos[i].cachedPrototypeStructure);
m_methodCallLinkInfos[i].cachedPrototypeStructure->deref();
}
}
......
......@@ -36,6 +36,7 @@
#include "JSGlobalObject.h"
#include "JumpTable.h"
#include "Nodes.h"
#include "PtrAndFlags.h"
#include "RegExp.h"
#include "UString.h"
#include <wtf/FastAllocBase.h>
......@@ -54,6 +55,10 @@ static const int FirstConstantRegisterIndex = 0x40000000;
namespace JSC {
enum HasSeenShouldRepatch {
hasSeenShouldRepatch
};
class ExecState;
enum CodeType { GlobalCode, EvalCode, FunctionCode, NativeCode };
......@@ -105,25 +110,44 @@ namespace JSC {
CodeLocationNearCall callReturnLocation;
CodeLocationDataLabelPtr hotPathBegin;
CodeLocationNearCall hotPathOther;
CodeBlock* ownerCodeBlock;
PtrAndFlags<CodeBlock, HasSeenShouldRepatch> ownerCodeBlock;
CodeBlock* callee;
unsigned position;
void setUnlinked() { callee = 0; }
bool isLinked() { return callee; }
bool seenOnce()
{
return ownerCodeBlock.isFlagSet(hasSeenShouldRepatch);
}
void setSeen()
{
ownerCodeBlock.setFlag(hasSeenShouldRepatch);
}
};
struct MethodCallLinkInfo {
MethodCallLinkInfo()
: cachedStructure(0)
, cachedPrototypeStructure(0)
{
}
bool seenOnce()
{
return cachedPrototypeStructure.isFlagSet(hasSeenShouldRepatch);
}
void setSeen()
{
cachedPrototypeStructure.setFlag(hasSeenShouldRepatch);
}
CodeLocationCall callReturnLocation;
CodeLocationDataLabelPtr structureLabel;
Structure* cachedStructure;
Structure* cachedPrototypeStructure;
PtrAndFlags<Structure, HasSeenShouldRepatch> cachedPrototypeStructure;
};
struct FunctionRegisterInfo {
......
......@@ -31,44 +31,44 @@ namespace JSC {
#if ENABLE(JIT)
void StructureStubInfo::deref()
{
switch (opcodeID) {
case op_get_by_id_self:
switch (accessType) {
case access_get_by_id_self:
u.getByIdSelf.baseObjectStructure->deref();
return;
case op_get_by_id_proto:
case access_get_by_id_proto:
u.getByIdProto.baseObjectStructure->deref();
u.getByIdProto.prototypeStructure->deref();
return;
case op_get_by_id_chain:
case access_get_by_id_chain:
u.getByIdChain.baseObjectStructure->deref();
u.getByIdChain.chain->deref();
return;
case op_get_by_id_self_list: {
case access_get_by_id_self_list: {
PolymorphicAccessStructureList* polymorphicStructures = u.getByIdSelfList.structureList;
polymorphicStructures->derefStructures(u.getByIdSelfList.listSize);
delete polymorphicStructures;
return;
}
case op_get_by_id_proto_list: {
case access_get_by_id_proto_list: {
PolymorphicAccessStructureList* polymorphicStructures = u.getByIdProtoList.structureList;
polymorphicStructures->derefStructures(u.getByIdProtoList.listSize);
delete polymorphicStructures;
return;
}
case op_put_by_id_transition:
case access_put_by_id_transition:
u.putByIdTransition.previousStructure->deref();
u.putByIdTransition.structure->deref();
u.putByIdTransition.chain->deref();
return;
case op_put_by_id_replace:
case access_put_by_id_replace:
u.putByIdReplace.baseObjectStructure->deref();
return;
case op_get_by_id:
case op_put_by_id:
case op_get_by_id_generic:
case op_put_by_id_generic:
case op_get_array_length:
case op_get_string_length:
case access_get_by_id:
case access_put_by_id:
case access_get_by_id_generic:
case access_put_by_id_generic:
case access_get_array_length:
case access_get_string_length:
// These instructions don't ref their Structures.
return;
default:
......
......@@ -35,15 +35,32 @@
namespace JSC {
enum AccessType {
access_get_by_id_self,
access_get_by_id_proto,
access_get_by_id_chain,
access_get_by_id_self_list,
access_get_by_id_proto_list,
access_put_by_id_transition,
access_put_by_id_replace,
access_get_by_id,
access_put_by_id,
access_get_by_id_generic,
access_put_by_id_generic,
access_get_array_length,
access_get_string_length,
};
struct StructureStubInfo {
StructureStubInfo(OpcodeID opcodeID)
: opcodeID(opcodeID)
StructureStubInfo(AccessType accessType)
: accessType(accessType)
, seen(false)
{
}
void initGetByIdSelf(Structure* baseObjectStructure)
{
opcodeID = op_get_by_id_self;
accessType = access_get_by_id_self;
u.getByIdSelf.baseObjectStructure = baseObjectStructure;
baseObjectStructure->ref();
......@@ -51,7 +68,7 @@ namespace JSC {
void initGetByIdProto(Structure* baseObjectStructure, Structure* prototypeStructure)
{
opcodeID = op_get_by_id_proto;
accessType = access_get_by_id_proto;
u.getByIdProto.baseObjectStructure = baseObjectStructure;
baseObjectStructure->ref();
......@@ -62,7 +79,7 @@ namespace JSC {
void initGetByIdChain(Structure* baseObjectStructure, StructureChain* chain)
{
opcodeID = op_get_by_id_chain;
accessType = access_get_by_id_chain;
u.getByIdChain.baseObjectStructure = baseObjectStructure;
baseObjectStructure->ref();
......@@ -73,7 +90,7 @@ namespace JSC {
void initGetByIdSelfList(PolymorphicAccessStructureList* structureList, int listSize)
{
opcodeID = op_get_by_id_self_list;
accessType = access_get_by_id_self_list;
u.getByIdProtoList.structureList = structureList;
u.getByIdProtoList.listSize = listSize;
......@@ -81,7 +98,7 @@ namespace JSC {
void initGetByIdProtoList(PolymorphicAccessStructureList* structureList, int listSize)
{
opcodeID = op_get_by_id_proto_list;
accessType = access_get_by_id_proto_list;
u.getByIdProtoList.structureList = structureList;
u.getByIdProtoList.listSize = listSize;
......@@ -91,7 +108,7 @@ namespace JSC {
void initPutByIdTransition(Structure* previousStructure, Structure* structure, StructureChain* chain)
{
opcodeID = op_put_by_id_transition;
accessType = access_put_by_id_transition;
u.putByIdTransition.previousStructure = previousStructure;
previousStructure->ref();
......@@ -105,7 +122,7 @@ namespace JSC {
void initPutByIdReplace(Structure* baseObjectStructure)
{
opcodeID = op_put_by_id_replace;
accessType = access_put_by_id_replace;
u.putByIdReplace.baseObjectStructure = baseObjectStructure;
baseObjectStructure->ref();
......@@ -113,7 +130,19 @@ namespace JSC {
void deref();
OpcodeID opcodeID;
bool seenOnce()
{
return seen;
}
void setSeen()
{
seen = true;
}
int accessType : 31;
int seen : 1;
union {
struct {
Structure* baseObjectStructure;
......
......@@ -1190,7 +1190,7 @@ void BytecodeGenerator::emitMethodCheck()
RegisterID* BytecodeGenerator::emitGetById(RegisterID* dst, RegisterID* base, const Identifier& property)
{
#if ENABLE(JIT)
m_codeBlock->addStructureStubInfo(StructureStubInfo(op_get_by_id));
m_codeBlock->addStructureStubInfo(StructureStubInfo(access_get_by_id));
#else
m_codeBlock->addPropertyAccessInstruction(instructions().size());
#endif
......@@ -1209,7 +1209,7 @@ RegisterID* BytecodeGenerator::emitGetById(RegisterID* dst, RegisterID* base, co
RegisterID* BytecodeGenerator::emitPutById(RegisterID* base, const Identifier& property, RegisterID* value)
{
#if ENABLE(JIT)
m_codeBlock->addStructureStubInfo(StructureStubInfo(op_put_by_id));
m_codeBlock->addStructureStubInfo(StructureStubInfo(access_put_by_id));
#else
m_codeBlock->addPropertyAccessInstruction(instructions().size());
#endif
......
......@@ -577,7 +577,7 @@ void JIT::unlinkCall(CallLinkInfo* callLinkInfo)
// When the JSFunction is deleted the pointer embedded in the instruction stream will no longer be valid
// (and, if a new JSFunction happened to be constructed at the same location, we could get a false positive
// match). Reset the check so it no longer matches.
RepatchBuffer repatchBuffer(callLinkInfo->ownerCodeBlock);
RepatchBuffer repatchBuffer(callLinkInfo->ownerCodeBlock.get());
#if USE(JSVALUE32_64)
repatchBuffer.repatch(callLinkInfo->hotPathBegin, 0);
#else
......
......@@ -317,15 +317,15 @@ namespace JSC {
jit.privateCompilePutByIdTransition(stubInfo, oldStructure, newStructure, cachedOffset, chain, returnAddress);
}
static void compileCTIMachineTrampolines(JSGlobalData* globalData, RefPtr<ExecutablePool>* executablePool, CodePtr* ctiStringLengthTrampoline, CodePtr* ctiVirtualCallPreLink, CodePtr* ctiVirtualCallLink, CodePtr* ctiVirtualCall, CodePtr* ctiNativeCallThunk)
static void compileCTIMachineTrampolines(JSGlobalData* globalData, RefPtr<ExecutablePool>* executablePool, CodePtr* ctiStringLengthTrampoline, CodePtr* ctiVirtualCallLink, CodePtr* ctiVirtualCall, CodePtr* ctiNativeCallThunk)
{
JIT jit(globalData);
jit.privateCompileCTIMachineTrampolines(executablePool, globalData, ctiStringLengthTrampoline, ctiVirtualCallPreLink, ctiVirtualCallLink, ctiVirtualCall, ctiNativeCallThunk);
jit.privateCompileCTIMachineTrampolines(executablePool, globalData, ctiStringLengthTrampoline, ctiVirtualCallLink, ctiVirtualCall, ctiNativeCallThunk);
}
static void patchGetByIdSelf(CodeBlock* codeblock, StructureStubInfo*, Structure*, size_t cachedOffset, ReturnAddressPtr returnAddress);
static void patchPutByIdReplace(CodeBlock* codeblock, StructureStubInfo*, Structure*, size_t cachedOffset, ReturnAddressPtr returnAddress);
static void patchMethodCallProto(CodeBlock* codeblock, MethodCallLinkInfo&, JSFunction*, Structure*, JSObject*);
static void patchMethodCallProto(CodeBlock* codeblock, MethodCallLinkInfo&, JSFunction*, Structure*, JSObject*, ReturnAddressPtr);
static void compilePatchGetArrayLength(JSGlobalData* globalData, CodeBlock* codeBlock, ReturnAddressPtr returnAddress)
{
......@@ -361,7 +361,7 @@ namespace JSC {
void privateCompileGetByIdChain(StructureStubInfo*, Structure*, StructureChain*, size_t count, size_t cachedOffset, ReturnAddressPtr returnAddress, CallFrame* callFrame);
void privateCompilePutByIdTransition(StructureStubInfo*, Structure*, Structure*, size_t cachedOffset, StructureChain*, ReturnAddressPtr returnAddress);
void privateCompileCTIMachineTrampolines(RefPtr<ExecutablePool>* executablePool, JSGlobalData* data, CodePtr* ctiStringLengthTrampoline, CodePtr* ctiVirtualCallPreLink, CodePtr* ctiVirtualCallLink, CodePtr* ctiVirtualCall, CodePtr* ctiNativeCallThunk);
void privateCompileCTIMachineTrampolines(RefPtr<ExecutablePool>* executablePool, JSGlobalData* data, CodePtr* ctiStringLengthTrampoline, CodePtr* ctiVirtualCallLink, CodePtr* ctiVirtualCall, CodePtr* ctiNativeCallThunk);
void privateCompilePatchGetArrayLength(ReturnAddressPtr returnAddress);
void addSlowCase(Jump);
......
......@@ -405,7 +405,7 @@ void JIT::compileOpCallSlowCase(Instruction* instruction, Vector<SlowCaseEntry>:
addPtr(Imm32(registerOffset * static_cast<int>(sizeof(Register))), callFrameRegister);
move(Imm32(argCount), regT1);
m_callStructureStubCompilationInfo[callLinkInfoIndex].callReturnLocation = emitNakedCall(m_globalData->jitStubs.ctiVirtualCallPreLink());
m_callStructureStubCompilationInfo[callLinkInfoIndex].callReturnLocation = emitNakedCall(m_globalData->jitStubs.ctiVirtualCallLink());
// Put the return value in dst.
emitStore(dst, regT1, regT0);;
......@@ -688,7 +688,7 @@ void JIT::compileOpCallSlowCase(Instruction* instruction, Vector<SlowCaseEntry>:
addPtr(Imm32(registerOffset * static_cast<int>(sizeof(Register))), callFrameRegister);
move(Imm32(argCount), regT1);
m_callStructureStubCompilationInfo[callLinkInfoIndex].callReturnLocation = emitNakedCall(m_globalData->jitStubs.ctiVirtualCallPreLink());
m_callStructureStubCompilationInfo[callLinkInfoIndex].callReturnLocation = emitNakedCall(m_globalData->jitStubs.ctiVirtualCallLink());
// Put the return value in dst.
emitPutVirtualRegister(dst);
......
......@@ -39,7 +39,7 @@ namespace JSC {
#if USE(JSVALUE32_64)
void JIT::privateCompileCTIMachineTrampolines(RefPtr<ExecutablePool>* executablePool, JSGlobalData* globalData, CodePtr* ctiStringLengthTrampoline, CodePtr* ctiVirtualCallPreLink, CodePtr* ctiVirtualCallLink, CodePtr* ctiVirtualCall, CodePtr* ctiNativeCallThunk)
void JIT::privateCompileCTIMachineTrampolines(RefPtr<ExecutablePool>* executablePool, JSGlobalData* globalData, CodePtr* ctiStringLengthTrampoline, CodePtr* ctiVirtualCallLink, CodePtr* ctiVirtualCall, CodePtr* ctiNativeCallThunk)
{
#if ENABLE(JIT_OPTIMIZE_PROPERTY_ACCESS)
// (1) This function provides fast property access for string length
......@@ -64,51 +64,6 @@ void JIT::privateCompileCTIMachineTrampolines(RefPtr<ExecutablePool>* executable
// (2) Trampolines for the slow cases of op_call / op_call_eval / op_construct.
#if ENABLE(JIT_OPTIMIZE_CALL)
/* VirtualCallPreLink Trampoline */
Label virtualCallPreLinkBegin = align();
// regT0 holds callee, regT1 holds argCount.
loadPtr(Address(regT0, OBJECT_OFFSETOF(JSFunction, m_body)), regT2);
loadPtr(Address(regT2, OBJECT_OFFSETOF(FunctionBodyNode, m_code)), regT2);
Jump hasCodeBlock1 = branchTestPtr(NonZero, regT2);
// Lazily generate a CodeBlock.
preserveReturnAddressAfterCall(regT3); // return address
restoreArgumentReference();
Call callJSFunction1 = call();
move(regT0, regT2);
emitGetJITStubArg(1, regT0); // callee
emitGetJITStubArg(5, regT1); // argCount
restoreReturnAddressBeforeReturn(regT3); // return address
hasCodeBlock1.link(this);
// regT2 holds codeBlock.
Jump isNativeFunc1 = branch32(Equal, Address(regT2, OBJECT_OFFSETOF(CodeBlock, m_codeType)), Imm32(NativeCode));
// Check argCount matches callee arity.
Jump arityCheckOkay1 = branch32(Equal, Address(regT2, OBJECT_OFFSETOF(CodeBlock, m_numParameters)), regT1);
preserveReturnAddressAfterCall(regT3);
emitPutJITStubArg(regT3, 3); // return address
emitPutJITStubArg(regT2, 7); // codeBlock
restoreArgumentReference();
Call callArityCheck1 = call();
move(regT1, callFrameRegister);
emitGetJITStubArg(1, regT0); // callee
emitGetJITStubArg(5, regT1); // argCount
restoreReturnAddressBeforeReturn(regT3); // return address
arityCheckOkay1.link(this);
isNativeFunc1.link(this);
compileOpCallInitializeCallFrame();
preserveReturnAddressAfterCall(regT3);
emitPutJITStubArg(regT3, 3);
restoreArgumentReference();
Call callDontLazyLinkCall = call();
restoreReturnAddressBeforeReturn(regT3);
jump(regT0);
/* VirtualCallLink Trampoline */
Label virtualCallLinkBegin = align();
......@@ -166,7 +121,7 @@ void JIT::privateCompileCTIMachineTrampolines(RefPtr<ExecutablePool>* executable
// Lazily generate a CodeBlock.
preserveReturnAddressAfterCall(regT3); // return address
restoreArgumentReference();
Call callJSFunction3 = call();
Call callJSFunction1 = call();
move(regT0, regT2);
emitGetJITStubArg(1, regT0); // callee
emitGetJITStubArg(5, regT1); // argCount
......@@ -182,7 +137,7 @@ void JIT::privateCompileCTIMachineTrampolines(RefPtr<ExecutablePool>* executable
emitPutJITStubArg(regT3, 3); // return address
emitPutJITStubArg(regT2, 7); // codeBlock
restoreArgumentReference();
Call callArityCheck3 = call();
Call callArityCheck1 = call();
move(regT1, callFrameRegister);
emitGetJITStubArg(1, regT0); // callee
emitGetJITStubArg(5, regT1); // argCount
......@@ -354,16 +309,13 @@ void JIT::privateCompileCTIMachineTrampolines(RefPtr<ExecutablePool>* executable
patchBuffer.link(string_failureCases2Call, FunctionPtr(cti_op_get_by_id_string_fail));
patchBuffer.link(string_failureCases3Call, FunctionPtr(cti_op_get_by_id_string_fail));
#endif
#if ENABLE(JIT_OPTIMIZE_CALL)
patchBuffer.link(callArityCheck1, FunctionPtr(cti_op_call_arityCheck));
patchBuffer.link(callJSFunction1, FunctionPtr(cti_op_call_JSFunction));
#if ENABLE(JIT_OPTIMIZE_CALL)
patchBuffer.link(callArityCheck2, FunctionPtr(cti_op_call_arityCheck));
patchBuffer.link(callJSFunction2, FunctionPtr(cti_op_call_JSFunction));
patchBuffer.link(callDontLazyLinkCall, FunctionPtr(cti_vm_dontLazyLinkCall));
patchBuffer.link(callLazyLinkCall, FunctionPtr(cti_vm_lazyLinkCall));
#endif
patchBuffer.link(callArityCheck3, FunctionPtr(cti_op_call_arityCheck));
patchBuffer.link(callJSFunction3, FunctionPtr(cti_op_call_JSFunction));
CodeRef finalCode = patchBuffer.finalizeCode();
*executablePool = finalCode.m_executablePool;
......@@ -376,10 +328,8 @@ void JIT::privateCompileCTIMachineTrampolines(RefPtr<ExecutablePool>* executable
UNUSED_PARAM(ctiStringLengthTrampoline);
#endif
#if ENABLE(JIT_OPTIMIZE_CALL)
*ctiVirtualCallPreLink = trampolineAt(finalCode, virtualCallPreLinkBegin);
*ctiVirtualCallLink = trampolineAt(finalCode, virtualCallLinkBegin);
#else
UNUSED_PARAM(ctiVirtualCallPreLink);
UNUSED_PARAM(ctiVirtualCallLink);
#endif
}
......@@ -1509,7 +1459,7 @@ void JIT::emit_op_profile_did_call(Instruction* currentInstruction)
#define RECORD_JUMP_TARGET(targetOffset) \
do { m_labels[m_bytecodeIndex + (targetOffset)].used(); } while (false)
void JIT::privateCompileCTIMachineTrampolines(RefPtr<ExecutablePool>* executablePool, JSGlobalData* globalData, CodePtr* ctiStringLengthTrampoline, CodePtr* ctiVirtualCallPreLink, CodePtr* ctiVirtualCallLink, CodePtr* ctiVirtualCall, CodePtr* ctiNativeCallThunk)
void JIT::privateCompileCTIMachineTrampolines(RefPtr<ExecutablePool>* executablePool, JSGlobalData* globalData, CodePtr* ctiStringLengthTrampoline, CodePtr* ctiVirtualCallLink, CodePtr* ctiVirtualCall, CodePtr* ctiNativeCallThunk)
{
#if ENABLE(JIT_OPTIMIZE_PROPERTY_ACCESS)
// (2) The second function provides fast property access for string length
......@@ -1534,47 +1484,6 @@ void JIT::privateCompileCTIMachineTrampolines(RefPtr<ExecutablePool>* executable
// (3) Trampolines for the slow cases of op_call / op_call_eval / op_construct.
COMPILE_ASSERT(sizeof(CodeType) == 4, CodeTypeEnumMustBe32Bit);
Label virtualCallPreLinkBegin = align();
// Load the callee CodeBlock* into eax
loadPtr(Address(regT2, OBJECT_OFFSETOF(JSFunction, m_body)), regT3);
loadPtr(Address(regT3, OBJECT_OFFSETOF(FunctionBodyNode, m_code)), regT0);
Jump hasCodeBlock1 = branchTestPtr(NonZero, regT0);
preserveReturnAddressAfterCall(regT3);
restoreArgumentReference();
Call callJSFunction1 = call();
emitGetJITStubArg(1, regT2);
emitGetJITStubArg(3, regT1);
restoreReturnAddressBeforeReturn(regT3);
hasCodeBlock1.link(this);
Jump isNativeFunc1 = branch32(Equal, Address(regT0, OBJECT_OFFSETOF(CodeBlock, m_codeType)), Imm32(NativeCode));
// Check argCount matches callee arity.
Jump arityCheckOkay1 = branch32(Equal, Address(regT0, OBJECT_OFFSETOF(CodeBlock, m_numParameters)), regT1);
preserveReturnAddressAfterCall(regT3);
emitPutJITStubArg(regT3, 2);
emitPutJITStubArg(regT0, 4);
restoreArgumentReference();
Call callArityCheck1 = call();
move(regT1, callFrameRegister);
emitGetJITStubArg(1, regT2);
emitGetJITStubArg(3, regT1);
restoreReturnAddressBeforeReturn(regT3);
arityCheckOkay1.link(this);
isNativeFunc1.link(this);
compileOpCallInitializeCallFrame();
preserveReturnAddressAfterCall(regT3);
emitPutJITStubArg(regT3, 2);
restoreArgumentReference();
Call callDontLazyLinkCall = call();
emitGetJITStubArg(1, regT2);
restoreReturnAddressBeforeReturn(regT3);
jump(regT0);
Label virtualCallLinkBegin = align();
// Load the callee CodeBlock* into eax
......@@ -1623,7 +1532,7 @@ void JIT::privateCompileCTIMachineTrampolines(RefPtr<ExecutablePool>* executable
Jump hasCodeBlock3 = branchTestPtr(NonZero, regT0);
preserveReturnAddressAfterCall(regT3);
restoreArgumentReference();
Call callJSFunction3 = call();
Call callJSFunction1 = call();
emitGetJITStubArg(1, regT2);
emitGetJITStubArg(3, regT1);
restoreReturnAddressBeforeReturn(regT3);
......@@ -1638,7 +1547,7 @@ void JIT::privateCompileCTIMachineTrampolines(RefPtr<ExecutablePool>* executable
emitPutJITStubArg(regT3, 2);
emitPutJITStubArg(regT0, 4);
restoreArgumentReference();
Call callArityCheck3 = call();
Call callArityCheck1 = call();
move(regT1, callFrameRegister);
emitGetJITStubArg(1, regT2);
emitGetJITStubArg(3, regT1);
......@@ -1881,21 +1790,17 @@ void JIT::privateCompileCTIMachineTrampolines(RefPtr<ExecutablePool>* executable
patchBuffer.link(string_failureCases2Call, FunctionPtr(cti_op_get_by_id_string_fail));
patchBuffer.link(string_failureCases3Call, FunctionPtr(cti_op_get_by_id_string_fail));
#endif
#if ENABLE(JIT_OPTIMIZE_CALL)
patchBuffer.link(callArityCheck1, FunctionPtr(cti_op_call_arityCheck));
patchBuffer.link(callArityCheck2, FunctionPtr(cti_op_call_arityCheck));
patchBuffer.link(callJSFunction1, FunctionPtr(cti_op_call_JSFunction));
#if ENABLE(JIT_OPTIMIZE_CALL)
patchBuffer.link(callArityCheck2, FunctionPtr(cti_op_call_arityCheck));
patchBuffer.link(callJSFunction2, FunctionPtr(cti_op_call_JSFunction));
patchBuffer.link(callDontLazyLinkCall, FunctionPtr(cti_vm_dontLazyLinkCall));
patchBuffer.link(callLazyLinkCall, FunctionPtr(cti_vm_lazyLinkCall));
#endif
patchBuffer.link(callArityCheck3, FunctionPtr(cti_op_call_arityCheck));
patchBuffer.link(callJSFunction3, FunctionPtr(cti_op_call_JSFunction));
CodeRef finalCode = patchBuffer.finalizeCode();
*executablePool = finalCode.m_executablePool;
*ctiVirtualCallPreLink = trampolineAt(finalCode, virtualCallPreLinkBegin);
*ctiVirtualCallLink = trampolineAt(finalCode, virtualCallLinkBegin);
*ctiVirtualCall = trampolineAt(finalCode, virtualCallBegin);
*ctiNativeCallThunk = trampolineAt(finalCode, nativeCallThunk);
......
......@@ -634,7 +634,7 @@ void JIT::patchGetByIdSelf(CodeBlock* codeBlock, StructureStubInfo* stubInfo, St
repatchBuffer.repatch(stubInfo->hotPathBegin.dataLabel32AtOffset(patchOffsetGetByIdPropertyMapOffset2), offset + 4); // tag
}