Commit 5e135773 authored by fpizlo@apple.com's avatar fpizlo@apple.com

DFG should have fast virtual calls

https://bugs.webkit.org/show_bug.cgi?id=90924

Source/JavaScriptCore: 

Reviewed by Gavin Barraclough.
        
Implements virtual call support in the style of the old JIT, with the
caveat that we still use the same slow path for both InternalFunction
calls and JSFunction calls. Also rationalized the way that our
CodeOrigin indices tie into exception checks (previously it was a
strange one-to-one mapping with fairly limited assertions; now it's a
one-to-many mapping for CodeOrigins to exception checks, respectively).
I also took the opportunity to clean up
CallLinkInfo::callReturnLocation, which previously was either a Call or
a NearCall. Now it's just a NearCall. As well, exceptions during slow
path call resolution are now handled by returning an exception throwing
thunk rather than returning null. And finally, I made a few things
public that were previously private-with-lots-of-friends, because I
truly despise the thought of listing each thunk generating function as
a friend of JSValue and friends.
        
* bytecode/CallLinkInfo.cpp:
(JSC::CallLinkInfo::unlink):
* bytecode/CallLinkInfo.h:
(CallLinkInfo):
* bytecode/CodeOrigin.h:
(JSC::CodeOrigin::CodeOrigin):
(JSC::CodeOrigin::isSet):
* dfg/DFGAssemblyHelpers.h:
(JSC::DFG::AssemblyHelpers::AssemblyHelpers):
* dfg/DFGCCallHelpers.h:
(JSC::DFG::CCallHelpers::CCallHelpers):
* dfg/DFGGPRInfo.h:
(GPRInfo):
* dfg/DFGJITCompiler.cpp:
(JSC::DFG::JITCompiler::link):
(JSC::DFG::JITCompiler::compileFunction):
* dfg/DFGJITCompiler.h:
(JSC::DFG::CallBeginToken::CallBeginToken):
(JSC::DFG::CallBeginToken::~CallBeginToken):
(CallBeginToken):
(JSC::DFG::CallBeginToken::set):
(JSC::DFG::CallBeginToken::registerWithExceptionCheck):
(JSC::DFG::CallBeginToken::codeOrigin):
(JSC::DFG::CallExceptionRecord::CallExceptionRecord):
(CallExceptionRecord):
(JSC::DFG::JITCompiler::currentCodeOriginIndex):
(JITCompiler):
(JSC::DFG::JITCompiler::beginCall):
(JSC::DFG::JITCompiler::notifyCall):
(JSC::DFG::JITCompiler::prepareForExceptionCheck):
(JSC::DFG::JITCompiler::addExceptionCheck):
(JSC::DFG::JITCompiler::addFastExceptionCheck):
* dfg/DFGOperations.cpp:
* dfg/DFGRepatch.cpp:
(JSC::DFG::dfgLinkFor):
* dfg/DFGSpeculativeJIT.h:
(JSC::DFG::SpeculativeJIT::appendCallWithExceptionCheck):
* dfg/DFGSpeculativeJIT32_64.cpp:
(JSC::DFG::SpeculativeJIT::emitCall):
* dfg/DFGSpeculativeJIT64.cpp:
(JSC::DFG::SpeculativeJIT::emitCall):
* dfg/DFGThunks.cpp:
(JSC::DFG::emitPointerValidation):
(DFG):
(JSC::DFG::throwExceptionFromCallSlowPathGenerator):
(JSC::DFG::slowPathFor):
(JSC::DFG::linkForThunkGenerator):
(JSC::DFG::linkCallThunkGenerator):
(JSC::DFG::linkConstructThunkGenerator):
(JSC::DFG::virtualForThunkGenerator):
(JSC::DFG::virtualCallThunkGenerator):
(JSC::DFG::virtualConstructThunkGenerator):
* dfg/DFGThunks.h:
(DFG):
* jit/JIT.cpp:
(JSC::JIT::privateCompile):
(JSC::JIT::linkFor):
* runtime/Executable.h:
(ExecutableBase):
(JSC::ExecutableBase::offsetOfJITCodeFor):
(JSC::ExecutableBase::offsetOfNumParametersFor):
* runtime/JSValue.h:
(JSValue):

LayoutTests: 

Rubber stamped by Oliver Hunt.

This changes which piece of code appears on top of the stack at the point of a stack
overflow. As far as Oliver and I can tell, it doesn't matter, so I just rebased the
test.

* fast/js/stack-trace-expected.txt:



git-svn-id: http://svn.webkit.org/repository/webkit/trunk@122392 268f45cc-cd09-0410-ab3c-d52691b4dbfc
parent ca8621e4
2012-07-11 Filip Pizlo <fpizlo@apple.com>
DFG should have fast virtual calls
https://bugs.webkit.org/show_bug.cgi?id=90924
Rubber stamped by Oliver Hunt.
This changes which piece of code appears on top of the stack at the point of a stack
overflow. As far as Oliver and I can tell, it doesn't matter, so I just rebased the
test.
* fast/js/stack-trace-expected.txt:
2012-07-11 Adam Barth <abarth@webkit.org>
Unreviewed. Remove empty directories.
......@@ -151,7 +151,7 @@ On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE
99 selfRecursive1 at stack-trace.js:52
--> Stack Trace:
0 selfRecursive2 at stack-trace.js:58
0 selfRecursive2 at stack-trace.js:62
1 selfRecursive2 at stack-trace.js:62
2 selfRecursive2 at stack-trace.js:62
3 selfRecursive2 at stack-trace.js:62
......
2012-07-11 Filip Pizlo <fpizlo@apple.com>
DFG should have fast virtual calls
https://bugs.webkit.org/show_bug.cgi?id=90924
Reviewed by Gavin Barraclough.
Implements virtual call support in the style of the old JIT, with the
caveat that we still use the same slow path for both InternalFunction
calls and JSFunction calls. Also rationalized the way that our
CodeOrigin indices tie into exception checks (previously it was a
strange one-to-one mapping with fairly limited assertions; now it's a
one-to-many mapping for CodeOrigins to exception checks, respectively).
I also took the opportunity to clean up
CallLinkInfo::callReturnLocation, which previously was either a Call or
a NearCall. Now it's just a NearCall. As well, exceptions during slow
path call resolution are now handled by returning an exception throwing
thunk rather than returning null. And finally, I made a few things
public that were previously private-with-lots-of-friends, because I
truly despise the thought of listing each thunk generating function as
a friend of JSValue and friends.
* bytecode/CallLinkInfo.cpp:
(JSC::CallLinkInfo::unlink):
* bytecode/CallLinkInfo.h:
(CallLinkInfo):
* bytecode/CodeOrigin.h:
(JSC::CodeOrigin::CodeOrigin):
(JSC::CodeOrigin::isSet):
* dfg/DFGAssemblyHelpers.h:
(JSC::DFG::AssemblyHelpers::AssemblyHelpers):
* dfg/DFGCCallHelpers.h:
(JSC::DFG::CCallHelpers::CCallHelpers):
* dfg/DFGGPRInfo.h:
(GPRInfo):
* dfg/DFGJITCompiler.cpp:
(JSC::DFG::JITCompiler::link):
(JSC::DFG::JITCompiler::compileFunction):
* dfg/DFGJITCompiler.h:
(JSC::DFG::CallBeginToken::CallBeginToken):
(JSC::DFG::CallBeginToken::~CallBeginToken):
(CallBeginToken):
(JSC::DFG::CallBeginToken::set):
(JSC::DFG::CallBeginToken::registerWithExceptionCheck):
(JSC::DFG::CallBeginToken::codeOrigin):
(JSC::DFG::CallExceptionRecord::CallExceptionRecord):
(CallExceptionRecord):
(JSC::DFG::JITCompiler::currentCodeOriginIndex):
(JITCompiler):
(JSC::DFG::JITCompiler::beginCall):
(JSC::DFG::JITCompiler::notifyCall):
(JSC::DFG::JITCompiler::prepareForExceptionCheck):
(JSC::DFG::JITCompiler::addExceptionCheck):
(JSC::DFG::JITCompiler::addFastExceptionCheck):
* dfg/DFGOperations.cpp:
* dfg/DFGRepatch.cpp:
(JSC::DFG::dfgLinkFor):
* dfg/DFGSpeculativeJIT.h:
(JSC::DFG::SpeculativeJIT::appendCallWithExceptionCheck):
* dfg/DFGSpeculativeJIT32_64.cpp:
(JSC::DFG::SpeculativeJIT::emitCall):
* dfg/DFGSpeculativeJIT64.cpp:
(JSC::DFG::SpeculativeJIT::emitCall):
* dfg/DFGThunks.cpp:
(JSC::DFG::emitPointerValidation):
(DFG):
(JSC::DFG::throwExceptionFromCallSlowPathGenerator):
(JSC::DFG::slowPathFor):
(JSC::DFG::linkForThunkGenerator):
(JSC::DFG::linkCallThunkGenerator):
(JSC::DFG::linkConstructThunkGenerator):
(JSC::DFG::virtualForThunkGenerator):
(JSC::DFG::virtualCallThunkGenerator):
(JSC::DFG::virtualConstructThunkGenerator):
* dfg/DFGThunks.h:
(DFG):
* jit/JIT.cpp:
(JSC::JIT::privateCompile):
(JSC::JIT::linkFor):
* runtime/Executable.h:
(ExecutableBase):
(JSC::ExecutableBase::offsetOfJITCodeFor):
(JSC::ExecutableBase::offsetOfNumParametersFor):
* runtime/JSValue.h:
(JSValue):
2012-07-11 Filip Pizlo <fpizlo@apple.com>
Accidentally used the wrong license (3-clause instead of 2-clause) in some
......
......@@ -27,6 +27,7 @@
#include "CallLinkInfo.h"
#include "DFGOperations.h"
#include "DFGThunks.h"
#include "RepatchBuffer.h"
#if ENABLE(JIT)
......@@ -38,12 +39,12 @@ void CallLinkInfo::unlink(JSGlobalData& globalData, RepatchBuffer& repatchBuffer
if (isDFG) {
#if ENABLE(DFG_JIT)
repatchBuffer.relink(CodeLocationCall(callReturnLocation), callType == Construct ? DFG::operationLinkConstruct : DFG::operationLinkCall);
repatchBuffer.relink(callReturnLocation, (callType == Construct ? globalData.getCTIStub(DFG::linkConstructThunkGenerator) : globalData.getCTIStub(DFG::linkCallThunkGenerator)).code());
#else
ASSERT_NOT_REACHED();
#endif
} else
repatchBuffer.relink(CodeLocationNearCall(callReturnLocation), callType == Construct ? globalData.jitStubs->ctiVirtualConstructLink() : globalData.jitStubs->ctiVirtualCallLink());
repatchBuffer.relink(callReturnLocation, callType == Construct ? globalData.jitStubs->ctiVirtualConstructLink() : globalData.jitStubs->ctiVirtualCallLink());
hasSeenShouldRepatch = false;
callee.clear();
......
......@@ -65,7 +65,7 @@ struct CallLinkInfo : public BasicRawSentinelNode<CallLinkInfo> {
remove();
}
CodeLocationLabel callReturnLocation; // it's a near call in the old JIT, or a normal call in DFG
CodeLocationNearCall callReturnLocation;
CodeLocationDataLabelPtr hotPathBegin;
CodeLocationNearCall hotPathOther;
JITWriteBarrier<JSFunction> callee;
......
......@@ -39,6 +39,8 @@ class ExecutableBase;
class JSFunction;
struct CodeOrigin {
static const unsigned maximumBytecodeIndex = (1u << 29) - 1;
// Bytecode offset that you'd use to re-execute this instruction.
unsigned bytecodeIndex : 29;
// Bytecode offset corresponding to the opcode that gives the result (needed to handle
......@@ -48,7 +50,7 @@ struct CodeOrigin {
InlineCallFrame* inlineCallFrame;
CodeOrigin()
: bytecodeIndex(std::numeric_limits<uint32_t>::max())
: bytecodeIndex(maximumBytecodeIndex)
, valueProfileOffset(0)
, inlineCallFrame(0)
{
......@@ -59,11 +61,11 @@ struct CodeOrigin {
, valueProfileOffset(valueProfileOffset)
, inlineCallFrame(inlineCallFrame)
{
ASSERT(bytecodeIndex < (1u << 29));
ASSERT(bytecodeIndex <= maximumBytecodeIndex);
ASSERT(valueProfileOffset < (1u << 3));
}
bool isSet() const { return bytecodeIndex != std::numeric_limits<uint32_t>::max(); }
bool isSet() const { return bytecodeIndex != maximumBytecodeIndex; }
unsigned bytecodeIndexForValueProfile() const
{
......
......@@ -46,12 +46,13 @@ public:
AssemblyHelpers(JSGlobalData* globalData, CodeBlock* codeBlock)
: m_globalData(globalData)
, m_codeBlock(codeBlock)
, m_baselineCodeBlock(codeBlock->baselineVersion())
, m_baselineCodeBlock(codeBlock ? codeBlock->baselineVersion() : 0)
{
ASSERT(m_codeBlock);
ASSERT(m_baselineCodeBlock);
ASSERT(!m_baselineCodeBlock->alternative());
ASSERT(m_baselineCodeBlock->getJITType() == JITCode::BaselineJIT);
if (m_codeBlock) {
ASSERT(m_baselineCodeBlock);
ASSERT(!m_baselineCodeBlock->alternative());
ASSERT(m_baselineCodeBlock->getJITType() == JITCode::BaselineJIT);
}
}
CodeBlock* codeBlock() { return m_codeBlock; }
......
......@@ -37,7 +37,7 @@ namespace JSC { namespace DFG {
class CCallHelpers : public AssemblyHelpers {
public:
CCallHelpers(JSGlobalData* globalData, CodeBlock* codeBlock)
CCallHelpers(JSGlobalData* globalData, CodeBlock* codeBlock = 0)
: AssemblyHelpers(globalData, codeBlock)
{
}
......
......@@ -273,6 +273,7 @@ public:
static const GPRReg argumentGPR1 = X86Registers::edx; // regT1
static const GPRReg nonArgGPR0 = X86Registers::eax; // regT0
static const GPRReg nonArgGPR1 = X86Registers::ebx; // regT3
static const GPRReg nonArgGPR2 = X86Registers::esi; // regT4
static const GPRReg returnValueGPR = X86Registers::eax; // regT0
static const GPRReg returnValueGPR2 = X86Registers::edx; // regT1
static const GPRReg nonPreservedNonReturnGPR = X86Registers::ecx;
......@@ -344,6 +345,7 @@ public:
static const GPRReg argumentGPR5 = X86Registers::r9; // regT7
static const GPRReg nonArgGPR0 = X86Registers::eax; // regT0
static const GPRReg nonArgGPR1 = X86Registers::ebx; // regT3
static const GPRReg nonArgGPR2 = X86Registers::r10; // regT8
static const GPRReg returnValueGPR = X86Registers::eax; // regT0
static const GPRReg returnValueGPR2 = X86Registers::edx; // regT1
static const GPRReg nonPreservedNonReturnGPR = X86Registers::esi;
......@@ -416,6 +418,7 @@ public:
static const GPRReg argumentGPR3 = ARMRegisters::r3; // FIXME!
static const GPRReg nonArgGPR0 = ARMRegisters::r4; // regT3
static const GPRReg nonArgGPR1 = ARMRegisters::r8; // regT4
static const GPRReg nonArgGPR2 = ARMRegisters::r9; // regT5
static const GPRReg returnValueGPR = ARMRegisters::r0; // regT0
static const GPRReg returnValueGPR2 = ARMRegisters::r1; // regT1
static const GPRReg nonPreservedNonReturnGPR = ARMRegisters::r2;
......
......@@ -153,7 +153,6 @@ void JITCompiler::link(LinkBuffer& linkBuffer)
unsigned returnAddressOffset = linkBuffer.returnAddressOffset(m_exceptionChecks[i].m_call);
codeOrigins[i].codeOrigin = record.m_codeOrigin;
codeOrigins[i].callReturnOffset = returnAddressOffset;
record.m_token.assertCodeOriginIndex(i);
}
m_codeBlock->setNumberOfStructureStubInfos(m_propertyAccesses.size());
......@@ -189,7 +188,8 @@ void JITCompiler::link(LinkBuffer& linkBuffer)
CallLinkInfo& info = m_codeBlock->callLinkInfo(i);
info.callType = m_jsCalls[i].m_callType;
info.isDFG = true;
info.callReturnLocation = CodeLocationLabel(linkBuffer.locationOf(m_jsCalls[i].m_slowCall));
linkBuffer.link(m_jsCalls[i].m_slowCall, FunctionPtr((m_globalData->getCTIStub(info.callType == CallLinkInfo::Construct ? linkConstructThunkGenerator : linkCallThunkGenerator)).code().executableAddress()));
info.callReturnLocation = linkBuffer.locationOfNearCall(m_jsCalls[i].m_slowCall);
info.hotPathBegin = linkBuffer.locationOf(m_jsCalls[i].m_targetToCheck);
info.hotPathOther = linkBuffer.locationOfNearCall(m_jsCalls[i].m_fastCall);
}
......@@ -280,7 +280,8 @@ bool JITCompiler::compileFunction(JITCode& entry, MacroAssemblerCodePtr& entryWi
move(stackPointerRegister, GPRInfo::argumentGPR0);
poke(GPRInfo::callFrameRegister, OBJECT_OFFSETOF(struct JITStackFrame, callFrame) / sizeof(void*));
CallBeginToken token = beginCall();
CallBeginToken token;
beginCall(CodeOrigin(0), token);
Call callRegisterFileCheck = call();
notifyCall(callRegisterFileCheck, CodeOrigin(0), token);
jump(fromRegisterFileCheck);
......@@ -297,7 +298,7 @@ bool JITCompiler::compileFunction(JITCode& entry, MacroAssemblerCodePtr& entryWi
branch32(AboveOrEqual, GPRInfo::regT1, TrustedImm32(m_codeBlock->numParameters())).linkTo(fromArityCheck, this);
move(stackPointerRegister, GPRInfo::argumentGPR0);
poke(GPRInfo::callFrameRegister, OBJECT_OFFSETOF(struct JITStackFrame, callFrame) / sizeof(void*));
token = beginCall();
beginCall(CodeOrigin(0), token);
Call callArityCheck = call();
notifyCall(callArityCheck, CodeOrigin(0), token);
move(GPRInfo::regT0, GPRInfo::callFrameRegister);
......
......@@ -76,28 +76,58 @@ class CallBeginToken {
public:
CallBeginToken()
#if !ASSERT_DISABLED
: m_codeOriginIndex(UINT_MAX)
: m_registered(false)
, m_exceptionCheckIndex(std::numeric_limits<unsigned>::max())
#endif
{
}
explicit CallBeginToken(unsigned codeOriginIndex)
~CallBeginToken()
{
ASSERT(m_registered || !m_codeOrigin.isSet());
ASSERT(m_codeOrigin.isSet() == (m_exceptionCheckIndex != std::numeric_limits<unsigned>::max()));
}
void set(CodeOrigin codeOrigin, unsigned index)
{
#if !ASSERT_DISABLED
: m_codeOriginIndex(codeOriginIndex)
ASSERT(m_registered || !m_codeOrigin.isSet());
ASSERT(m_codeOrigin.isSet() == (m_exceptionCheckIndex != std::numeric_limits<unsigned>::max()));
m_codeOrigin = codeOrigin;
m_registered = false;
m_exceptionCheckIndex = index;
#else
UNUSED_PARAM(codeOrigin);
UNUSED_PARAM(index);
#endif
{
UNUSED_PARAM(codeOriginIndex);
}
void assertCodeOriginIndex(unsigned codeOriginIndex) const
void registerWithExceptionCheck(CodeOrigin codeOrigin, unsigned index)
{
ASSERT_UNUSED(codeOriginIndex, codeOriginIndex < UINT_MAX);
ASSERT_UNUSED(codeOriginIndex, codeOriginIndex == m_codeOriginIndex);
#if !ASSERT_DISABLED
ASSERT(m_codeOrigin == codeOrigin);
if (m_registered)
return;
ASSERT(m_exceptionCheckIndex == index);
m_registered = true;
#else
UNUSED_PARAM(codeOrigin);
UNUSED_PARAM(index);
#endif
}
#if !ASSERT_DISABLED
const CodeOrigin& codeOrigin() const
{
return m_codeOrigin;
}
#endif
private:
#if !ASSERT_DISABLED
unsigned m_codeOriginIndex;
CodeOrigin m_codeOrigin;
bool m_registered;
unsigned m_exceptionCheckIndex;
#endif
};
......@@ -107,25 +137,22 @@ private:
// Calls that might throw an exception also record the Jump taken on exception
// (unset if not present) and code origin used to recover handler/source info.
struct CallExceptionRecord {
CallExceptionRecord(MacroAssembler::Call call, CodeOrigin codeOrigin, CallBeginToken token)
CallExceptionRecord(MacroAssembler::Call call, CodeOrigin codeOrigin)
: m_call(call)
, m_codeOrigin(codeOrigin)
, m_token(token)
{
}
CallExceptionRecord(MacroAssembler::Call call, MacroAssembler::Jump exceptionCheck, CodeOrigin codeOrigin, CallBeginToken token)
CallExceptionRecord(MacroAssembler::Call call, MacroAssembler::Jump exceptionCheck, CodeOrigin codeOrigin)
: m_call(call)
, m_exceptionCheck(exceptionCheck)
, m_codeOrigin(codeOrigin)
, m_token(token)
{
}
MacroAssembler::Call m_call;
MacroAssembler::Jump m_exceptionCheck;
CodeOrigin m_codeOrigin;
CallBeginToken m_token;
};
struct PropertyAccessRecord {
......@@ -257,19 +284,27 @@ public:
m_disassembler->setEndOfCode(labelIgnoringWatchpoints());
}
unsigned currentCodeOriginIndex() const
{
return m_currentCodeOriginIndex;
}
// Get a token for beginning a call, and set the current code origin index in
// the call frame.
CallBeginToken beginCall()
// the call frame. For each beginCall() there must be at least one exception
// check, and all of the exception checks must have the same CodeOrigin as the
// beginCall().
void beginCall(CodeOrigin codeOrigin, CallBeginToken& token)
{
unsigned codeOriginIndex = m_currentCodeOriginIndex++;
store32(TrustedImm32(codeOriginIndex), tagFor(static_cast<VirtualRegister>(RegisterFile::ArgumentCount)));
return CallBeginToken(codeOriginIndex);
unsigned index = m_exceptionChecks.size();
store32(TrustedImm32(index), tagFor(static_cast<VirtualRegister>(RegisterFile::ArgumentCount)));
token.set(codeOrigin, index);
}
// Notify the JIT of a call that does not require linking.
void notifyCall(Call functionCall, CodeOrigin codeOrigin, CallBeginToken token)
void notifyCall(Call functionCall, CodeOrigin codeOrigin, CallBeginToken& token)
{
m_exceptionChecks.append(CallExceptionRecord(functionCall, codeOrigin, token));
token.registerWithExceptionCheck(codeOrigin, m_exceptionChecks.size());
m_exceptionChecks.append(CallExceptionRecord(functionCall, codeOrigin));
}
// Add a call out from JIT code, without an exception check.
......@@ -279,20 +314,27 @@ public:
m_calls.append(CallLinkRecord(functionCall, function));
return functionCall;
}
void prepareForExceptionCheck()
{
move(TrustedImm32(m_exceptionChecks.size()), GPRInfo::nonPreservedNonReturnGPR);
}
// Add a call out from JIT code, with an exception check.
void addExceptionCheck(Call functionCall, CodeOrigin codeOrigin, CallBeginToken token)
void addExceptionCheck(Call functionCall, CodeOrigin codeOrigin, CallBeginToken& token)
{
move(TrustedImm32(m_exceptionChecks.size()), GPRInfo::nonPreservedNonReturnGPR);
m_exceptionChecks.append(CallExceptionRecord(functionCall, emitExceptionCheck(), codeOrigin, token));
prepareForExceptionCheck();
token.registerWithExceptionCheck(codeOrigin, m_exceptionChecks.size());
m_exceptionChecks.append(CallExceptionRecord(functionCall, emitExceptionCheck(), codeOrigin));
}
// Add a call out from JIT code, with a fast exception check that tests if the return value is zero.
void addFastExceptionCheck(Call functionCall, CodeOrigin codeOrigin, CallBeginToken token)
void addFastExceptionCheck(Call functionCall, CodeOrigin codeOrigin, CallBeginToken& token)
{
move(TrustedImm32(m_exceptionChecks.size()), GPRInfo::nonPreservedNonReturnGPR);
prepareForExceptionCheck();
Jump exceptionCheck = branchTestPtr(Zero, GPRInfo::returnValueGPR);
m_exceptionChecks.append(CallExceptionRecord(functionCall, exceptionCheck, codeOrigin, token));
token.registerWithExceptionCheck(codeOrigin, m_exceptionChecks.size());
m_exceptionChecks.append(CallExceptionRecord(functionCall, exceptionCheck, codeOrigin));
}
// Helper methods to get predictions
......
......@@ -30,6 +30,7 @@
#include "CodeBlock.h"
#include "DFGOSRExit.h"
#include "DFGRepatch.h"
#include "DFGThunks.h"
#include "HostCallReturnValue.h"
#include "GetterSetter.h"
#include "Interpreter.h"
......@@ -849,7 +850,6 @@ static void* handleHostCall(ExecState* execCallee, JSValue callee, CodeSpecializ
execCallee->setScopeChain(exec->scopeChain());
execCallee->setCodeBlock(0);
execCallee->clearReturnPC();
if (kind == CodeForCall) {
CallData callData;
......@@ -862,14 +862,14 @@ static void* handleHostCall(ExecState* execCallee, JSValue callee, CodeSpecializ
execCallee->setCallee(asObject(callee));
globalData->hostCallReturnValue = JSValue::decode(callData.native.function(execCallee));
if (globalData->exception)
return 0;
return globalData->getCTIStub(throwExceptionFromCallSlowPathGenerator).code().executableAddress();
return reinterpret_cast<void*>(getHostCallReturnValue);
}
ASSERT(callType == CallTypeNone);
exec->globalData().exception = createNotAFunctionError(exec, callee);
return 0;
return globalData->getCTIStub(throwExceptionFromCallSlowPathGenerator).code().executableAddress();
}
ASSERT(kind == CodeForConstruct);
......@@ -884,17 +884,17 @@ static void* handleHostCall(ExecState* execCallee, JSValue callee, CodeSpecializ
execCallee->setCallee(asObject(callee));
globalData->hostCallReturnValue = JSValue::decode(constructData.native.function(execCallee));
if (globalData->exception)
return 0;
return globalData->getCTIStub(throwExceptionFromCallSlowPathGenerator).code().executableAddress();
return reinterpret_cast<void*>(getHostCallReturnValue);
}
ASSERT(constructType == ConstructTypeNone);
exec->globalData().exception = createNotAConstructorError(exec, callee);
return 0;
return globalData->getCTIStub(throwExceptionFromCallSlowPathGenerator).code().executableAddress();
}
inline void* linkFor(ExecState* execCallee, ReturnAddressPtr returnAddress, CodeSpecializationKind kind)
inline void* linkFor(ExecState* execCallee, CodeSpecializationKind kind)
{
ExecState* exec = execCallee->callerFrame();
JSGlobalData* globalData = &exec->globalData();
......@@ -918,7 +918,7 @@ inline void* linkFor(ExecState* execCallee, ReturnAddressPtr returnAddress, Code
JSObject* error = functionExecutable->compileFor(execCallee, callee->scope(), kind);
if (error) {
globalData->exception = createStackOverflowError(exec);
return 0;
return globalData->getCTIStub(throwExceptionFromCallSlowPathGenerator).code().executableAddress();
}
codeBlock = &functionExecutable->generatedBytecodeFor(kind);
if (execCallee->argumentCountIncludingThis() < static_cast<size_t>(codeBlock->numParameters()))
......@@ -926,7 +926,7 @@ inline void* linkFor(ExecState* execCallee, ReturnAddressPtr returnAddress, Code
else
codePtr = functionExecutable->generatedJITCodeFor(kind).addressForCall();
}
CallLinkInfo& callLinkInfo = exec->codeBlock()->getCallLinkInfo(returnAddress);
CallLinkInfo& callLinkInfo = exec->codeBlock()->getCallLinkInfo(execCallee->returnPC());
if (!callLinkInfo.seenOnce())
callLinkInfo.setSeen();
else
......@@ -934,16 +934,14 @@ inline void* linkFor(ExecState* execCallee, ReturnAddressPtr returnAddress, Code
return codePtr.executableAddress();
}
P_FUNCTION_WRAPPER_WITH_RETURN_ADDRESS_E(operationLinkCall);
void* DFG_OPERATION operationLinkCallWithReturnAddress(ExecState* execCallee, ReturnAddressPtr returnAddress)
void* DFG_OPERATION operationLinkCall(ExecState* execCallee)
{
return linkFor(execCallee, returnAddress, CodeForCall);
return linkFor(execCallee, CodeForCall);
}
P_FUNCTION_WRAPPER_WITH_RETURN_ADDRESS_E(operationLinkConstruct);
void* DFG_OPERATION operationLinkConstructWithReturnAddress(ExecState* execCallee, ReturnAddressPtr returnAddress)
void* DFG_OPERATION operationLinkConstruct(ExecState* execCallee)
{
return linkFor(execCallee, returnAddress, CodeForConstruct);
return linkFor(execCallee, CodeForConstruct);
}
inline void* virtualFor(ExecState* execCallee, CodeSpecializationKind kind)
......@@ -965,7 +963,7 @@ inline void* virtualFor(ExecState* execCallee, CodeSpecializationKind kind)
JSObject* error = functionExecutable->compileFor(execCallee, function->scope(), kind);
if (error) {
exec->globalData().exception = error;
return 0;
return globalData->getCTIStub(throwExceptionFromCallSlowPathGenerator).code().executableAddress();
}
}
return executable->generatedJITCodeWithArityCheckFor(kind).executableAddress();
......
......@@ -30,6 +30,7 @@
#include "DFGCCallHelpers.h"
#include "DFGSpeculativeJIT.h"
#include "DFGThunks.h"
#include "GCAwareJITStubRoutine.h"
#include "LinkBuffer.h"
#include "Operations.h"
......@@ -916,6 +917,7 @@ void dfgBuildPutByIdList(ExecState* exec, JSValue baseValue, const Identifier& p
void dfgLinkFor(ExecState* exec, CallLinkInfo& callLinkInfo, CodeBlock* calleeCodeBlock, JSFunction* callee, MacroAssemblerCodePtr codePtr, CodeSpecializationKind kind)
{
CodeBlock* callerCodeBlock = exec->callerFrame()->codeBlock();
JSGlobalData* globalData = callerCodeBlock->globalData();
RepatchBuffer repatchBuffer(callerCodeBlock);
......@@ -928,11 +930,11 @@ void dfgLinkFor(ExecState* exec, CallLinkInfo& callLinkInfo, CodeBlock* calleeCo
calleeCodeBlock->linkIncomingCall(&callLinkInfo);
if (kind == CodeForCall) {
repatchBuffer.relink(CodeLocationCall(callLinkInfo.callReturnLocation), operationVirtualCall);
repatchBuffer.relink(callLinkInfo.callReturnLocation, globalData->getCTIStub(virtualCallThunkGenerator).code());
return;
}
ASSERT(kind == CodeForConstruct);
repatchBuffer.relink(CodeLocationCall(callLinkInfo.callReturnLocation), operationVirtualConstruct);
repatchBuffer.relink(callLinkInfo.callReturnLocation, globalData->getCTIStub(virtualConstructThunkGenerator).code());
}
void dfgResetGetByID(RepatchBuffer& repatchBuffer, StructureStubInfo& stubInfo)
......
......@@ -1738,7 +1738,8 @@ public:
{
prepareForExternalCall();
CodeOrigin codeOrigin = at(m_compileIndex).codeOrigin;
CallBeginToken token = m_jit.beginCall();
CallBeginToken token;
m_jit.beginCall(codeOrigin, token);
JITCompiler::Call call = m_jit.appendCall(function);
m_jit.addExceptionCheck(call, codeOrigin, token);
return call;
......
......@@ -1007,16 +1007,18 @@ void SpeculativeJIT::emitCall(Node& node)
JITCompiler::DataLabelPtr targetToCheck;
JITCompiler::JumpList slowPath;
CallBeginToken token;
m_jit.beginCall(node.codeOrigin, token);
m_jit.addPtr(TrustedImm32(m_jit.codeBlock()->m_numCalleeRegisters * sizeof(Register)), GPRInfo::callFrameRegister);
slowPath.append(m_jit.branchPtrWithPatch(MacroAssembler::NotEqual, calleePayloadGPR, targetToCheck));
slowPath.append(m_jit.branch32(MacroAssembler::NotEqual, calleeTagGPR, TrustedImm32(JSValue::CellTag)));
m_jit.loadPtr(MacroAssembler::Address(calleePayloadGPR, OBJECT_OFFSETOF(JSFunction, m_scopeChain)), resultPayloadGPR);
m_jit.storePtr(resultPayloadGPR, callFramePayloadSlot(RegisterFile::ScopeChain));
m_jit.store32(MacroAssembler::TrustedImm32(JSValue::CellTag), callFrameTagSlot(RegisterFile::ScopeChain));
m_jit.addPtr(TrustedImm32(m_jit.codeBlock()->m_numCalleeRegisters * sizeof(Register)), GPRInfo::callFrameRegister);
m_jit.storePtr(resultPayloadGPR, MacroAssembler::Address(GPRInfo::callFrameRegister, static_cast<ptrdiff_t>(sizeof(Register)) * RegisterFile::ScopeChain + OBJECT_OFFSETOF(EncodedValueDescriptor, asBits.payload)));
m_jit.store32(MacroAssembler::TrustedImm32(JSValue::CellTag), MacroAssembler::Address(GPRInfo::callFrameRegister, static_cast<ptrdiff_t>(sizeof(Register)) * RegisterFile::ScopeChain + OBJECT_OFFSETOF(EncodedValueDescriptor, asBits.tag)));
CodeOrigin codeOrigin = at(m_compileIndex).codeOrigin;
CallBeginToken token = m_jit.beginCall();
JITCompiler::Call fastCall = m_jit.nearCall();
m_jit.notifyCall(fastCall, codeOrigin, token);
......@@ -1024,15 +1026,20 @@ void SpeculativeJIT::emitCall(Node& node)
slowPath.link(&m_jit);
m_jit.addPtr(TrustedImm32(m_jit.codeBlock()->m_numCalleeRegisters * sizeof(Register)), GPRInfo::callFrameRegister, GPRInfo::argumentGPR0);
m_jit.poke(GPRInfo::argumentGPR0);
token = m_jit.beginCall();
JITCompiler::Call slowCall = m_jit.appendCall(slowCallFunction);
m_jit.addFastExceptionCheck(slowCall, codeOrigin, token);
m_jit.addPtr(TrustedImm32(m_jit.codeBlock()->m_numCalleeRegisters * sizeof(Register)), GPRInfo::callFrameRegister);
token = m_jit.beginCall();
JITCompiler::Call theCall = m_jit.call(GPRInfo::returnValueGPR);
m_jit.notifyCall(theCall, codeOrigin, token);
if (calleeTagGPR == GPRInfo::nonArgGPR0) {
if (calleePayloadGPR == GPRInfo::nonArgGPR1)