Commit 0af1468f authored by ggaren@apple.com's avatar ggaren@apple.com

v8 benchmark takes 12-13 million function call slow paths due to extra arguments

https://bugs.webkit.org/show_bug.cgi?id=74244

Reviewed by Filip Pizlo.
        
.arguments function of order the Reversed
        
10% speedup on v8-raytrace, 1.7% speedup on v8 overall, neutral on Kraken
and SunSpider.

* bytecode/CodeBlock.h:
(JSC::CodeBlock::valueProfileForArgument): Clarified that the interface
to this function is an argument number.

* bytecompiler/BytecodeGenerator.cpp:
(JSC::BytecodeGenerator::BytecodeGenerator):
(JSC::BytecodeGenerator::emitCall):
(JSC::BytecodeGenerator::emitConstruct):
(JSC::BytecodeGenerator::isArgumentNumber): Switched to using CallFrame
helper functions for computing offsets for arguments, rather than doing
the math by hand.
        
Switched to iterating argument offsets backwards (--) instead of forwards (++).

* bytecompiler/BytecodeGenerator.h:
(JSC::CallArguments::thisRegister):
(JSC::CallArguments::argumentRegister):
(JSC::CallArguments::registerOffset): Updated for arguments being reversed.

* bytecompiler/NodesCodegen.cpp: Allocate arguments in reverse order.

* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::getArgument):
(JSC::DFG::ByteCodeParser::setArgument):
(JSC::DFG::ByteCodeParser::flush):
(JSC::DFG::ByteCodeParser::addCall):
(JSC::DFG::ByteCodeParser::handleCall):
(JSC::DFG::ByteCodeParser::handleInlining):
(JSC::DFG::ByteCodeParser::handleMinMax):
(JSC::DFG::ByteCodeParser::handleIntrinsic):
(JSC::DFG::ByteCodeParser::parseBlock):
(JSC::DFG::ByteCodeParser::processPhiStack): Use abstract argument indices
that just-in-time convert to bytecode operands (i.e., indexes in the register
file) through helper functions. This means only one piece of code needs
to know how arguments are laid out in the register file.

* dfg/DFGGraph.cpp:
(JSC::DFG::Graph::dump): Ditto.

* dfg/DFGGraph.h:
(JSC::DFG::Graph::valueProfileFor): Ditto.

* dfg/DFGJITCompiler.cpp:
(JSC::DFG::JITCompiler::compileFunction): The whole point of this patch:
Treat too many arguments as an arity match.

* dfg/DFGOSRExit.h:
(JSC::DFG::OSRExit::variableForIndex):
(JSC::DFG::OSRExit::operandForIndex): Use helper functions, as above.

* dfg/DFGOperands.h:
(JSC::DFG::operandToArgument):
(JSC::DFG::argumentToOperand): These are now the only two lines of code in
the DFG compiler that know how arguments are laid out in memory.

(JSC::DFG::Operands::operand):
(JSC::DFG::Operands::setOperand): Use helper functions, as above.

* dfg/DFGOperations.cpp: The whole point of this patch:
Treat too many arguments as an arity match.

* dfg/DFGSpeculativeJIT32_64.cpp:
(JSC::DFG::SpeculativeJIT::emitCall): Use helper functions, as above.
        
Also, don't tag the caller frame slot as a cell, because it's not a cell.

* dfg/DFGSpeculativeJIT64.cpp:
(JSC::DFG::SpeculativeJIT::emitCall): Use helper functions, as above.

* dfg/DFGSpeculativeJIT.cpp:
(JSC::DFG::SpeculativeJIT::compile): Use helper functions, as above.

(JSC::DFG::SpeculativeJIT::checkArgumentTypes): Use already-computed
argument virtual register instead of recomputing by hand.

* dfg/DFGSpeculativeJIT.h:
(JSC::DFG::SpeculativeJIT::callFrameSlot):
(JSC::DFG::SpeculativeJIT::argumentSlot):
(JSC::DFG::SpeculativeJIT::callFrameTagSlot):
(JSC::DFG::SpeculativeJIT::callFramePayloadSlot):
(JSC::DFG::SpeculativeJIT::argumentTagSlot):
(JSC::DFG::SpeculativeJIT::argumentPayloadSlot): Added a few helper
functions for dealing with callee arguments specifically. These still
build on top of our other helper functions, and have no direct knowledge
of how arguments are laid out in the register file.

(JSC::DFG::SpeculativeJIT::resetCallArguments):
(JSC::DFG::SpeculativeJIT::addCallArgument): Renamed argumentIndex to
argumentOffset to match CallFrame naming.

(JSC::DFG::SpeculativeJIT::valueSourceReferenceForOperand): Use helper
functions, as above.

* interpreter/CallFrame.h:
(JSC::ExecState::argumentOffset):
(JSC::ExecState::argumentOffsetIncludingThis):
(JSC::ExecState::argument):
(JSC::ExecState::setArgument):
(JSC::ExecState::thisArgumentOffset):
(JSC::ExecState::thisValue):
(JSC::ExecState::setThisValue):
(JSC::ExecState::offsetFor):
(JSC::ExecState::hostThisRegister):
(JSC::ExecState::hostThisValue): Added a bunch of helper functions for
computing where an argument is in the register file. Anything in the
runtime that needs to access arguments should use these helpers.

* interpreter/CallFrameClosure.h:
(JSC::CallFrameClosure::setThis):
(JSC::CallFrameClosure::setArgument):
(JSC::CallFrameClosure::resetCallFrame): This stuff is a lot simpler, now
that too many arguments counts as an arity match and doesn't require
preserving two copies of our arguments.

* interpreter/Interpreter.cpp:
(JSC::Interpreter::slideRegisterWindowForCall): Only need to do something
special if the caller provided too few arguments.
        
Key simplification: We never need to maintain two copies of our arguments
anymore.

(JSC::eval):
(JSC::loadVarargs): Use helper functions.

(JSC::Interpreter::unwindCallFrame): Updated for new interface.

(JSC::Interpreter::execute):
(JSC::Interpreter::executeCall):
(JSC::Interpreter::executeConstruct):
(JSC::Interpreter::prepareForRepeatCall): Seriously, though: use helper
functions.

(JSC::Interpreter::privateExecute): No need to check for stack overflow
when calling host functions because they have zero callee registers.

(JSC::Interpreter::retrieveArguments): Explicitly tear off the arguments
object, since there's no special constructor for this anymore.

* interpreter/Interpreter.h: Reduced the C++ re-entry depth because some
workers tests were hitting stack overflow in some of my testing. We should
make this test more exact in future.

* interpreter/RegisterFile.h: Death to all runtime knowledge of argument
location that does not belong to the CallFrame class!

* jit/JIT.cpp:
(JSC::JIT::privateCompile): I am a broken record and I use helper functions.
        
Also, the whole point of this patch: Treat too many arguments as an arity match.

* jit/JITCall32_64.cpp:
(JSC::JIT::compileLoadVarargs):
* jit/JITCall.cpp:
(JSC::JIT::compileLoadVarargs): Updated the argument copying math to use
helper functions, for backwards-correctness. Removed the condition
pertaining to declared argument count because, now that arguments are
always in just one place, this optimization is valid for all functions.
Standardized the if predicate for each line of the optimization. This might
fix a bug, but I couldn't get the bug to crash in practice.

* jit/JITOpcodes32_64.cpp:
(JSC::JIT::emit_op_create_arguments):
(JSC::JIT::emit_op_get_argument_by_val):
(JSC::JIT::emitSlow_op_get_argument_by_val):
* jit/JITOpcodes.cpp:
(JSC::JIT::emit_op_create_arguments):
(JSC::JIT::emit_op_get_argument_by_val):
(JSC::JIT::emitSlow_op_get_argument_by_val): Removed cti_op_create_arguments_no_params
optimization because it's no longer an optimization, now that arguments
are always contiguous in a known location.
        
Updated argument access opcode math for backwards-correctness.

* jit/JITStubs.cpp:
(JSC::arityCheckFor): Updated just like slideRegisterWindowForCall. This
function is slightly different because it copies the call frame in
addition to the arguments. (In the Interpreter, the call frame is not
set up by this point.)

(JSC::lazyLinkFor): The whole point of this patch: Treat too many
arguments as an arity match.

(JSC::DEFINE_STUB_FUNCTION): Updated for new iterface to tearOff().

* jit/JITStubs.h:
* jit/SpecializedThunkJIT.h:
(JSC::SpecializedThunkJIT::loadDoubleArgument):
(JSC::SpecializedThunkJIT::loadCellArgument):
(JSC::SpecializedThunkJIT::loadInt32Argument): Use helper functions! They
build strong bones and teeth!

* runtime/ArgList.cpp:
(JSC::ArgList::getSlice):
(JSC::MarkedArgumentBuffer::slowAppend):
* runtime/ArgList.h:
(JSC::MarkedArgumentBuffer::MarkedArgumentBuffer):
(JSC::MarkedArgumentBuffer::~MarkedArgumentBuffer):
(JSC::MarkedArgumentBuffer::at):
(JSC::MarkedArgumentBuffer::clear):
(JSC::MarkedArgumentBuffer::append):
(JSC::MarkedArgumentBuffer::removeLast):
(JSC::MarkedArgumentBuffer::last):
(JSC::ArgList::ArgList):
(JSC::ArgList::at): Updated for backwards-correctness. WTF::Vector doesn't
play nice with backwards-ness, so I changed to using manual allocation.
        
Fixed a FIXME about not all values being marked in the case of out-of-line
arguments. I had to rewrite the loop anyway, and I didn't feel like
maintaining fidelity to its old bugs.

* runtime/Arguments.cpp:
(JSC::Arguments::visitChildren):
(JSC::Arguments::copyToArguments):
(JSC::Arguments::fillArgList):
(JSC::Arguments::getOwnPropertySlotByIndex):
(JSC::Arguments::getOwnPropertySlot):
(JSC::Arguments::getOwnPropertyDescriptor):
(JSC::Arguments::putByIndex):
(JSC::Arguments::put):
(JSC::Arguments::tearOff):
* runtime/Arguments.h:
(JSC::Arguments::create):
(JSC::Arguments::Arguments):
(JSC::Arguments::argument):
(JSC::Arguments::finishCreation): Secondary benefit of this patch: deleted
lots of tricky code designed to maintain two different copies of function
arguments. Now that arguments are always contiguous in one place in memory,
this complexity can go away.
        
Reduced down to one create function for the Arguments class, from three.

Moved tearOff() into an out-of-line function because it's huge.
        
Moved logic about whether to tear off eagerly into the Arguments class,
so we didn't have to duplicate it elsewhere.

* runtime/JSActivation.cpp:
(JSC::JSActivation::JSActivation):
(JSC::JSActivation::visitChildren): Renamed m_numParametersMinusThis to
m_numCapturedArgs because if the value really were m_numParametersMinusThis
we would be marking too much. (We shouldn't mark 'this' because it can't
be captured.) Also, use helper functions.

* runtime/JSActivation.h:
(JSC::JSActivation::tearOff): Use helper functions.

* runtime/JSArray.cpp:
(JSC::JSArray::copyToArguments):
* runtime/JSArray.h: Use helper functions, as above.


git-svn-id: http://svn.webkit.org/repository/webkit/trunk@102545 268f45cc-cd09-0410-ab3c-d52691b4dbfc
parent cc83be37
This diff is collapsed.
......@@ -24,7 +24,7 @@
#include "CompilerVersion.xcconfig"
COMPILER_SPECIFIC_WARNING_CFLAGS = $(COMPILER_SPECIFIC_WARNING_CFLAGS_$(TARGET_GCC_VERSION));
COMPILER_SPECIFIC_WARNING_CFLAGS_LLVM_COMPILER = -Wglobal-constructors -Wexit-time-destructors;
COMPILER_SPECIFIC_WARNING_CFLAGS_LLVM_COMPILER = ;
CLANG_WARN_CXX0X_EXTENSIONS = NO;
DEBUG_INFORMATION_FORMAT = dwarf;
......
......@@ -651,12 +651,12 @@ namespace JSC {
{
return WTF::genericBinarySearch<ValueProfile, int, getValueProfileBytecodeOffset>(m_valueProfiles, m_valueProfiles.size(), bytecodeOffset);
}
ValueProfile* valueProfileForArgument(int argumentIndex)
ValueProfile* valueProfileForArgument(int argument)
{
int index = argumentIndex;
if (static_cast<unsigned>(index) >= m_valueProfiles.size())
size_t index = argument;
if (index >= m_valueProfiles.size())
return 0;
ValueProfile* result = valueProfile(argumentIndex);
ValueProfile* result = valueProfile(index);
if (result->m_bytecodeOffset != -1)
return 0;
return result;
......
......@@ -202,7 +202,7 @@ BytecodeGenerator::BytecodeGenerator(ProgramNode* programNode, ScopeChainNode* s
, m_symbolTable(symbolTable)
, m_scopeNode(programNode)
, m_codeBlock(codeBlock)
, m_thisRegister(RegisterFile::ProgramCodeThisRegister)
, m_thisRegister(CallFrame::thisArgumentOffset())
, m_finallyDepth(0)
, m_dynamicScopeDepth(0)
, m_baseScopeDepth(0)
......@@ -396,16 +396,15 @@ BytecodeGenerator::BytecodeGenerator(FunctionBodyNode* functionBody, ScopeChainN
codeBlock->m_numCapturedVars = codeBlock->m_numVars;
FunctionParameters& parameters = *functionBody->parameters();
size_t parameterCount = parameters.size();
int nextParameterIndex = -RegisterFile::CallFrameHeaderSize - parameterCount - 1;
m_parameters.grow(1 + parameterCount); // reserve space for "this"
m_parameters.grow(parameters.size() + 1); // reserve space for "this"
// Add "this" as a parameter
m_thisRegister.setIndex(nextParameterIndex);
int nextParameterIndex = CallFrame::thisArgumentOffset();
m_thisRegister.setIndex(nextParameterIndex--);
++m_codeBlock->m_numParameters;
for (size_t i = 0; i < parameterCount; ++i)
addParameter(parameters[i], ++nextParameterIndex);
for (size_t i = 0; i < parameters.size(); ++i)
addParameter(parameters[i], nextParameterIndex--);
preserveLastVar();
......@@ -435,7 +434,7 @@ BytecodeGenerator::BytecodeGenerator(EvalNode* evalNode, ScopeChainNode* scopeCh
, m_symbolTable(symbolTable)
, m_scopeNode(evalNode)
, m_codeBlock(codeBlock)
, m_thisRegister(RegisterFile::ProgramCodeThisRegister)
, m_thisRegister(CallFrame::thisArgumentOffset())
, m_finallyDepth(0)
, m_dynamicScopeDepth(0)
, m_baseScopeDepth(codeBlock->baseScopeDepth())
......@@ -1801,9 +1800,9 @@ RegisterID* BytecodeGenerator::emitCall(OpcodeID opcodeID, RegisterID* dst, Regi
emitMove(callArguments.profileHookRegister(), func);
// Generate code for arguments.
unsigned argumentIndex = 0;
unsigned argument = 0;
for (ArgumentListNode* n = callArguments.argumentsNode()->m_listNode; n; n = n->m_next)
emitNode(callArguments.argumentRegister(argumentIndex++), n);
emitNode(callArguments.argumentRegister(argument++), n);
// Reserve space for call frame.
Vector<RefPtr<RegisterID>, RegisterFile::CallFrameHeaderSize> callFrame;
......@@ -1820,7 +1819,7 @@ RegisterID* BytecodeGenerator::emitCall(OpcodeID opcodeID, RegisterID* dst, Regi
// Emit call.
emitOpcode(opcodeID);
instructions().append(func->index()); // func
instructions().append(callArguments.count()); // argCount
instructions().append(callArguments.argumentCountIncludingThis()); // argCount
instructions().append(callArguments.registerOffset()); // registerOffset
if (dst != ignoredResult()) {
emitOpcode(op_call_put_result);
......@@ -1900,10 +1899,10 @@ RegisterID* BytecodeGenerator::emitConstruct(RegisterID* dst, RegisterID* func,
emitMove(callArguments.profileHookRegister(), func);
// Generate code for arguments.
unsigned argumentIndex = 0;
unsigned argument = 0;
if (ArgumentsNode* argumentsNode = callArguments.argumentsNode()) {
for (ArgumentListNode* n = argumentsNode->m_listNode; n; n = n->m_next)
emitNode(callArguments.argumentRegister(argumentIndex++), n);
emitNode(callArguments.argumentRegister(argument++), n);
}
if (m_shouldEmitProfileHooks) {
......@@ -1920,7 +1919,7 @@ RegisterID* BytecodeGenerator::emitConstruct(RegisterID* dst, RegisterID* func,
emitOpcode(op_construct);
instructions().append(func->index()); // func
instructions().append(callArguments.count()); // argCount
instructions().append(callArguments.argumentCountIncludingThis()); // argCount
instructions().append(callArguments.registerOffset()); // registerOffset
if (dst != ignoredResult()) {
emitOpcode(op_call_put_result);
......@@ -2362,7 +2361,7 @@ bool BytecodeGenerator::isArgumentNumber(const Identifier& ident, int argumentNu
RegisterID* registerID = registerFor(ident);
if (!registerID || registerID->index() >= 0)
return 0;
return registerID->index() - m_thisRegister.index() - 1 == argumentNumber;
return registerID->index() == CallFrame::argumentOffset(argumentNumber);
}
} // namespace JSC
......@@ -55,8 +55,8 @@ namespace JSC {
RegisterID* thisRegister() { return m_argv[0].get(); }
RegisterID* argumentRegister(unsigned i) { return m_argv[i + 1].get(); }
unsigned registerOffset() { return thisRegister()->index() + count() + RegisterFile::CallFrameHeaderSize; }
unsigned count() { return m_argv.size(); }
unsigned registerOffset() { return m_argv.last()->index() + CallFrame::offsetFor(argumentCountIncludingThis()); }
unsigned argumentCountIncludingThis() { return m_argv.size(); }
RegisterID* profileHookRegister() { return m_profileHookRegister.get(); }
ArgumentsNode* argumentsNode() { return m_argumentsNode; }
......
......@@ -319,11 +319,17 @@ inline CallArguments::CallArguments(BytecodeGenerator& generator, ArgumentsNode*
if (generator.shouldEmitProfileHooks())
m_profileHookRegister = generator.newTemporary();
newArgument(generator); // 'this' register.
if (!argumentsNode)
return;
for (ArgumentListNode* n = argumentsNode->m_listNode; n; n = n->m_next)
newArgument(generator);
size_t argumentCountIncludingThis = 1; // 'this' register.
if (argumentsNode) {
for (ArgumentListNode* node = argumentsNode->m_listNode; node; node = node->m_next)
++argumentCountIncludingThis;
}
m_argv.grow(argumentCountIncludingThis);
for (int i = argumentCountIncludingThis - 1; i >= 0; --i) {
m_argv[i] = generator.newTemporary();
ASSERT(static_cast<size_t>(i) == m_argv.size() - 1 || m_argv[i]->index() == m_argv[i + 1]->index() + 1);
}
}
inline void CallArguments::newArgument(BytecodeGenerator& generator)
......
......@@ -217,7 +217,7 @@ void Graph::dump(NodeIndex nodeIndex, CodeBlock* codeBlock)
VariableAccessData* variableAccessData = node.variableAccessData();
int operand = variableAccessData->operand();
if (operandIsArgument(operand))
printf("%sarg%u(%s)", hasPrinted ? ", " : "", operand - codeBlock->thisRegister(), nameOfVariableAccessData(variableAccessData));
printf("%sarg%u(%s)", hasPrinted ? ", " : "", operandToArgument(operand), nameOfVariableAccessData(variableAccessData));
else
printf("%sr%u(%s)", hasPrinted ? ", " : "", operand, nameOfVariableAccessData(variableAccessData));
hasPrinted = true;
......
......@@ -197,7 +197,7 @@ public:
case GetLocal: {
if (!operandIsArgument(node.local()))
return 0;
int argument = node.local() + m_arguments.size() + RegisterFile::CallFrameHeaderSize;
int argument = operandToArgument(node.local());
if (node.variableAccessData() != at(m_arguments[argument]).variableAccessData())
return 0;
return profiledBlock->valueProfileForArgument(argument);
......
......@@ -256,7 +256,7 @@ void JITCompiler::compileFunction(JITCode& entry, MacroAssemblerCodePtr& entryWi
compileEntry();
load32(Address(GPRInfo::callFrameRegister, RegisterFile::ArgumentCount * static_cast<int>(sizeof(Register))), GPRInfo::regT1);
branch32(Equal, GPRInfo::regT1, Imm32(m_codeBlock->m_numParameters)).linkTo(fromArityCheck, this);
branch32(AboveOrEqual, GPRInfo::regT1, Imm32(m_codeBlock->m_numParameters)).linkTo(fromArityCheck, this);
move(stackPointerRegister, GPRInfo::argumentGPR0);
poke(GPRInfo::callFrameRegister, OBJECT_OFFSETOF(struct JITStackFrame, callFrame) / sizeof(void*));
Call callArityCheck = call();
......
......@@ -34,6 +34,7 @@
#include "DFGCommon.h"
#include "DFGCorrectableJumpPoint.h"
#include "DFGGPRInfo.h"
#include "DFGOperands.h"
#include "MacroAssembler.h"
#include "ValueProfile.h"
#include "ValueRecovery.h"
......@@ -112,14 +113,10 @@ struct OSRExit {
{
return index - m_arguments.size();
}
int operandForArgument(int argument) const
{
return argument - m_arguments.size() - RegisterFile::CallFrameHeaderSize;
}
int operandForIndex(int index) const
{
if (index < (int)m_arguments.size())
return operandForArgument(index);
return operandToArgument(index);
return index - m_arguments.size();
}
......
......@@ -30,12 +30,15 @@
#if ENABLE(DFG_JIT)
#include "CallFrame.h"
#include <wtf/Vector.h>
namespace JSC { namespace DFG {
// helper function to distinguish vars & temporaries from arguments.
// argument 0 is 'this'.
inline bool operandIsArgument(int operand) { return operand < 0; }
inline int operandToArgument(int operand) { return -operand + CallFrame::thisArgumentOffset(); }
inline int argumentToOperand(int argument) { return -argument + CallFrame::thisArgumentOffset(); }
template<typename T> struct OperandValueTraits;
......@@ -105,7 +108,7 @@ public:
T& operand(int operand)
{
if (operandIsArgument(operand)) {
int argument = operand + m_arguments.size() + RegisterFile::CallFrameHeaderSize;
int argument = operandToArgument(operand);
return m_arguments[argument];
}
......@@ -117,7 +120,7 @@ public:
void setOperand(int operand, const T& value)
{
if (operandIsArgument(operand)) {
int argument = operand + m_arguments.size() + RegisterFile::CallFrameHeaderSize;
int argument = operandToArgument(operand);
m_arguments[argument] = value;
return;
}
......
......@@ -658,10 +658,10 @@ inline void* linkFor(ExecState* execCallee, ReturnAddressPtr returnAddress, Code
return 0;
}
codeBlock = &functionExecutable->generatedBytecodeFor(kind);
if (execCallee->argumentCountIncludingThis() == static_cast<size_t>(codeBlock->m_numParameters))
codePtr = functionExecutable->generatedJITCodeFor(kind).addressForCall();
else
if (execCallee->argumentCountIncludingThis() < static_cast<size_t>(codeBlock->m_numParameters))
codePtr = functionExecutable->generatedJITCodeWithArityCheckFor(kind);
else
codePtr = functionExecutable->generatedJITCodeFor(kind).addressForCall();
}
CallLinkInfo& callLinkInfo = exec->codeBlock()->getCallLinkInfo(returnAddress);
if (!callLinkInfo.seenOnce())
......
......@@ -963,9 +963,9 @@ void SpeculativeJIT::compile(BasicBlock& block)
case InlineStart: {
InlineCallFrame* inlineCallFrame = node.codeOrigin.inlineCallFrame;
unsigned argumentsStart = inlineCallFrame->stackOffset - RegisterFile::CallFrameHeaderSize - inlineCallFrame->arguments.size();
for (unsigned i = 0; i < inlineCallFrame->arguments.size(); ++i) {
ValueRecovery recovery = computeValueRecoveryFor(m_variables[argumentsStart + i]);
int argumentCountIncludingThis = inlineCallFrame->arguments.size();
for (int i = 0; i < argumentCountIncludingThis; ++i) {
ValueRecovery recovery = computeValueRecoveryFor(m_variables[inlineCallFrame->stackOffset + CallFrame::argumentOffsetIncludingThis(i)]);
// The recovery cannot point to registers, since the call frame reification isn't
// as smart as OSR, so it can't handle that. The exception is the this argument,
// which we don't really need to be able to recover.
......@@ -1020,8 +1020,8 @@ void SpeculativeJIT::compile(BasicBlock& block)
}
#if DFG_ENABLE(VERBOSE_VALUE_RECOVERIES)
for (int operand = -m_arguments.size() - RegisterFile::CallFrameHeaderSize; operand < -RegisterFile::CallFrameHeaderSize; ++operand)
computeValueRecoveryFor(operand).dump(stderr);
for (size_t i = 0; i < m_arguments.size(); ++i)
computeValueRecoveryFor(argumentToOperand(i)).dump(stderr);
fprintf(stderr, " : ");
......@@ -1054,8 +1054,9 @@ void SpeculativeJIT::checkArgumentTypes()
m_variables[i] = ValueSource(ValueInRegisterFile);
for (int i = 0; i < m_jit.codeBlock()->m_numParameters; ++i) {
VirtualRegister virtualRegister = (VirtualRegister)(m_jit.codeBlock()->thisRegister() + i);
PredictedType predictedType = at(m_jit.graph().m_arguments[i]).variableAccessData()->prediction();
VariableAccessData* variableAccessData = at(m_jit.graph().m_arguments[i]).variableAccessData();
VirtualRegister virtualRegister = variableAccessData->local();
PredictedType predictedType = variableAccessData->prediction();
#if USE(JSVALUE64)
if (isInt32Prediction(predictedType))
speculationCheck(JSValueRegs(), NoNode, m_jit.branchPtr(MacroAssembler::Below, JITCompiler::addressFor(virtualRegister), GPRInfo::tagTypeNumberRegister));
......
......@@ -890,20 +890,37 @@ private:
void compileInstanceOfForObject(Node&, GPRReg valueReg, GPRReg prototypeReg, GPRReg scratchAndResultReg);
void compileInstanceOf(Node&);
MacroAssembler::Address addressOfCallData(int idx)
// Access to our fixed callee CallFrame.
MacroAssembler::Address callFrameSlot(int slot)
{
return MacroAssembler::Address(GPRInfo::callFrameRegister, (m_jit.codeBlock()->m_numCalleeRegisters + idx) * static_cast<int>(sizeof(Register)));
return MacroAssembler::Address(GPRInfo::callFrameRegister, (m_jit.codeBlock()->m_numCalleeRegisters + slot) * static_cast<int>(sizeof(Register)));
}
// Access to our fixed callee CallFrame.
MacroAssembler::Address argumentSlot(int argument)
{
return MacroAssembler::Address(GPRInfo::callFrameRegister, (m_jit.codeBlock()->m_numCalleeRegisters + argumentToOperand(argument)) * static_cast<int>(sizeof(Register)));
}
#if USE(JSVALUE32_64)
MacroAssembler::Address tagOfCallData(int idx)
MacroAssembler::Address callFrameTagSlot(int slot)
{
return MacroAssembler::Address(GPRInfo::callFrameRegister, (m_jit.codeBlock()->m_numCalleeRegisters + slot) * static_cast<int>(sizeof(Register)) + OBJECT_OFFSETOF(EncodedValueDescriptor, asBits.tag));
}
MacroAssembler::Address callFramePayloadSlot(int slot)
{
return MacroAssembler::Address(GPRInfo::callFrameRegister, (m_jit.codeBlock()->m_numCalleeRegisters + slot) * static_cast<int>(sizeof(Register)) + OBJECT_OFFSETOF(EncodedValueDescriptor, asBits.payload));
}
MacroAssembler::Address argumentTagSlot(int argument)
{
return MacroAssembler::Address(GPRInfo::callFrameRegister, (m_jit.codeBlock()->m_numCalleeRegisters + idx) * static_cast<int>(sizeof(Register)) + OBJECT_OFFSETOF(EncodedValueDescriptor, asBits.tag));
return MacroAssembler::Address(GPRInfo::callFrameRegister, (m_jit.codeBlock()->m_numCalleeRegisters + argumentToOperand(argument)) * static_cast<int>(sizeof(Register)) + OBJECT_OFFSETOF(EncodedValueDescriptor, asBits.tag));
}
MacroAssembler::Address payloadOfCallData(int idx)
MacroAssembler::Address argumentPayloadSlot(int argument)
{
return MacroAssembler::Address(GPRInfo::callFrameRegister, (m_jit.codeBlock()->m_numCalleeRegisters + idx) * static_cast<int>(sizeof(Register)) + OBJECT_OFFSETOF(EncodedValueDescriptor, asBits.payload));
return MacroAssembler::Address(GPRInfo::callFrameRegister, (m_jit.codeBlock()->m_numCalleeRegisters + argumentToOperand(argument)) * static_cast<int>(sizeof(Register)) + OBJECT_OFFSETOF(EncodedValueDescriptor, asBits.payload));
}
#endif
......@@ -1045,26 +1062,26 @@ private:
// stack. On other architectures we may need to sort values into the
// correct registers.
#if !NUMBER_OF_ARGUMENT_REGISTERS
unsigned m_callArgumentIndex;
void resetCallArguments() { m_callArgumentIndex = 0; }
unsigned m_callArgumentOffset;
void resetCallArguments() { m_callArgumentOffset = 0; }
// These methods are using internally to implement the callOperation methods.
void addCallArgument(GPRReg value)
{
m_jit.poke(value, m_callArgumentIndex++);
m_jit.poke(value, m_callArgumentOffset++);
}
void addCallArgument(TrustedImm32 imm)
{
m_jit.poke(imm, m_callArgumentIndex++);
m_jit.poke(imm, m_callArgumentOffset++);
}
void addCallArgument(TrustedImmPtr pointer)
{
m_jit.poke(pointer, m_callArgumentIndex++);
m_jit.poke(pointer, m_callArgumentOffset++);
}
void addCallArgument(FPRReg value)
{
m_jit.storeDouble(value, JITCompiler::Address(JITCompiler::stackPointerRegister, m_callArgumentIndex * sizeof(void*)));
m_callArgumentIndex += sizeof(double) / sizeof(void*);
m_jit.storeDouble(value, JITCompiler::Address(JITCompiler::stackPointerRegister, m_callArgumentOffset * sizeof(void*)));
m_callArgumentOffset += sizeof(double) / sizeof(void*);
}
ALWAYS_INLINE void setupArguments(FPRReg arg1)
......@@ -2075,7 +2092,7 @@ private:
ValueSource& valueSourceReferenceForOperand(int operand)
{
if (operandIsArgument(operand)) {
int argument = operand + m_arguments.size() + RegisterFile::CallFrameHeaderSize;
int argument = operandToArgument(operand);
return m_arguments[argument];
}
......
......@@ -1258,43 +1258,27 @@ void SpeculativeJIT::emitCall(Node& node)
GPRReg calleePayloadGPR = callee.payloadGPR();
use(calleeNodeIndex);
// the call instruction's first child is either the function (normal call) or the
// The call instruction's first child is either the function (normal call) or the
// receiver (method call). subsequent children are the arguments.
int numArgs = node.numChildren() - 1;
int numPassedArgs = node.numChildren() - 1;
// For constructors, the this argument is not passed but we have to make space
// for it.
int numPassedArgs = numArgs + dummyThisArgument;
// amount of stuff (in units of sizeof(Register)) that we need to place at the
// top of the JS stack.
int callDataSize = 0;
// first there are the arguments
callDataSize += numPassedArgs;
m_jit.store32(MacroAssembler::TrustedImm32(numPassedArgs + dummyThisArgument), callFramePayloadSlot(RegisterFile::ArgumentCount));
m_jit.store32(MacroAssembler::TrustedImm32(JSValue::Int32Tag), callFrameTagSlot(RegisterFile::ArgumentCount));
m_jit.storePtr(GPRInfo::callFrameRegister, callFramePayloadSlot(RegisterFile::CallerFrame));
m_jit.store32(calleePayloadGPR, callFramePayloadSlot(RegisterFile::Callee));
m_jit.store32(calleeTagGPR, callFrameTagSlot(RegisterFile::Callee));
// and then there is the call frame header
callDataSize += RegisterFile::CallFrameHeaderSize;
m_jit.store32(MacroAssembler::TrustedImm32(numPassedArgs), payloadOfCallData(RegisterFile::ArgumentCount));
m_jit.store32(MacroAssembler::TrustedImm32(JSValue::Int32Tag), tagOfCallData(RegisterFile::ArgumentCount));
m_jit.storePtr(GPRInfo::callFrameRegister, payloadOfCallData(RegisterFile::CallerFrame));
m_jit.store32(MacroAssembler::TrustedImm32(JSValue::CellTag), tagOfCallData(RegisterFile::CallerFrame));
for (int argIdx = 0; argIdx < numArgs; argIdx++) {
NodeIndex argNodeIndex = m_jit.graph().m_varArgChildren[node.firstChild() + 1 + argIdx];
for (int i = 0; i < numPassedArgs; i++) {
NodeIndex argNodeIndex = m_jit.graph().m_varArgChildren[node.firstChild() + 1 + i];
JSValueOperand arg(this, argNodeIndex);
GPRReg argTagGPR = arg.tagGPR();
GPRReg argPayloadGPR = arg.payloadGPR();
use(argNodeIndex);
m_jit.store32(argTagGPR, tagOfCallData(-callDataSize + argIdx + dummyThisArgument));
m_jit.store32(argPayloadGPR, payloadOfCallData(-callDataSize + argIdx + dummyThisArgument));
m_jit.store32(argTagGPR, argumentTagSlot(i + dummyThisArgument));
m_jit.store32(argPayloadGPR, argumentPayloadSlot(i + dummyThisArgument));
}
m_jit.store32(calleeTagGPR, tagOfCallData(RegisterFile::Callee));
m_jit.store32(calleePayloadGPR, payloadOfCallData(RegisterFile::Callee));
flushRegisters();
GPRResult resultPayload(this);
......@@ -1308,8 +1292,8 @@ void SpeculativeJIT::emitCall(Node& node)
slowPath.append(m_jit.branchPtrWithPatch(MacroAssembler::NotEqual, calleePayloadGPR, targetToCheck));
slowPath.append(m_jit.branch32(MacroAssembler::NotEqual, calleeTagGPR, TrustedImm32(JSValue::CellTag)));
m_jit.loadPtr(MacroAssembler::Address(calleePayloadGPR, OBJECT_OFFSETOF(JSFunction, m_scopeChain)), resultPayloadGPR);
m_jit.storePtr(resultPayloadGPR, payloadOfCallData(RegisterFile::ScopeChain));
m_jit.store32(MacroAssembler::TrustedImm32(JSValue::CellTag), tagOfCallData(RegisterFile::ScopeChain));
m_jit.storePtr(resultPayloadGPR, callFramePayloadSlot(RegisterFile::ScopeChain));
m_jit.store32(MacroAssembler::TrustedImm32(JSValue::CellTag), callFrameTagSlot(RegisterFile::ScopeChain));
m_jit.addPtr(Imm32(m_jit.codeBlock()->m_numCalleeRegisters * sizeof(Register)), GPRInfo::callFrameRegister);
......
......@@ -1217,47 +1217,34 @@ void SpeculativeJIT::emitCall(Node& node)
GPRReg calleeGPR = callee.gpr();
use(calleeNodeIndex);
// the call instruction's first child is either the function (normal call) or the
// The call instruction's first child is either the function (normal call) or the
// receiver (method call). subsequent children are the arguments.
int numArgs = node.numChildren() - 1;
int numPassedArgs = node.numChildren() - 1;
int numPassedArgs = numArgs + dummyThisArgument;
m_jit.storePtr(MacroAssembler::TrustedImmPtr(JSValue::encode(jsNumber(numPassedArgs + dummyThisArgument))), callFrameSlot(RegisterFile::ArgumentCount));
m_jit.storePtr(GPRInfo::callFrameRegister, callFrameSlot(RegisterFile::CallerFrame));
m_jit.storePtr(calleeGPR, callFrameSlot(RegisterFile::Callee));
// amount of stuff (in units of sizeof(Register)) that we need to place at the
// top of the JS stack.
int callDataSize = 0;
// first there are the arguments
callDataSize += numPassedArgs;
// and then there is the call frame header
callDataSize += RegisterFile::CallFrameHeaderSize;
m_jit.storePtr(MacroAssembler::TrustedImmPtr(JSValue::encode(jsNumber(numPassedArgs))), addressOfCallData(RegisterFile::ArgumentCount));
m_jit.storePtr(GPRInfo::callFrameRegister, addressOfCallData(RegisterFile::CallerFrame));
for (int argIdx = 0; argIdx < numArgs; argIdx++) {
NodeIndex argNodeIndex = m_jit.graph().m_varArgChildren[node.firstChild() + 1 + argIdx];
for (int i = 0; i < numPassedArgs; i++) {
NodeIndex argNodeIndex = m_jit.graph().m_varArgChildren[node.firstChild() + 1 + i];
JSValueOperand arg(this, argNodeIndex);
GPRReg argGPR = arg.gpr();
use(argNodeIndex);
m_jit.storePtr(argGPR, addressOfCallData(-callDataSize + argIdx + dummyThisArgument));
m_jit.storePtr(argGPR, argumentSlot(i + dummyThisArgument));
}
m_jit.storePtr(calleeGPR, addressOfCallData(RegisterFile::Callee));
flushRegisters();
GPRResult result(this);
GPRReg resultGPR = result.gpr();
JITCompiler::DataLabelPtr targetToCheck;
JITCompiler::Jump slowPath;
slowPath = m_jit.branchPtrWithPatch(MacroAssembler::NotEqual, calleeGPR, targetToCheck, MacroAssembler::TrustedImmPtr(JSValue::encode(JSValue())));
m_jit.loadPtr(MacroAssembler::Address(calleeGPR, OBJECT_OFFSETOF(JSFunction, m_scopeChain)), resultGPR);
m_jit.storePtr(resultGPR, addressOfCallData(RegisterFile::ScopeChain));
m_jit.storePtr(resultGPR, callFrameSlot(RegisterFile::ScopeChain));
m_jit.addPtr(Imm32(m_jit.codeBlock()->m_numCalleeRegisters * sizeof(Register)), GPRInfo::callFrameRegister);
......
......@@ -143,18 +143,32 @@ namespace JSC {
inline Register& uncheckedR(int);
// Access to arguments.
int hostThisRegister() { return -RegisterFile::CallFrameHeaderSize - argumentCountIncludingThis(); }
JSValue hostThisValue() { return this[hostThisRegister()].jsValue(); }
size_t argumentCount() const { return argumentCountIncludingThis() - 1; }
size_t argumentCountIncludingThis() const { return this[RegisterFile::ArgumentCount].i(); }
JSValue argument(int argumentNumber)
static int argumentOffset(size_t argument) { return s_firstArgumentOffset - argument; }
static int argumentOffsetIncludingThis(size_t argument) { return s_thisArgumentOffset - argument; }
JSValue argument(size_t argument)
{
if (argument >= argumentCount())
return jsUndefined();
return this[argumentOffset(argument)].jsValue();
}
void setArgument(size_t argument, JSValue value)
{
int argumentIndex = -RegisterFile::CallFrameHeaderSize - this[RegisterFile::ArgumentCount].i() + argumentNumber + 1;
if (argumentIndex >= -RegisterFile::CallFrameHeaderSize)
return jsUndefined();
return this[argumentIndex].jsValue();
this[argumentOffset(argument)] = value;
}
static int thisArgumentOffset() { return argumentOffsetIncludingThis(0); }
JSValue thisValue() { return this[thisArgumentOffset()].jsValue(); }
void setThisValue(JSValue value) { this[thisArgumentOffset()] = value; }
static int offsetFor(size_t argumentCountIncludingThis) { return argumentCountIncludingThis + RegisterFile::CallFrameHeaderSize; }
// FIXME: Remove these.
int hostThisRegister() { return thisArgumentOffset(); }
JSValue hostThisValue() { return thisValue(); }
static CallFrame* noCaller() { return reinterpret_cast<CallFrame*>(HostCallFrameFlag); }
bool hasHostCallFrameFlag() const { return reinterpret_cast<intptr_t>(this) & HostCallFrameFlag; }
......@@ -182,6 +196,9 @@ namespace JSC {
private:
static const intptr_t HostCallFrameFlag = 1;
static const int s_thisArgumentOffset = -1 - RegisterFile::CallFrameHeaderSize;
static const int s_firstArgumentOffset = s_thisArgumentOffset - 1;
#ifndef NDEBUG
RegisterFile* registerFile();
#endif
......
......@@ -41,25 +41,19 @@ struct CallFrameClosure {
void setThis(JSValue value)
{
newCallFrame[-RegisterFile::CallFrameHeaderSize - parameterCountIncludingThis] = value;
if (argumentCountIncludingThis > parameterCountIncludingThis)
newCallFrame[-RegisterFile::CallFrameHeaderSize - parameterCountIncludingThis - argumentCountIncludingThis] = value;
newCallFrame->setThisValue(value);
}
void setArgument(int argument, JSValue value)
{
if (argument + 1 < parameterCountIncludingThis)
newCallFrame[argument + 1 - RegisterFile::CallFrameHeaderSize - parameterCountIncludingThis] = value;