Commit 87ff87df authored by mhahnenberg@apple.com's avatar mhahnenberg@apple.com

Implement fast path for op_new_array in the baseline JIT

https://bugs.webkit.org/show_bug.cgi?id=78612

Reviewed by Filip Pizlo.

heap/CopiedAllocator.h:
(CopiedAllocator): Friended the JIT to allow access to m_currentOffset.
* heap/CopiedSpace.h:
(CopiedSpace): Friended the JIT to allow access to isOversize.
(JSC::CopiedSpace::allocator):
* heap/Heap.h:
(JSC::Heap::storageAllocator): Added a getter for the CopiedAllocator class so the JIT
can use it for simple allocation i.e. when we can just bump the offset without having to 
do anything else.
* jit/JIT.cpp:
(JSC::JIT::privateCompileSlowCases): Added new slow case for op_new_array for when
we have to bail out because the fast allocation path fails for whatever reason.
* jit/JIT.h:
(JIT):
* jit/JITInlineMethods.h:
(JSC::JIT::emitAllocateBasicStorage): Added utility function that allows objects to 
allocate generic backing stores. This function is used by emitAllocateJSArray.
(JSC):
(JSC::JIT::emitAllocateJSArray): Added utility function that allows the client to 
more easily allocate JSArrays. This function is used by emit_op_new_array and I expect 
it will also be used for emit_op_new_array_buffer.
* jit/JITOpcodes.cpp:
(JSC::JIT::emit_op_new_array): Changed to do inline allocation of JSArrays. Still does 
a stub call for oversize arrays.
(JSC):
(JSC::JIT::emitSlow_op_new_array): New slow path that just bails out to a stub call if we 
fail in any way on the fast path.
* runtime/JSArray.cpp:
(JSC):
* runtime/JSArray.h: Added lots of offset functions for all the fields that we need to 
initialize in the JIT.
(ArrayStorage):
(JSC::ArrayStorage::lengthOffset):
(JSC::ArrayStorage::numValuesInVectorOffset):
(JSC::ArrayStorage::allocBaseOffset):
(JSC::ArrayStorage::vectorOffset):
(JSArray):
(JSC::JSArray::sparseValueMapOffset):
(JSC::JSArray::subclassDataOffset):
(JSC::JSArray::indexBiasOffset):
(JSC):
(JSC::JSArray::storageSize): Moved this function from being a static function in the cpp file
to being a static function in the JSArray class. This move allows the JIT to call it to 
see what size it should allocate.


git-svn-id: http://svn.webkit.org/repository/webkit/trunk@108934 268f45cc-cd09-0410-ab3c-d52691b4dbfc
parent f5af4c40
2012-02-26 Mark Hahnenberg <mhahnenberg@apple.com>
Implement fast path for op_new_array in the baseline JIT
https://bugs.webkit.org/show_bug.cgi?id=78612
Reviewed by Filip Pizlo.
heap/CopiedAllocator.h:
(CopiedAllocator): Friended the JIT to allow access to m_currentOffset.
* heap/CopiedSpace.h:
(CopiedSpace): Friended the JIT to allow access to isOversize.
(JSC::CopiedSpace::allocator):
* heap/Heap.h:
(JSC::Heap::storageAllocator): Added a getter for the CopiedAllocator class so the JIT
can use it for simple allocation i.e. when we can just bump the offset without having to
do anything else.
* jit/JIT.cpp:
(JSC::JIT::privateCompileSlowCases): Added new slow case for op_new_array for when
we have to bail out because the fast allocation path fails for whatever reason.
* jit/JIT.h:
(JIT):
* jit/JITInlineMethods.h:
(JSC::JIT::emitAllocateBasicStorage): Added utility function that allows objects to
allocate generic backing stores. This function is used by emitAllocateJSArray.
(JSC):
(JSC::JIT::emitAllocateJSArray): Added utility function that allows the client to
more easily allocate JSArrays. This function is used by emit_op_new_array and I expect
it will also be used for emit_op_new_array_buffer.
* jit/JITOpcodes.cpp:
(JSC::JIT::emit_op_new_array): Changed to do inline allocation of JSArrays. Still does
a stub call for oversize arrays.
(JSC):
(JSC::JIT::emitSlow_op_new_array): New slow path that just bails out to a stub call if we
fail in any way on the fast path.
* runtime/JSArray.cpp:
(JSC):
* runtime/JSArray.h: Added lots of offset functions for all the fields that we need to
initialize in the JIT.
(ArrayStorage):
(JSC::ArrayStorage::lengthOffset):
(JSC::ArrayStorage::numValuesInVectorOffset):
(JSC::ArrayStorage::allocBaseOffset):
(JSC::ArrayStorage::vectorOffset):
(JSArray):
(JSC::JSArray::sparseValueMapOffset):
(JSC::JSArray::subclassDataOffset):
(JSC::JSArray::indexBiasOffset):
(JSC):
(JSC::JSArray::storageSize): Moved this function from being a static function in the cpp file
to being a static function in the JSArray class. This move allows the JIT to call it to
see what size it should allocate.
2012-02-26 Patrick Gansterer <paroga@webkit.org>
Unreviewed. Build fix for ENABLE(CLASSIC_INTERPRETER) after r108681.
......@@ -31,6 +31,7 @@
namespace JSC {
class CopiedAllocator {
friend class JIT;
public:
CopiedAllocator();
void* allocate(size_t);
......
......@@ -46,6 +46,7 @@ class HeapBlock;
class CopiedSpace {
friend class SlotVisitor;
friend class JIT;
public:
CopiedSpace(Heap*);
void init();
......@@ -53,6 +54,8 @@ public:
CheckedBoolean tryAllocate(size_t, void**);
CheckedBoolean tryReallocate(void**, size_t, size_t);
CopiedAllocator& allocator() { return m_allocator; }
void startedCopying();
void doneCopying();
bool isInCopyPhase() { return m_inCopyingPhase; }
......
......@@ -99,6 +99,7 @@ namespace JSC {
MarkedAllocator& firstAllocatorWithoutDestructors() { return m_objectSpace.firstAllocator(); }
MarkedAllocator& allocatorForObjectWithoutDestructor(size_t bytes) { return m_objectSpace.allocatorFor(bytes); }
MarkedAllocator& allocatorForObjectWithDestructor(size_t bytes) { return m_objectSpace.destructorAllocatorFor(bytes); }
CopiedAllocator& storageAllocator() { return m_storageSpace.allocator(); }
CheckedBoolean tryAllocateStorage(size_t, void**);
CheckedBoolean tryReallocateStorage(void**, size_t, size_t);
......
......@@ -479,6 +479,7 @@ void JIT::privateCompileSlowCases()
DEFINE_SLOWCASE_OP(op_negate)
#endif
DEFINE_SLOWCASE_OP(op_neq)
DEFINE_SLOWCASE_OP(op_new_array)
DEFINE_SLOWCASE_OP(op_new_object)
DEFINE_SLOWCASE_OP(op_new_func)
DEFINE_SLOWCASE_OP(op_new_func_exp)
......
......@@ -336,8 +336,10 @@ namespace JSC {
void emitWriteBarrier(JSCell* owner, RegisterID value, RegisterID scratch, WriteBarrierMode, WriteBarrierUseKind);
template<typename ClassType, bool destructor, typename StructureType> void emitAllocateBasicJSObject(StructureType, RegisterID result, RegisterID storagePtr);
void emitAllocateBasicStorage(size_t, RegisterID result, RegisterID storagePtr);
template<typename T> void emitAllocateJSFinalObject(T structure, RegisterID result, RegisterID storagePtr);
void emitAllocateJSFunction(FunctionExecutable*, RegisterID scopeChain, RegisterID result, RegisterID storagePtr);
void emitAllocateJSArray(unsigned valuesRegister, unsigned length, RegisterID cellResult, RegisterID storageResult, RegisterID storagePtr);
#if ENABLE(VALUE_PROFILER)
// This assumes that the value to profile is in regT0 and that regT3 is available for
......@@ -957,7 +959,7 @@ namespace JSC {
void emitSlow_op_urshift(Instruction*, Vector<SlowCaseEntry>::iterator&);
void emitSlow_op_new_func(Instruction*, Vector<SlowCaseEntry>::iterator&);
void emitSlow_op_new_func_exp(Instruction*, Vector<SlowCaseEntry>::iterator&);
void emitSlow_op_new_array(Instruction*, Vector<SlowCaseEntry>::iterator&);
void emitRightShift(Instruction*, bool isUnsigned);
void emitRightShiftSlowCase(Instruction*, Vector<SlowCaseEntry>::iterator&, bool isUnsigned);
......
......@@ -459,6 +459,78 @@ inline void JIT::emitAllocateJSFunction(FunctionExecutable* executable, Register
#endif
}
inline void JIT::emitAllocateBasicStorage(size_t size, RegisterID result, RegisterID storagePtr)
{
CopiedAllocator* allocator = &m_globalData->heap.storageAllocator();
// FIXME: We need to check for wrap-around.
// Check to make sure that the allocation will fit in the current block.
loadPtr(&allocator->m_currentOffset, result);
addPtr(TrustedImm32(size), result);
loadPtr(&allocator->m_currentBlock, storagePtr);
addPtr(TrustedImm32(HeapBlock::s_blockSize), storagePtr);
addSlowCase(branchPtr(AboveOrEqual, result, storagePtr));
// Load the original offset.
loadPtr(&allocator->m_currentOffset, result);
// Bump the pointer forward.
move(result, storagePtr);
addPtr(TrustedImm32(size), storagePtr);
storePtr(storagePtr, &allocator->m_currentOffset);
}
inline void JIT::emitAllocateJSArray(unsigned valuesRegister, unsigned length, RegisterID cellResult, RegisterID storageResult, RegisterID storagePtr)
{
unsigned initialLength = std::max(length, 4U);
size_t initialStorage = JSArray::storageSize(initialLength);
// Allocate the cell for the array.
emitAllocateBasicJSObject<JSArray, false>(TrustedImmPtr(m_codeBlock->globalObject()->arrayStructure()), cellResult, storagePtr);
// Allocate the backing store for the array.
emitAllocateBasicStorage(initialStorage, storageResult, storagePtr);
// Store all the necessary info in the ArrayStorage.
storePtr(storageResult, Address(storageResult, ArrayStorage::allocBaseOffset()));
store32(Imm32(length), Address(storageResult, ArrayStorage::lengthOffset()));
store32(Imm32(length), Address(storageResult, ArrayStorage::numValuesInVectorOffset()));
// Store the newly allocated ArrayStorage.
storePtr(storageResult, Address(cellResult, JSArray::storageOffset()));
// Store the vector length and index bias.
store32(Imm32(initialLength), Address(cellResult, JSArray::vectorLengthOffset()));
store32(TrustedImm32(0), Address(cellResult, JSArray::indexBiasOffset()));
// Initialize the subclass data and the sparse value map.
storePtr(TrustedImmPtr(0), Address(cellResult, JSArray::subclassDataOffset()));
storePtr(TrustedImmPtr(0), Address(cellResult, JSArray::sparseValueMapOffset()));
// Store the values we have.
for (unsigned i = 0; i < length; i++) {
#if USE(JSVALUE64)
loadPtr(Address(callFrameRegister, (valuesRegister + i) * sizeof(Register)), storagePtr);
storePtr(storagePtr, Address(storageResult, ArrayStorage::vectorOffset() + sizeof(WriteBarrier<Unknown>) * i));
#else
load32(Address(callFrameRegister, (valuesRegister + i) * sizeof(Register)), storagePtr);
store32(storagePtr, Address(storageResult, ArrayStorage::vectorOffset() + sizeof(WriteBarrier<Unknown>) * i));
load32(Address(callFrameRegister, (valuesRegister + i) * sizeof(Register) + sizeof(uint32_t)), storagePtr);
store32(storagePtr, Address(storageResult, ArrayStorage::vectorOffset() + sizeof(WriteBarrier<Unknown>) * i + sizeof(uint32_t)));
#endif
}
// Zero out the remaining slots.
for (unsigned i = length; i < initialLength; i++) {
#if USE(JSVALUE64)
storePtr(TrustedImmPtr(0), Address(storageResult, ArrayStorage::vectorOffset() + sizeof(WriteBarrier<Unknown>) * i));
#else
store32(TrustedImm32(static_cast<int>(JSValue::EmptyValueTag)), Address(storageResult, ArrayStorage::vectorOffset() + sizeof(WriteBarrier<Unknown>) * i + OBJECT_OFFSETOF(JSValue, u.asBits.tag)));
store32(TrustedImm32(0), Address(storageResult, ArrayStorage::vectorOffset() + sizeof(WriteBarrier<Unknown>) * i + OBJECT_OFFSETOF(JSValue, u.asBits.payload)));
#endif
}
}
#if ENABLE(VALUE_PROFILER)
inline void JIT::emitValueProfilingSite(ValueProfile* valueProfile)
{
......
......@@ -29,6 +29,7 @@
#include "JIT.h"
#include "Arguments.h"
#include "CopiedSpaceInlineMethods.h"
#include "Heap.h"
#include "JITInlineMethods.h"
#include "JITStubCall.h"
......@@ -1645,6 +1646,28 @@ void JIT::emitSlow_op_new_func_exp(Instruction* currentInstruction, Vector<SlowC
void JIT::emit_op_new_array(Instruction* currentInstruction)
{
int length = currentInstruction[3].u.operand;
if (CopiedSpace::isOversize(JSArray::storageSize(length))) {
JITStubCall stubCall(this, cti_op_new_array);
stubCall.addArgument(Imm32(currentInstruction[2].u.operand));
stubCall.addArgument(Imm32(currentInstruction[3].u.operand));
stubCall.call(currentInstruction[1].u.operand);
return;
}
int dst = currentInstruction[1].u.operand;
int values = currentInstruction[2].u.operand;
emitAllocateJSArray(values, length, regT0, regT1, regT2);
emitStoreCell(dst, regT0);
}
void JIT::emitSlow_op_new_array(Instruction* currentInstruction, Vector<SlowCaseEntry>::iterator& iter)
{
int length = currentInstruction[3].u.operand;
if (CopiedSpace::isOversize(JSArray::storageSize(length)))
return;
linkSlowCase(iter); // Not enough space in MarkedSpace for cell.
linkSlowCase(iter); // Not enough space in CopiedSpace for storage.
JITStubCall stubCall(this, cti_op_new_array);
stubCall.addArgument(Imm32(currentInstruction[2].u.operand));
stubCall.addArgument(Imm32(currentInstruction[3].u.operand));
......
......@@ -104,20 +104,6 @@ const ClassInfo JSArray::s_info = {"Array", &JSNonFinalObject::s_info, 0, 0, CRE
// This value is capped by the constant FIRST_VECTOR_GROW defined above.
static unsigned lastArraySize = 0;
static inline size_t storageSize(unsigned vectorLength)
{
ASSERT(vectorLength <= MAX_STORAGE_VECTOR_LENGTH);
// MAX_STORAGE_VECTOR_LENGTH is defined such that provided (vectorLength <= MAX_STORAGE_VECTOR_LENGTH)
// - as asserted above - the following calculation cannot overflow.
size_t size = (sizeof(ArrayStorage) - sizeof(WriteBarrier<Unknown>)) + (vectorLength * sizeof(WriteBarrier<Unknown>));
// Assertion to detect integer overflow in previous calculation (should not be possible, provided that
// MAX_STORAGE_VECTOR_LENGTH is correctly defined).
ASSERT(((size - (sizeof(ArrayStorage) - sizeof(WriteBarrier<Unknown>))) / sizeof(WriteBarrier<Unknown>) == vectorLength) && (size >= (sizeof(ArrayStorage) - sizeof(WriteBarrier<Unknown>))));
return size;
}
static inline bool isDenseEnoughForVector(unsigned length, unsigned numValues)
{
return length <= MIN_SPARSE_ARRAY_INDEX || length / minDensityMultiplier <= numValues;
......
......@@ -122,11 +122,17 @@ namespace JSC {
uintptr_t m_padding;
#endif
WriteBarrier<Unknown> m_vector[1];
static ptrdiff_t lengthOffset() { return OBJECT_OFFSETOF(ArrayStorage, m_length); }
static ptrdiff_t numValuesInVectorOffset() { return OBJECT_OFFSETOF(ArrayStorage, m_numValuesInVector); }
static ptrdiff_t allocBaseOffset() { return OBJECT_OFFSETOF(ArrayStorage, m_allocBase); }
static ptrdiff_t vectorOffset() { return OBJECT_OFFSETOF(ArrayStorage, m_vector); }
};
class JSArray : public JSNonFinalObject {
friend class LLIntOffsetsExtractor;
friend class Walker;
friend class JIT;
protected:
JS_EXPORT_PRIVATE explicit JSArray(JSGlobalData&, Structure*);
......@@ -262,6 +268,7 @@ namespace JSC {
JS_EXPORT_PRIVATE void setSubclassData(void*);
private:
static size_t storageSize(unsigned vectorLength);
bool isLengthWritable()
{
SparseArrayValueMap* map = m_sparseValueMap;
......@@ -293,6 +300,10 @@ namespace JSC {
// FIXME: Maybe SparseArrayValueMap should be put into its own JSCell?
SparseArrayValueMap* m_sparseValueMap;
void* m_subclassData; // A JSArray subclass can use this to fill the vector lazily.
static ptrdiff_t sparseValueMapOffset() { return OBJECT_OFFSETOF(JSArray, m_sparseValueMap); }
static ptrdiff_t subclassDataOffset() { return OBJECT_OFFSETOF(JSArray, m_subclassData); }
static ptrdiff_t indexBiasOffset() { return OBJECT_OFFSETOF(JSArray, m_indexBias); }
};
inline JSArray* JSArray::create(JSGlobalData& globalData, Structure* structure, unsigned initialLength)
......@@ -334,6 +345,30 @@ namespace JSC {
return i;
}
} // namespace JSC
// The definition of MAX_STORAGE_VECTOR_LENGTH is dependant on the definition storageSize
// function below - the MAX_STORAGE_VECTOR_LENGTH limit is defined such that the storage
// size calculation cannot overflow. (sizeof(ArrayStorage) - sizeof(WriteBarrier<Unknown>)) +
// (vectorLength * sizeof(WriteBarrier<Unknown>)) must be <= 0xFFFFFFFFU (which is maximum value of size_t).
#define MAX_STORAGE_VECTOR_LENGTH static_cast<unsigned>((0xFFFFFFFFU - (sizeof(ArrayStorage) - sizeof(WriteBarrier<Unknown>))) / sizeof(WriteBarrier<Unknown>))
// These values have to be macros to be used in max() and min() without introducing
// a PIC branch in Mach-O binaries, see <rdar://problem/5971391>.
#define MIN_SPARSE_ARRAY_INDEX 10000U
#define MAX_STORAGE_VECTOR_INDEX (MAX_STORAGE_VECTOR_LENGTH - 1)
inline size_t JSArray::storageSize(unsigned vectorLength)
{
ASSERT(vectorLength <= MAX_STORAGE_VECTOR_LENGTH);
// MAX_STORAGE_VECTOR_LENGTH is defined such that provided (vectorLength <= MAX_STORAGE_VECTOR_LENGTH)
// - as asserted above - the following calculation cannot overflow.
size_t size = (sizeof(ArrayStorage) - sizeof(WriteBarrier<Unknown>)) + (vectorLength * sizeof(WriteBarrier<Unknown>));
// Assertion to detect integer overflow in previous calculation (should not be possible, provided that
// MAX_STORAGE_VECTOR_LENGTH is correctly defined).
ASSERT(((size - (sizeof(ArrayStorage) - sizeof(WriteBarrier<Unknown>))) / sizeof(WriteBarrier<Unknown>) == vectorLength) && (size >= (sizeof(ArrayStorage) - sizeof(WriteBarrier<Unknown>))));
return size;
}
} // namespace JSC
#endif // JSArray_h
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment