Commit 6a429819 authored by ggaren@apple.com's avatar ggaren@apple.com

JavaScriptCore: Changed GC from mark-sweep to mark-allocate.

        
Reviewed by Sam Weinig.

Added WeakGCMap to keep WebCore blissfully ignorant about objects that
have become garbage but haven't run their destructors yet.
        
1% SunSpider speedup.
7.6% v8 speedup (37% splay speedup).
17% speedup on bench-alloc-nonretained.js.
18% speedup on bench-alloc-retained.js.

* API/JSBase.cpp:
(JSGarbageCollect):
* API/JSContextRef.cpp:
* JavaScriptCore.exp:
* JavaScriptCore.xcodeproj/project.pbxproj: Updated for renames and new
files.

* debugger/Debugger.cpp:
(JSC::Debugger::recompileAllJSFunctions): Updated to use the Collector
iterator abstraction.

* jsc.cpp:
(functionGC): Updated for rename.

* runtime/Collector.cpp: Slightly reduced the number of allocations per
collection, so that small workloads only allocate on collector block,
rather than two.

(JSC::Heap::Heap): Updated to use the new allocateBlock function.

(JSC::Heap::destroy): Updated to use the new freeBlocks function.

(JSC::Heap::allocateBlock): New function to initialize a block when
allocating it.

(JSC::Heap::freeBlock): Consolidated the responsibility for running
destructors into this function.

(JSC::Heap::freeBlocks): Updated to use freeBlock.

(JSC::Heap::recordExtraCost): Sweep the heap in this reporting function,
so that allocation, which is more common, doesn't have to check extraCost.

(JSC::Heap::heapAllocate): Run destructors right before recycling a
garbage cell. This has better cache utilization than a separate sweep phase.

(JSC::Heap::resizeBlocks):
(JSC::Heap::growBlocks):
(JSC::Heap::shrinkBlocks): New set of functions for managing the size of
the heap, now that the heap doesn't maintain any information about its
size.

(JSC::isPointerAligned):
(JSC::isHalfCellAligned):
(JSC::isPossibleCell):
(JSC::isCellAligned):
(JSC::Heap::markConservatively): Cleaned up this code a bit.

(JSC::Heap::clearMarkBits):
(JSC::Heap::markedCells): Some helper functions for examining the the mark
bitmap.

(JSC::Heap::sweep): Simplified this function by using a DeadObjectIterator.

(JSC::Heap::markRoots): Reordered some operations for clarity.

(JSC::Heap::objectCount):
(JSC::Heap::addToStatistics):
(JSC::Heap::statistics): Rewrote these functions to calculate an object
count on demand, since the heap doesn't maintain this information by 
itself.

(JSC::Heap::reset): New function for resetting the heap once we've
exhausted heap space.

(JSC::Heap::collectAllGarbage): This function matches the old collect()
behavior, but it's now an uncommon function used only by API.

* runtime/Collector.h:
(JSC::CollectorBitmap::count):
(JSC::CollectorBitmap::isEmpty): Added some helper functions for managing
the collector mark bitmap.

(JSC::Heap::reportExtraMemoryCost): Changed reporting from cell equivalents
to bytes, so it's easier to understand.
        
* runtime/CollectorHeapIterator.h:
(JSC::CollectorHeapIterator::CollectorHeapIterator):
(JSC::CollectorHeapIterator::operator!=):
(JSC::CollectorHeapIterator::operator*):
(JSC::CollectorHeapIterator::advance):
(JSC::::LiveObjectIterator):
(JSC::::operator):
(JSC::::DeadObjectIterator):
(JSC::::ObjectIterator): New iterators for encapsulating details about
heap layout, and what's live and dead on the heap.

* runtime/JSArray.cpp:
(JSC::JSArray::putSlowCase):
(JSC::JSArray::increaseVectorLength): Delay reporting extra cost until
we're fully constructed, so the heap mark phase won't visit us in an
invalid state.

* runtime/JSCell.h:
(JSC::JSCell::):
(JSC::JSCell::createDummyStructure):
(JSC::JSCell::JSCell):
* runtime/JSGlobalData.cpp:
(JSC::JSGlobalData::JSGlobalData):
* runtime/JSGlobalData.h: Added a dummy cell to simplify allocation logic.

* runtime/JSString.h:
(JSC::jsSubstring): Don't report extra cost for substrings, since they
share a buffer that's already reported extra cost.

* runtime/Tracing.d:
* runtime/Tracing.h: Changed these dtrace hooks not to report object
counts, since they're no longer cheap to compute.

* runtime/UString.h: Updated for renames.

* runtime/WeakGCMap.h: Added.
(JSC::WeakGCMap::isEmpty):
(JSC::WeakGCMap::uncheckedGet):
(JSC::WeakGCMap::uncheckedBegin):
(JSC::WeakGCMap::uncheckedEnd):
(JSC::::get):
(JSC::::take):
(JSC::::set):
(JSC::::uncheckedRemove): Mentioned above.

* wtf/StdLibExtras.h:
(WTF::bitCount): Added a bit population count function, so the heap can
count live objects to fulfill statistics questions.

JavaScriptGlue: Changed GC from mark-sweep to mark-allocate.
        
Reviewed by Sam Weinig.

* JavaScriptGlue.cpp:
(JSCollect): Updated for rename. Fixed a bug where JSGlue would not check
to avoid nested GC calls.

WebCore: Changed GC from mark-sweep to mark-allocate.

Reviewed by Sam Weinig.

* ForwardingHeaders/runtime/WeakGCMap.h: Added.
* bindings/js/GCController.cpp:
(WebCore::collect):
(WebCore::GCController::gcTimerFired):
(WebCore::GCController::garbageCollectNow): Updated for rename.

* bindings/js/JSDOMBinding.cpp:
(WebCore::removeWrappers):
(WebCore::hasCachedDOMObjectWrapperUnchecked):
(WebCore::hasCachedDOMObjectWrapper):
(WebCore::hasCachedDOMNodeWrapperUnchecked):
(WebCore::forgetDOMObject):
(WebCore::forgetDOMNode):
(WebCore::isObservableThroughDOM):
(WebCore::markDOMNodesForDocument):
(WebCore::markDOMObjectWrapper):
(WebCore::markDOMNodeWrapper):
* bindings/js/JSDOMBinding.h: Changed DOM wrapper maps to be WeakGCMaps.
Don't ASSERT that an item must be in the WeakGCMap when its destructor
runs, since it might have been overwritten in the map first.

* bindings/js/JSDocumentCustom.cpp:
(WebCore::toJS): Changed Document from a DOM object wrapper to a DOM node
wrapper, to simplify some code.

* bindings/js/JSInspectedObjectWrapper.cpp:
(WebCore::JSInspectedObjectWrapper::JSInspectedObjectWrapper):
(WebCore::JSInspectedObjectWrapper::~JSInspectedObjectWrapper):
* bindings/js/JSInspectorCallbackWrapper.cpp: Use a WeakGCMap for these
wrappers.

* bindings/js/JSNodeCustom.cpp:
(WebCore::JSNode::markChildren): Updated for WeakGCMap and Document using
a DOM node wrapper instead of a DOM object wrapper.

* bindings/js/JSSVGPODTypeWrapper.h:
(WebCore::JSSVGDynamicPODTypeWrapperCache::wrapperMap):
(WebCore::JSSVGDynamicPODTypeWrapperCache::lookupOrCreateWrapper):
(WebCore::JSSVGDynamicPODTypeWrapperCache::forgetWrapper):
(WebCore::::~JSSVGDynamicPODTypeWrapper): Shined a small beam of sanity light
on this code. Use hashtable-based lookup in JSSVGPODTypeWrapper.h instead
of linear lookup through iteration, since that's what hashtables were
invented for. Make JSSVGPODTypeWrapper.h responsible for reomving itself
from the table, instead of its JS wrapper, to decouple these objects from
GC, and because these objects are refCounted, not solely owned by their
JS wrappers.

* bindings/scripts/CodeGeneratorJS.pm:
* dom/Document.h: Adopted changes above.



git-svn-id: http://svn.webkit.org/repository/webkit/trunk@52082 268f45cc-cd09-0410-ab3c-d52691b4dbfc
parent 596de121
......@@ -99,7 +99,7 @@ void JSGarbageCollect(JSContextRef ctx)
JSLock lock(globalData.isSharedInstance ? LockForReal : SilenceAssertionsOnly);
if (!globalData.heap.isBusy())
globalData.heap.collect();
globalData.heap.collectAllGarbage();
// FIXME: Perhaps we should trigger a second mark and sweep
// once the garbage collector is done if this is called when
......
......@@ -133,7 +133,7 @@ void JSGlobalContextRelease(JSGlobalContextRef ctx)
ASSERT(!globalData.heap.isBusy());
globalData.heap.destroy();
} else
globalData.heap.collect();
globalData.heap.collectAllGarbage();
globalData.deref();
}
......
2009-12-13 Geoffrey Garen <ggaren@apple.com>
Reviewed by Sam Weinig.
Changed GC from mark-sweep to mark-allocate.
Added WeakGCMap to keep WebCore blissfully ignorant about objects that
have become garbage but haven't run their destructors yet.
1% SunSpider speedup.
7.6% v8 speedup (37% splay speedup).
17% speedup on bench-alloc-nonretained.js.
18% speedup on bench-alloc-retained.js.
* API/JSBase.cpp:
(JSGarbageCollect):
* API/JSContextRef.cpp:
* JavaScriptCore.exp:
* JavaScriptCore.xcodeproj/project.pbxproj: Updated for renames and new
files.
* debugger/Debugger.cpp:
(JSC::Debugger::recompileAllJSFunctions): Updated to use the Collector
iterator abstraction.
* jsc.cpp:
(functionGC): Updated for rename.
* runtime/Collector.cpp: Slightly reduced the number of allocations per
collection, so that small workloads only allocate on collector block,
rather than two.
(JSC::Heap::Heap): Updated to use the new allocateBlock function.
(JSC::Heap::destroy): Updated to use the new freeBlocks function.
(JSC::Heap::allocateBlock): New function to initialize a block when
allocating it.
(JSC::Heap::freeBlock): Consolidated the responsibility for running
destructors into this function.
(JSC::Heap::freeBlocks): Updated to use freeBlock.
(JSC::Heap::recordExtraCost): Sweep the heap in this reporting function,
so that allocation, which is more common, doesn't have to check extraCost.
(JSC::Heap::heapAllocate): Run destructors right before recycling a
garbage cell. This has better cache utilization than a separate sweep phase.
(JSC::Heap::resizeBlocks):
(JSC::Heap::growBlocks):
(JSC::Heap::shrinkBlocks): New set of functions for managing the size of
the heap, now that the heap doesn't maintain any information about its
size.
(JSC::isPointerAligned):
(JSC::isHalfCellAligned):
(JSC::isPossibleCell):
(JSC::isCellAligned):
(JSC::Heap::markConservatively): Cleaned up this code a bit.
(JSC::Heap::clearMarkBits):
(JSC::Heap::markedCells): Some helper functions for examining the the mark
bitmap.
(JSC::Heap::sweep): Simplified this function by using a DeadObjectIterator.
(JSC::Heap::markRoots): Reordered some operations for clarity.
(JSC::Heap::objectCount):
(JSC::Heap::addToStatistics):
(JSC::Heap::statistics): Rewrote these functions to calculate an object
count on demand, since the heap doesn't maintain this information by
itself.
(JSC::Heap::reset): New function for resetting the heap once we've
exhausted heap space.
(JSC::Heap::collectAllGarbage): This function matches the old collect()
behavior, but it's now an uncommon function used only by API.
* runtime/Collector.h:
(JSC::CollectorBitmap::count):
(JSC::CollectorBitmap::isEmpty): Added some helper functions for managing
the collector mark bitmap.
(JSC::Heap::reportExtraMemoryCost): Changed reporting from cell equivalents
to bytes, so it's easier to understand.
* runtime/CollectorHeapIterator.h:
(JSC::CollectorHeapIterator::CollectorHeapIterator):
(JSC::CollectorHeapIterator::operator!=):
(JSC::CollectorHeapIterator::operator*):
(JSC::CollectorHeapIterator::advance):
(JSC::::LiveObjectIterator):
(JSC::::operator):
(JSC::::DeadObjectIterator):
(JSC::::ObjectIterator): New iterators for encapsulating details about
heap layout, and what's live and dead on the heap.
* runtime/JSArray.cpp:
(JSC::JSArray::putSlowCase):
(JSC::JSArray::increaseVectorLength): Delay reporting extra cost until
we're fully constructed, so the heap mark phase won't visit us in an
invalid state.
* runtime/JSCell.h:
(JSC::JSCell::):
(JSC::JSCell::createDummyStructure):
(JSC::JSCell::JSCell):
* runtime/JSGlobalData.cpp:
(JSC::JSGlobalData::JSGlobalData):
* runtime/JSGlobalData.h: Added a dummy cell to simplify allocation logic.
* runtime/JSString.h:
(JSC::jsSubstring): Don't report extra cost for substrings, since they
share a buffer that's already reported extra cost.
* runtime/Tracing.d:
* runtime/Tracing.h: Changed these dtrace hooks not to report object
counts, since they're no longer cheap to compute.
* runtime/UString.h: Updated for renames.
* runtime/WeakGCMap.h: Added.
(JSC::WeakGCMap::isEmpty):
(JSC::WeakGCMap::uncheckedGet):
(JSC::WeakGCMap::uncheckedBegin):
(JSC::WeakGCMap::uncheckedEnd):
(JSC::::get):
(JSC::::take):
(JSC::::set):
(JSC::::uncheckedRemove): Mentioned above.
* wtf/StdLibExtras.h:
(WTF::bitCount): Added a bit population count function, so the heap can
count live objects to fulfill statistics questions.
The very last cell in the block is not allocated -- should not be marked.
2009-12-13 Geoffrey Garen <ggaren@apple.com>
Windows build fix: Export some new symbols.
* JavaScriptCore.vcproj/JavaScriptCore/JavaScriptCore.def:
2009-12-13 Geoffrey Garen <ggaren@apple.com>
Windows build fix: Removed some old exports.
* JavaScriptCore.vcproj/JavaScriptCore/JavaScriptCore.def:
2009-12-13 Geoffrey Garen <ggaren@apple.com>
Windows build fix: Use unsigned instead of uint32_t to avoid dependencies.
* wtf/StdLibExtras.h:
(WTF::bitCount):
2009-12-13 Gavin Barraclough <barraclough@apple.com>
Reviewed by NOBODY (speculative Windows build fix).
......
......@@ -180,16 +180,15 @@ __ZN3JSC24createStackOverflowErrorEPNS_9ExecStateE
__ZN3JSC25evaluateInGlobalCallFrameERKNS_7UStringERNS_7JSValueEPNS_14JSGlobalObjectE
__ZN3JSC35createInterruptedExecutionExceptionEPNS_12JSGlobalDataE
__ZN3JSC3NaNE
__ZN3JSC4Heap11objectCountEv
__ZN3JSC4Heap14primaryHeapEndEv
__ZN3JSC4Heap15recordExtraCostEm
__ZN3JSC4Heap16primaryHeapBeginEv
__ZN3JSC4Heap17collectAllGarbageEv
__ZN3JSC4Heap17globalObjectCountEv
__ZN3JSC4Heap20protectedObjectCountEv
__ZN3JSC4Heap25protectedObjectTypeCountsEv
__ZN3JSC4Heap26protectedGlobalObjectCountEv
__ZN3JSC4Heap6isBusyEv
__ZN3JSC4Heap7collectEv
__ZN3JSC4Heap7destroyEv
__ZN3JSC4Heap7protectENS_7JSValueE
__ZN3JSC4Heap8allocateEm
......@@ -368,6 +367,7 @@ __ZNK3JSC18PropertyDescriptor6getterEv
__ZNK3JSC18PropertyDescriptor6setterEv
__ZNK3JSC18PropertyDescriptor8writableEv
__ZNK3JSC4Heap10statisticsEv
__ZNK3JSC4Heap11objectCountEv
__ZNK3JSC6JSCell11toPrimitiveEPNS_9ExecStateENS_22PreferredPrimitiveTypeE
__ZNK3JSC6JSCell12toThisObjectEPNS_9ExecStateE
__ZNK3JSC6JSCell12toThisStringEPNS_9ExecStateE
......
......@@ -62,7 +62,7 @@ EXPORTS
?classInfo@JSCell@JSC@@UBEPBUClassInfo@2@XZ
?className@JSObject@JSC@@UBE?AVUString@2@XZ
?collate@Collator@WTF@@QBE?AW4Result@12@PB_WI0I@Z
?collect@Heap@JSC@@QAE_NXZ
?collectAllGarbage@Heap@JSC@@QAEXXZ
?computeHash@Rep@UString@JSC@@SAIPBDH@Z
?computeHash@Rep@UString@JSC@@SAIPB_WH@Z
?configurable@PropertyDescriptor@JSC@@QBE_NXZ
......@@ -115,8 +115,8 @@ EXPORTS
?detach@Debugger@JSC@@UAEXPAVJSGlobalObject@2@@Z
?detachThread@WTF@@YAXI@Z
?didTimeOut@TimeoutChecker@JSC@@QAE_NPAVExecState@2@@Z
?dumpSampleData@JSGlobalData@JSC@@QAEXPAVExecState@2@@Z
?doubleToStringInJavaScriptFormat@WTF@@YAXNQADPAI@Z
?dumpSampleData@JSGlobalData@JSC@@QAEXPAVExecState@2@@Z
?enumerable@PropertyDescriptor@JSC@@QBE_NXZ
?equal@Identifier@JSC@@SA_NPBURep@UString@2@PBD@Z
?equal@JSC@@YA_NPBURep@UString@1@0@Z
......@@ -200,12 +200,10 @@ EXPORTS
?materializePropertyMap@Structure@JSC@@AAEXXZ
?name@InternalFunction@JSC@@QAEABVUString@2@PAVExecState@2@@Z
?nonInlineNaN@JSC@@YANXZ
?objectCount@Heap@JSC@@QAEIXZ
?objectCount@Heap@JSC@@QBEIXZ
?objectProtoFuncToString@JSC@@YI?AVJSValue@1@PAVExecState@1@PAVJSObject@1@V21@ABVArgList@1@@Z
?parse@Parser@JSC@@AAEXPAVJSGlobalData@2@PAHPAVUString@2@@Z
?parseDateFromNullTerminatedCharacters@WTF@@YANPBD@Z
?primaryHeapBegin@Heap@JSC@@QAE?AV?$CollectorHeapIterator@$0A@@2@XZ
?primaryHeapEnd@Heap@JSC@@QAE?AV?$CollectorHeapIterator@$0A@@2@XZ
?profiler@Profiler@JSC@@SAPAV12@XZ
?protect@Heap@JSC@@QAEXVJSValue@2@@Z
?protectedGlobalObjectCount@Heap@JSC@@QAEIXZ
......
......@@ -167,6 +167,7 @@
14BD59C50A3E8F9F00BAF59C /* JavaScriptCore.framework in Frameworks */ = {isa = PBXBuildFile; fileRef = 932F5BD90822A1C700736975 /* JavaScriptCore.framework */; };
14BD5A300A3E91F600BAF59C /* JSContextRef.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 14BD5A290A3E91F600BAF59C /* JSContextRef.cpp */; };
14BD5A320A3E91F600BAF59C /* JSValueRef.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 14BD5A2B0A3E91F600BAF59C /* JSValueRef.cpp */; };
14BFCE6910CDB1FC00364CCE /* WeakGCMap.h in Headers */ = {isa = PBXBuildFile; fileRef = 14BFCE6810CDB1FC00364CCE /* WeakGCMap.h */; settings = {ATTRIBUTES = (Private, ); }; };
14C5242B0F5355E900BA3D04 /* JITStubs.h in Headers */ = {isa = PBXBuildFile; fileRef = 14A6581A0F4E36F4000150FD /* JITStubs.h */; settings = {ATTRIBUTES = (Private, ); }; };
14E9D17B107EC469004DDA21 /* JSGlobalObjectFunctions.cpp in Sources */ = {isa = PBXBuildFile; fileRef = BC756FC60E2031B200DE7D12 /* JSGlobalObjectFunctions.cpp */; };
14F3488F0E95EF8A003648BC /* CollectorHeapIterator.h in Headers */ = {isa = PBXBuildFile; fileRef = 14F3488E0E95EF8A003648BC /* CollectorHeapIterator.h */; settings = {ATTRIBUTES = (); }; };
......@@ -637,6 +638,7 @@
14BD5A2A0A3E91F600BAF59C /* JSContextRef.h */ = {isa = PBXFileReference; fileEncoding = 30; lastKnownFileType = sourcecode.c.h; path = JSContextRef.h; sourceTree = "<group>"; };
14BD5A2B0A3E91F600BAF59C /* JSValueRef.cpp */ = {isa = PBXFileReference; fileEncoding = 30; lastKnownFileType = sourcecode.cpp.cpp; path = JSValueRef.cpp; sourceTree = "<group>"; };
14BD5A2D0A3E91F600BAF59C /* testapi.c */ = {isa = PBXFileReference; fileEncoding = 30; lastKnownFileType = sourcecode.c.c; name = testapi.c; path = API/tests/testapi.c; sourceTree = "<group>"; };
14BFCE6810CDB1FC00364CCE /* WeakGCMap.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WeakGCMap.h; sourceTree = "<group>"; };
14D792640DAA03FB001A9F05 /* RegisterFile.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = RegisterFile.h; sourceTree = "<group>"; };
14D857740A4696C80032146C /* testapi.js */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.javascript; name = testapi.js; path = API/tests/testapi.js; sourceTree = "<group>"; };
14DA818E0D99FD2000B0A4FB /* JSActivation.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JSActivation.h; sourceTree = "<group>"; };
......@@ -1607,6 +1609,9 @@
F692A8850255597D01FF60F7 /* UString.cpp */,
F692A8860255597D01FF60F7 /* UString.h */,
1420BE7A10AA6DDB00F455D2 /* WeakRandom.h */,
14BFCE6810CDB1FC00364CCE /* WeakGCMap.h */,
A7C2216810C745E000F97913 /* JSZombie.h */,
A7C2216B10C7469C00F97913 /* JSZombie.cpp */,
);
path = runtime;
sourceTree = "<group>";
......@@ -2013,6 +2018,7 @@
86CAFEE31035DDE60028A609 /* Executable.h in Headers */,
142D3939103E4560007DCB52 /* NumericStrings.h in Headers */,
A7FB61001040C38B0017A286 /* PropertyDescriptor.h in Headers */,
14BFCE6910CDB1FC00364CCE /* WeakGCMap.h in Headers */,
BC87CDB910712AD4000614CF /* JSONObject.lut.h in Headers */,
148CD1D8108CF902008163C6 /* JSContextRefPrivate.h in Headers */,
14A1563210966365006FA260 /* DateInstanceCache.h in Headers */,
......
......@@ -67,8 +67,9 @@ void Debugger::recompileAllJSFunctions(JSGlobalData* globalData)
FunctionExecutableSet functionExecutables;
SourceProviderMap sourceProviders;
Heap::iterator heapEnd = globalData->heap.primaryHeapEnd();
for (Heap::iterator it = globalData->heap.primaryHeapBegin(); it != heapEnd; ++it) {
LiveObjectIterator<PrimaryHeap> it = globalData->heap.primaryHeapBegin();
LiveObjectIterator<PrimaryHeap> heapEnd = globalData->heap.primaryHeapEnd();
for ( ; it != heapEnd; ++it) {
if (!(*it)->inherits(&JSFunction::info))
continue;
......
......@@ -194,7 +194,7 @@ JSValue JSC_HOST_CALL functionDebug(ExecState* exec, JSObject*, JSValue, const A
JSValue JSC_HOST_CALL functionGC(ExecState* exec, JSObject*, JSValue, const ArgList&)
{
JSLock lock(SilenceAssertionsOnly);
exec->heap()->collect();
exec->heap()->collectAllGarbage();
return jsUndefined();
}
......
This diff is collapsed.
......@@ -28,9 +28,9 @@
#include <wtf/HashSet.h>
#include <wtf/Noncopyable.h>
#include <wtf/OwnPtr.h>
#include <wtf/StdLibExtras.h>
#include <wtf/Threading.h>
// This is supremely lame that we require pthreads to build on windows.
#if ENABLE(JSC_MULTIPLE_THREADS)
#include <pthread.h>
#endif
......@@ -49,20 +49,19 @@ namespace JSC {
enum OperationInProgress { NoOperation, Allocation, Collection };
enum HeapType { PrimaryHeap, NumberHeap };
template <HeapType> class CollectorHeapIterator;
template <HeapType> class LiveObjectIterator;
struct CollectorHeap {
size_t nextBlock;
size_t nextCell;
CollectorBlock** blocks;
size_t numBlocks;
size_t usedBlocks;
size_t firstBlockWithPossibleSpace;
size_t numLiveObjects;
size_t numLiveObjectsAtLastCollect;
size_t extraCost;
#if ENABLE(JSC_ZOMBIES)
size_t numZombies;
#endif
bool didShrink;
OperationInProgress operationInProgress;
};
......@@ -70,21 +69,21 @@ namespace JSC {
class Heap : public Noncopyable {
public:
class Thread;
typedef CollectorHeapIterator<PrimaryHeap> iterator;
void destroy();
void* allocateNumber(size_t);
void* allocate(size_t);
bool collect();
bool isBusy(); // true if an allocation or collection is in progress
void collectAllGarbage();
static const size_t minExtraCostSize = 256;
static const size_t minExtraCost = 256;
static const size_t maxExtraCost = 1024 * 1024;
void reportExtraMemoryCost(size_t cost);
size_t objectCount();
size_t objectCount() const;
struct Statistics {
size_t size;
size_t free;
......@@ -114,13 +113,14 @@ namespace JSC {
JSGlobalData* globalData() const { return m_globalData; }
static bool isNumber(JSCell*);
// Iterators for the object heap.
iterator primaryHeapBegin();
iterator primaryHeapEnd();
LiveObjectIterator<PrimaryHeap> primaryHeapBegin();
LiveObjectIterator<PrimaryHeap> primaryHeapEnd();
private:
template <HeapType heapType> void* heapAllocate(size_t);
template <HeapType heapType> size_t sweep();
void reset();
void collectRemainingGarbage();
template <HeapType heapType> void sweep();
static CollectorBlock* cellBlock(const JSCell*);
static size_t cellOffset(const JSCell*);
......@@ -131,9 +131,20 @@ namespace JSC {
template <HeapType heapType> NEVER_INLINE CollectorBlock* allocateBlock();
template <HeapType heapType> NEVER_INLINE void freeBlock(size_t);
NEVER_INLINE void freeBlock(CollectorBlock*);
void freeBlocks(CollectorHeap*);
template <HeapType heapType> void freeBlocks();
template <HeapType heapType> void resizeBlocks();
template <HeapType heapType> void growBlocks(size_t neededBlocks);
template <HeapType heapType> void shrinkBlocks(size_t neededBlocks);
template <HeapType heapType> void clearMarkBits();
template <HeapType heapType> void clearMarkBits(CollectorBlock*);
template <HeapType heapType> size_t markedCells(size_t startBlock = 0, size_t startCell = 0) const;
void recordExtraCost(size_t);
template <HeapType heapType> void addToStatistics(Statistics&) const;
template <HeapType heapType> size_t objectCount() const;
void markRoots();
void markProtectedObjects(MarkStack&);
void markCurrentThreadConservatively(MarkStack&);
void markCurrentThreadConservativelyInternal(MarkStack&);
......@@ -189,44 +200,49 @@ namespace JSC {
const size_t SMALL_CELL_SIZE = CELL_SIZE / 2;
const size_t CELL_MASK = CELL_SIZE - 1;
const size_t CELL_ALIGN_MASK = ~CELL_MASK;
const size_t CELLS_PER_BLOCK = (BLOCK_SIZE * 8 - sizeof(uint32_t) * 8 - sizeof(void *) * 8 - 2 * (7 + 3 * 8)) / (CELL_SIZE * 8 + 2);
const size_t CELLS_PER_BLOCK = (BLOCK_SIZE - sizeof(Heap*) - sizeof(HeapType)) * 8 * CELL_SIZE / (8 * CELL_SIZE + 1) / CELL_SIZE; // one bitmap byte can represent 8 cells.
const size_t SMALL_CELLS_PER_BLOCK = 2 * CELLS_PER_BLOCK;
const size_t BITMAP_SIZE = (CELLS_PER_BLOCK + 7) / 8;
const size_t BITMAP_WORDS = (BITMAP_SIZE + 3) / sizeof(uint32_t);
struct CollectorBitmap {
uint32_t bits[BITMAP_WORDS];
bool get(size_t n) const { return !!(bits[n >> 5] & (1 << (n & 0x1F))); }
void set(size_t n) { bits[n >> 5] |= (1 << (n & 0x1F)); }
void clear(size_t n) { bits[n >> 5] &= ~(1 << (n & 0x1F)); }
void clearAll() { memset(bits, 0, sizeof(bits)); }
size_t count(size_t startCell = 0)
{
size_t result = 0;
for ( ; (startCell & 0x1F) != 0; ++startCell) {
if (get(startCell))
++result;
}
for (size_t i = startCell >> 5; i < BITMAP_WORDS; ++i)
result += WTF::bitCount(bits[i]);
return result;
}
size_t isEmpty() // Much more efficient than testing count() == 0.
{
for (size_t i = 0; i < BITMAP_WORDS; ++i)
if (bits[i] != 0)
return false;
return true;
}
};
struct CollectorCell {
union {
double memory[CELL_ARRAY_LENGTH];
struct {
void* zeroIfFree;
ptrdiff_t next;
} freeCell;
} u;
double memory[CELL_ARRAY_LENGTH];
};
struct SmallCollectorCell {
union {
double memory[CELL_ARRAY_LENGTH / 2];
struct {
void* zeroIfFree;
ptrdiff_t next;
} freeCell;
} u;
double memory[CELL_ARRAY_LENGTH / 2];
};
class CollectorBlock {
public:
CollectorCell cells[CELLS_PER_BLOCK];
uint32_t usedCells;
CollectorCell* freeList;
CollectorBitmap marked;
Heap* heap;
HeapType type;
......@@ -235,8 +251,6 @@ namespace JSC {
class SmallCellCollectorBlock {
public:
SmallCollectorCell cells[SMALL_CELLS_PER_BLOCK];
uint32_t usedCells;
SmallCollectorCell* freeList;
CollectorBitmap marked;
Heap* heap;
HeapType type;
......@@ -287,8 +301,8 @@ namespace JSC {
inline void Heap::reportExtraMemoryCost(size_t cost)
{
if (cost > minExtraCostSize)
recordExtraCost(cost / (CELL_SIZE * 2));
if (cost > minExtraCost)
recordExtraCost(cost);
}
} // namespace JSC
......
......@@ -31,58 +31,117 @@
namespace JSC {
template <HeapType heapType> class CollectorHeapIterator {
class CollectorHeapIterator {
public:
CollectorHeapIterator(CollectorBlock** block, CollectorBlock** endBlock);
bool operator!=(const CollectorHeapIterator<heapType>& other) { return m_block != other.m_block || m_cell != other.m_cell; }
CollectorHeapIterator<heapType>& operator++();
bool operator!=(const CollectorHeapIterator& other);
JSCell* operator*() const;
private:
typedef typename HeapConstants<heapType>::Block Block;
typedef typename HeapConstants<heapType>::Cell Cell;
Block** m_block;
Block** m_endBlock;
Cell* m_cell;
Cell* m_endCell;
protected:
CollectorHeapIterator(CollectorHeap&, size_t startBlock, size_t startCell);
void advance(size_t cellsPerBlock);
CollectorHeap& m_heap;
size_t m_block;
size_t m_cell;
};
template <HeapType heapType>
class LiveObjectIterator : public CollectorHeapIterator {
public:
LiveObjectIterator(CollectorHeap&, size_t startBlock, size_t startCell = 0);
LiveObjectIterator<heapType>& operator++();
};
template <HeapType heapType>
class DeadObjectIterator : public CollectorHeapIterator {
public:
DeadObjectIterator(CollectorHeap&, size_t startBlock, size_t startCell = 0);
DeadObjectIterator<heapType>& operator++();
};
template <HeapType heapType>
class ObjectIterator : public CollectorHeapIterator {
public:
ObjectIterator(CollectorHeap&, size_t startBlock, size_t startCell = 0);
ObjectIterator<heapType>& operator++();
};
template <HeapType heapType>
CollectorHeapIterator<heapType>::CollectorHeapIterator(CollectorBlock** block, CollectorBlock** endBlock)
: m_block(reinterpret_cast<Block**>(block))
, m_endBlock(reinterpret_cast<Block**>(endBlock))
, m_cell(m_block == m_endBlock ? 0 : (*m_block)->cells)
, m_endCell(m_block == m_endBlock ? 0 : (*m_block)->cells + HeapConstants<heapType>::cellsPerBlock)
inline CollectorHeapIterator::CollectorHeapIterator(CollectorHeap& heap, size_t startBlock, size_t startCell)
: m_heap(heap)
, m_block(startBlock)
, m_cell(startCell)
{
}
inline bool CollectorHeapIterator::operator!=(const CollectorHeapIterator& other)
{
return m_block != other.m_block || m_cell != other.m_cell;
}
inline JSCell* CollectorHeapIterator::operator*() const
{
return reinterpret_cast<JSCell*>(m_heap.blocks[m_block]->cells + m_cell);
}
inline void CollectorHeapIterator::advance(size_t cellsPerBlock)
{
++m_cell;
if (m_cell == cellsPerBlock) {
m_cell = 0;
++m_block;
}
}
template <HeapType heapType>
inline LiveObjectIterator<heapType>::LiveObjectIterator(CollectorHeap& heap, size_t startBlock, size_t startCell)
: CollectorHeapIterator(heap, startBlock, startCell - 1)
{
++(*this);
}
template <HeapType heapType>
inline LiveObjectIterator<heapType>& LiveObjectIterator<heapType>::operator++()
{
if (m_block < m_heap.nextBlock || m_cell < m_heap.nextCell) {
advance(HeapConstants<heapType>::cellsPerBlock);
return *this;
}
do {
advance(HeapConstants<heapType>::cellsPerBlock);
} while (m_block < m_heap.usedBlocks && !m_heap.blocks[m_block]->marked.get(m_cell));
return *this;
}
template <HeapType heapType>
inline DeadObjectIterator<heapType>::DeadObjectIterator(CollectorHeap& heap, size_t startBlock, size_t startCell)
: CollectorHeapIterator(heap, startBlock, startCell - 1)
{
if (m_cell && m_cell->u.freeCell.zeroIfFree == 0)
++*this;
++(*this);
}
template <HeapType heapType>
CollectorHeapIterator<heapType>& CollectorHeapIterator<heapType>::operator++()
template <HeapType heapType>
inline DeadObjectIterator<heapType>& DeadObjectIterator<heapType>::operator++()
{
do {
for (++m_cell; m_cell != m_endCell; ++m_cell)
if (m_cell->u.freeCell.zeroIfFree != 0) {
return *this;
}
if (++m_block != m_endBlock) {
m_cell = (*m_block)->cells;
m_endCell = (*m_block)->cells + HeapConstants<heapType>::cellsPerBlock;