JS heap out of memory issue

[reshadi]

while importing a large vbullletin forum, during private message import of 1.3M records, the process crashed with the following error. It seems a very huge JS Map is maintained in memory that does run out of JS heap memory.

nodebb_1 | FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
nodebb_1 |
nodebb_1 | <--- Last few GCs --->
nodebb_1 |
nodebb_1 | [127:0x416d080] 7139062 ms: Mark-sweep 1374.1 (1459.7) -> 1374.1 (1459.7) MB, 418.4 / 0.0 ms allocation failure GC in old space requested
nodebb_1 | [127:0x416d080] 7139615 ms: Mark-sweep 1374.1 (1459.7) -> 1374.1 (1427.2) MB, 502.4 / 0.0 ms last resort GC in old space requested
nodebb_1 | [127:0x416d080] 7140123 ms: Mark-sweep 1374.1 (1427.2) -> 1374.1 (1426.7) MB, 507.3 / 0.0 ms last resort GC in old space requested
nodebb_1 |
nodebb_1 |
nodebb_1 | <--- JS stacktrace --->
nodebb_1 |
nodebb_1 | ==== JS stack trace =========================================
nodebb_1 |
nodebb_1 | Security context: 0x264858f258b9
nodebb_1 | 1: deserializeObject(aka deserializeObject) [/usr/src/app/node_modules/bson/lib/bson/parser/deserializer.js:~41] [pc=0x16448d868f1e](this=0x35da052822d1 ,buffer=0xeda1ce5dde9 ,index=0,options=0xeda1ce5ddb9 ,isArray=0x35da052822d1 )
nodebb_1 | 2: deserialize [/usr/src/app/node_modules/bson/lib/bson/bson.js:~133] [pc=0x...
nodebb_1 |
nodebb_1 | 1: node::Abort() [/usr/local/bin/node]
nodebb_1 | 2: 0x8ccf9c [/usr/local/bin/node]
nodebb_1 | 3: v8::Utils::ReportOOMFailure(char const*, bool) [/usr/local/bin/node]
nodebb_1 | 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [/usr/local/bin/node]
nodebb_1 | 5: v8::internal::Factory::NewTransitionArray(int) [/usr/local/bin/node]
nodebb_1 | 6: v8::internal::TransitionsAccessor::Insert(v8::internal::Handlev8::internal::Name, v8::internal::Handlev8::internal::Map, v8::internal::SimpleTransitionFlag) [/usr/local/bin/node]
nodebb_1 | 7: v8::internal::Map::ConnectTransition(v8::internal::Handlev8::internal::Map, v8::internal::Handlev8::internal::Map, v8::internal::Handlev8::internal::Name, v8::internal::SimpleTransitionFlag) [/usr/local/bin/node]
nodebb_1 | 8: v8::internal::Map::CopyReplaceDescriptors(v8::internal::Handlev8::internal::Map, v8::internal::Handlev8::internal::DescriptorArray, v8::internal::Handlev8::internal::LayoutDescriptor, v8::internal::TransitionFlag, v8::internal::MaybeHandlev8::internal::Name, char const*, v8::internal::SimpleTransitionFlag) [/usr/local/bin/node]
nodebb_1 | 9: v8::internal::Map::CopyAddDescriptor(v8::internal::Handlev8::internal::Map, v8::internal::Descriptor*, v8::internal::TransitionFlag) [/usr/local/bin/node]
nodebb_1 | 10: v8::internal::Map::CopyWithField(v8::internal::Handlev8::internal::Map, v8::internal::Handlev8::internal::Name, v8::internal::Handlev8::internal::FieldType, v8::internal::PropertyAttributes, v8::internal::PropertyConstness, v8::internal::Representation, v8::internal::TransitionFlag) [/usr/local/bin/node]
nodebb_1 | 11: v8::internal::Map::TransitionToDataProperty(v8::internal::Handlev8::internal::Map, v8::internal::Handlev8::internal::Name, v8::internal::Handlev8::internal::Object, v8::internal::PropertyAttributes, v8::internal::PropertyConstness, v8::internal::Object::StoreFromKeyed, bool*) [/usr/local/bin/node]
nodebb_1 | 12: v8::internal::LookupIterator::PrepareTransitionToDataProperty(v8::internal::Handlev8::internal::JSObject, v8::internal::Handlev8::internal::Object, v8::internal::PropertyAttributes, v8::internal::Object::StoreFromKeyed) [/usr/local/bin/node]
nodebb_1 | 13: v8::internal::StoreIC::LookupForWrite(v8::internal::LookupIterator*, v8::internal::Handlev8::internal::Object, v8::internal::Object::StoreFromKeyed) [/usr/local/bin/node]
nodebb_1 | 14: v8::internal::StoreIC::UpdateCaches(v8::internal::LookupIterator*, v8::internal::Handlev8::internal::Object, v8::internal::Object::StoreFromKeyed, v8::internal::MaybeHandlev8::internal::Object) [/usr/local/bin/node]
nodebb_1 | 15: v8::internal::StoreIC::Store(v8::internal::Handlev8::internal::Object, v8::internal::Handlev8::internal::Name, v8::internal::Handlev8::internal::Object, v8::internal::Object::StoreFromKeyed) [/usr/local/bin/node]
nodebb_1 | 16: v8::internal::KeyedStoreIC::Store(v8::internal::Handlev8::internal::Object, v8::internal::Handlev8::internal::Object, v8::internal::Handlev8::internal::Object) [/usr/local/bin/node]
nodebb_1 | 17: v8::internal::Runtime_KeyedStoreIC_Miss(int, v8::internal::Object**, v8::internal::Isolate*) [/usr/local/bin/node]
nodebb_1 | 18: 0x16448d7042fd