OPERATIONAL DEFECT DATABASE
...
...
While doing some testing on replica set's I've found an odd behaviour. mongod will hang indefinitely (and only reacts to kill -9) if one tries to db.shutdownServer() while having db.fsyncLock() active. After a kill -9 on the mongod process, the MongoDB shell throws the following backtrace: Unable to find source-code formatter for language: shell. Available languages are: actionscript, ada, applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html, java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r, rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml nair(mongod-3.0.3)[SECONDARY] admin> db.adminCommand( {shutdown:1}) ^C2015-07-02T15:18:50.646+0200 I NETWORK Socket recv() errno:54 Connection reset by peer 127.0.0.1:30002 2015-07-02T15:18:50.648+0200 I NETWORK SocketException: remote: 127.0.0.1:30002 error: 9001 socket exception [RECV_ERROR] server [127.0.0.1:30002] 2015-07-02T15:18:50.648+0200 I NETWORK DBClientCursor::init call() failed 2015-07-02T15:18:50.648+0200 F - terminate() called. An exception is active; attempting to gather more information 2015-07-02T15:18:50.667+0200 F - DBException::toString(): 10276 DBClientBase::findN: transport error: 127.0.0.1:30002 ns: admin.$cmd.sys.inprog query: {} Actual exception type: mongo::UserException 0x10446c30a 0x10446ba02 0x7fff89cc81d1 0x7fff89cc7c5b 0x10440a228 0x10440a02a 0x1042f3672 0x1042f3770 0x1043fd199 0x1042a718d 0x1042a71d9 0x7fff86bc95aa 0x7fb10d81fd10 0x104425f4d 0x104426d02 0x1042fa8e6 0x10430de93 0x1042f5903 0x1042fc444 0x1043cd823 0x1043bfee2 0x3181f0289159 ----- BEGIN BACKTRACE ----- {"backtrace":[{"b":"1042A5000","o":"1C730A"},{"b":"1042A5000","o":"1C6A02"},{"b":"7FFF89CA4000","o":"241D1"},{"b":"7FFF89CA4000","o":"23C5B"},{"b":"1042A5000","o":"165228"},{"b":"1042A5000","o":"16502A"},{"b":"1042A5000","o":"4E672"},{"b":"1042A5000","o":"4E770"},{"b":"1042A5000","o":"158199"},{"b":"1042A5000","o":"218D"},{"b":"1042A5000","o":"21D9"},{"b":"7FFF86BC6000","o":"35AA"},{"b":"0","o":"7FB10D81FD10"},{"b":"1042A5000","o":"180F4D"},{"b":"1042A5000","o":"181D02"},{"b":"1042A5000","o":"558E6"},{"b":"1042A5000","o":"68E93"},{"b":"1042A5000","o":"50903"},{"b":"1042A5000","o":"57444"},{"b":"1042A5000","o":"128823"},{"b":"1042A5000","o":"11AEE2"},{"b":"0","o":"3181F0289159"}],"processInfo":{ "mongodbVersion" : "3.0.3", "gitVersion" : "b40106b36eecd1b4407eb1ad1af6bc60593c6105", "uname" : { "sysname" : "Darwin", "release" : "13.4.0", "version" : "Darwin Kernel Version 13.4.0: Wed Mar 18 16:20:14 PDT 2015; root:xnu-2422.115.14~1/RELEASE_X86_64", "machine" : "x86_64" }, "somap" : [ { "path" : "/Users/norberto/bin/mongo", "machType" : 2, "b" : "1042A5000", "buildId" : "7CBFA011798A3C2F92CB8C2332A3206B" }, { "path" : "/usr/lib/libSystem.B.dylib", "machType" : 6, "b" : "7FFF8EF74000", "buildId" : "E303F2F8A8CF3DF384B3F2D0EE41CCF6" }, { "path" : "/usr/lib/libc++.1.dylib", "machType" : 6, "b" : "7FFF86440000", "buildId" : "4F68DFC5207739A8A449CAC5FDEE7BDE" }, { "path" : "/usr/lib/system/libcache.dylib", "machType" : 6, "b" : "7FFF8ACAD000", "buildId" : "BDC1E65B72A13DA3A57CB23159CAAD0B" }, { "path" : "/usr/lib/system/libcommonCrypto.dylib", "machType" : 6, "b" : "7FFF8B712000", "buildId" : "8C4F0CA0389C3EDCB155E62DD2187E1D" }, { "path" : "/usr/lib/system/libcompiler_rt.dylib", "machType" : 6, "b" : "7FFF84C2C000", "buildId" : "4CD916B21B17362AB403EF24A1DAC141" }, { "path" : "/usr/lib/system/libcopyfile.dylib", "machType" : 6, "b" : "7FFF8ED55000", "buildId" : "CF29DFF605893590834C82E2316612E8" }, { "path" : "/usr/lib/system/libcorecrypto.dylib", "machType" : 6, "b" : "7FFF87F80000", "buildId" : "F3973C2814B63006BB2B00DD7F09ABC7" }, { "path" : "/usr/lib/system/libdispatch.dylib", "machType" : 6, "b" : "7FFF84CD5000", "buildId" : "C4E4A18D3C3B3C9C8709A4270D998DE7" }, { "path" : "/usr/lib/system/libdyld.dylib", "machType" : 6, "b" : "7FFF859BF000", "buildId" : "7C9EC3B7DDE333FF953F4067C743951D" }, { "path" : "/usr/lib/system/libkeymgr.dylib", "machType" : 6, "b" : "7FFF84CF0000", "buildId" : "3AA8D85DCF003BD3A5A0E28E1A32A6D8" }, { "path" : "/usr/lib/system/liblaunch.dylib", "machType" : 6, "b" : "7FFF8A7C4000", "buildId" : "A40A0C7B321639B48AE0B5D3BAF1DA8A" }, { "path" : "/usr/lib/system/libmacho.dylib", "machType" : 6, "b" : "7FFF8CD36000", "buildId" : "1D2910DFC0363A82A3FD44FF73B5FF9B" }, { "path" : "/usr/lib/system/libquarantine.dylib", "machType" : 6, "b" : "7FFF83F5A000", "buildId" : "7A1A2BCBC03D3A25BFA43E569B2D2C38" }, { "path" : "/usr/lib/system/libremovefile.dylib", "machType" : 6, "b" : "7FFF89C3C000", "buildId" : "3543F917928E3DB2A2F47AB73B4970EF" }, { "path" : "/usr/lib/system/libsystem_asl.dylib", "machType" : 6, "b" : "7FFF91059000", "buildId" : "655FB34352CF3E2FB14DBEBF5AAEF94D" }, { "path" : "/usr/lib/system/libsystem_blocks.dylib", "machType" : 6, "b" : "7FFF8BCC4000", "buildId" : "FB856CD12AEA39078E9B1E54B6827F82" }, { "path" : "/usr/lib/system/libsystem_c.dylib", "machType" : 6, "b" : "7FFF85AAB000", "buildId" : "6FD3A4004BB23B95B90CBE6E9D0D78FA" }, { "path" : "/usr/lib/system/libsystem_configuration.dylib", "machType" : 6, "b" : "7FFF8B5C1000", "buildId" : "4998CB6A9D54390A9F575D1AC53C135C" }, { "path" : "/usr/lib/system/libsystem_dnssd.dylib", "machType" : 6, "b" : "7FFF8FB1F000", "buildId" : "17B03FFD92C532829981EBB28B456207" }, { "path" : "/usr/lib/system/libsystem_info.dylib", "machType" : 6, "b" : "7FFF86935000", "buildId" : "7D41A156D2853849A2C3C04ADE797D98" }, { "path" : "/usr/lib/system/libsystem_kernel.dylib", "machType" : 6, "b" : "7FFF85A8E000", "buildId" : "8116098DB3F13E50A934576DD6369234" }, { "path" : "/usr/lib/system/libsystem_m.dylib", "machType" : 6, "b" : "7FFF8807E000", "buildId" : "B7F0E2E4277733FCA787D6430B630D54" }, { "path" : "/usr/lib/system/libsystem_malloc.dylib", "machType" : 6, "b" : "7FFF85975000", "buildId" : "A695B4E438E9332EA77229D31E3F1385" }, { "path" : "/usr/lib/system/libsystem_network.dylib", "machType" : 6, "b" : "7FFF86FA4000", "buildId" : "0D630D53C7723EC58257EFB0ACCE3153" }, { "path" : "/usr/lib/system/libsystem_notify.dylib", "machType" : 6, "b" : "7FFF8FDE2000", "buildId" : "9B34B4FEF5AD3F09A5F046AFF3571323" }, { "path" : "/usr/lib/system/libsystem_platform.dylib", "machType" : 6, "b" : "7FFF86BC6000", "buildId" : "3C3D3DA832B9324398ECD89B9A1670B3" }, { "path" : "/usr/lib/system/libsystem_pthread.dylib", "machType" : 6, "b" : "7FFF91808000", "buildId" : "AB498556B555310E9041F67EC9E00E2C" }, { "path" : "/usr/lib/system/libsystem_sandbox.dylib", "machType" : 6, "b" : "7FFF83E02000", "buildId" : "0C93EB2373643670B511212A7A524695" }, { "path" : "/usr/lib/system/libsystem_stats.dylib", "machType" : 6, "b" : "7FFF8C46A000", "buildId" : "4E51D5B092A03D0DB90E495A1ED3E391" }, { "path" : "/usr/lib/system/libunc.dylib", "machType" : 6, "b" : "7FFF88C77000", "buildId" : "62682455186236FE8A047A6B91256438" }, { "path" : "/usr/lib/system/libunwind.dylib", "machType" : 6, "b" : "7FFF8970C000", "buildId" : "78DCC3582FC1302EB3950155B47CB547" }, { "path" : "/usr/lib/system/libxpc.dylib", "machType" : 6, "b" : "7FFF859C3000", "buildId" : "AB40CD57F4543FD4B41563B3C0D5C624" }, { "path" : "/usr/lib/libobjc.A.dylib", "machType" : 6, "b" : "7FFF84A7E000", "buildId" : "AD7FD984271E30F4A3616B20319EC73B" }, { "path" : "/usr/lib/libauto.dylib", "machType" : 6, "b" : "7FFF8EED1000", "buildId" : "F45C36E8B6063886B5B1B6745E757CA8" }, { "path" : "/usr/lib/libc++abi.dylib", "machType" : 6, "b" : "7FFF89CA4000", "buildId" : "21A807D367323455B77F743E9F916DF0" }, { "path" : "/usr/lib/libDiagnosticMessagesClient.dylib", "machType" : 6, "b" : "7FFF892C3000", "buildId" : "4CDB0F7BC0AF3424BC39495696F0DB1E" } ] }} mongo(_ZN5mongo15printStackTraceERNSt3__113basic_ostreamIcNS0_11char_traitsIcEEEE+0x3A) [0x10446c30a] mongo(_ZN5mongo12_GLOBAL__N_111myTerminateEv+0x282) [0x10446ba02] libc++abi.dylib(_ZSt11__terminatePFvvE+0x8) [0x7fff89cc81d1] libc++abi.dylib(_ZN10__cxxabiv1L22exception_cleanup_funcE19_Unwind_Reason_CodeP17_Unwind_Exception+0x0) [0x7fff89cc7c5b] mongo(_ZN5mongo9uassertedEiPKc+0x1F8) [0x10440a228] mongo(_ZN5mongo9uassertedEiRKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEE+0x1A) [0x10440a02a] mongo(_ZN5mongo17DBClientInterface5findNERNSt3__16vectorINS_7BSONObjENS1_9allocatorIS3_EEEERKNS1_12basic_stringIcNS1_11char_traitsIcEENS4_IcEEEENS_5QueryEiiPKS3_i+0x632) [0x1042f3672] mongo(_ZN5mongo17DBClientInterface7findOneERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEERKNS_5QueryEPKNS_7BSONObjEi+0x60) [0x1042f3770] mongo(_ZNK5mongo11shell_utils18ConnectionRegistry30killOperationsOnAllConnectionsEb+0x1D9) [0x1043fd199] mongo(_ZN5mongo11exitCleanlyENS_8ExitCodeE+0x6D) [0x1042a718d] mongo(_Z10quitNicelyi+0x9) [0x1042a71d9] libsystem_platform.dylib(_sigtramp+0x1A) [0x7fff86bc95aa] ??? [0x7fb10d81fd10] mongo(_ZN5mongo13MessagingPort4recvERNS_7MessageE+0x7D) [0x104425f4d] mongo(_ZN5mongo13MessagingPort4recvERKNS_7MessageERS1_+0x22) [0x104426d02] mongo(_ZN5mongo18DBClientConnection4callERNS_7MessageES2_bPNSt3__112basic_stringIcNS3_11char_traitsIcEENS3_9allocatorIcEEEE+0x46) [0x1042fa8e6] mongo(_ZN5mongo14DBClientCursor4initEv+0x93) [0x10430de93] mongo(_ZN5mongo12DBClientBase5queryERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEENS_5QueryEiiPKNS_7BSONObjEii+0xA3) [0x1042f5903] mongo(_ZN5mongo18DBClientConnection5queryERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEENS_5QueryEiiPKNS_7BSONObjEii+0x84) [0x1042fc444] mongo(_ZN5mongo9mongoFindEPNS_7V8ScopeERKN2v89ArgumentsE+0x2C3) [0x1043cd823] mongo(_ZN5mongo7V8Scope10v8CallbackERKN2v89ArgumentsE+0x52) [0x1043bfee2] ??? [0x3181f0289159] ----- END BACKTRACE -----
ramon.fernandez commented on Thu, 2 Jul 2015 15:25:30 +0000: Hey norberto@10gen.com, this was reported earlier in SERVER-17589, so I'm closing this ticket as a duplicate. Regards, Ramón.
boot up mongod Unable to find source-code formatter for language: shell. Available languages are: actionscript, ada, applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html, java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r, rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml db.fsyncLock() db.adminCommand( {shutdown:1}) system hangs
MongoDB Integration
Learn more about where this data comes from
Bug Scrub Advisor
Streamline upgrades with automated vendor bug scrubs
BugZero Enterprise
Wish you caught this bug sooner? Get proactive today.
