Info

2017-01-12T06:21:26.938+0100 I CONTROL [initandlisten] MongoDB starting : pid=31896 port=27017 dbpath=/var/lib/mongodb 64-bit host=vps306768 2017-01-12T06:21:26.938+0100 I CONTROL [initandlisten] db version v3.2.9 2017-01-12T06:21:26.938+0100 I CONTROL [initandlisten] git version: 22ec9e93b40c85fc7cae7d56e7d6a02fd811088c 2017-01-12T06:21:26.938+0100 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.2g-fips 1 Mar 2016 2017-01-12T06:21:26.938+0100 I CONTROL [initandlisten] allocator: tcmalloc 2017-01-12T06:21:26.938+0100 I CONTROL [initandlisten] modules: none 2017-01-12T06:21:26.938+0100 I CONTROL [initandlisten] build environment: 2017-01-12T06:21:26.938+0100 I CONTROL [initandlisten] distmod: ubuntu1604 2017-01-12T06:21:26.938+0100 I CONTROL [initandlisten] distarch: x86_64 2017-01-12T06:21:26.938+0100 I CONTROL [initandlisten] target_arch: x86_64 2017-01-12T06:21:26.938+0100 I CONTROL [initandlisten] options: { repair: true, storage: { dbPath: "/var/lib/mongodb", engine: "wiredTiger" } } 2017-01-12T06:21:26.966+0100 I STORAGE [initandlisten] Detected WT journal files. Running recovery from last checkpoint. 2017-01-12T06:21:26.966+0100 I STORAGE [initandlisten] journal to nojournal transition config: create,cache_size=3G,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0), 2017-01-12T06:21:26.986+0100 E STORAGE [initandlisten] WiredTiger (0) [1484198486:986082][31896:0x7f0049b4ec80], file:WiredTiger.wt, connection: read checksum error for 4096B block at offset 53248: block header checksum of 2750094563 doesn't match expected checksum of 1661035712 2017-01-12T06:21:26.986+0100 E STORAGE [initandlisten] WiredTiger (0) [1484198486:986144][31896:0x7f0049b4ec80], file:WiredTiger.wt, connection: WiredTiger.wt: encountered an illegal file format or internal value 2017-01-12T06:21:26.986+0100 E STORAGE [initandlisten] WiredTiger (-31804) [1484198486:986158][31896:0x7f0049b4ec80], file:WiredTiger.wt, connection: the process must exit and restart: WT_PANIC: WiredTiger library panic 2017-01-12T06:21:26.986+0100 I - [initandlisten] Fatal Assertion 28558 2017-01-12T06:21:26.986+0100 I - [initandlisten] ***aborting after fassert() failure 2017-01-12T06:21:27.005+0100 F - [initandlisten] Got signal: 6 (Aborted). 0x1521902 0x15208a9 0x1521112 0x7f004873d3d0 0x7f0048398418 0x7f004839a01a 0x14a3713 0x12480f2 0x1c371b7 0x1c375ae 0x1c379b4 0x1b6421f 0x1b6474b 0x1b613e8 0x1b65827 0x1b83008 0x1bb77d0 0x1c35fcb 0x1c36560 0x1c366dc 0x1bc6bb8 0x1c33208 0x1bfc56f 0x1bfc62c 0x1bb3f2a 0x122cbae 0x1225f85 0x1121898 0x9c0070 0x975802 0x7f0048383830 0x9ba699 ----- BEGIN BACKTRACE ----- {"backtrace":[{"b":"400000","o":"1121902","s":"_ZN5mongo15printStackTraceERSo"},{"b":"400000","o":"11208A9"},{"b":"400000","o":"1121112"},{"b":"7F004872C000","o":"113D0"},{"b":"7F0048363000","o":"35418","s":"gsignal"},{"b":"7F0048363000","o":"3701A","s":"abort"},{"b":"400000","o":"10A3713","s":"_ZN5mongo13fassertFailedEi"},{"b":"400000","o":"E480F2"},{"b":"400000","o":"18371B7","s":"__wt_eventv"},{"b":"400000","o":"18375AE","s":"__wt_err"},{"b":"400000","o":"18379B4","s":"__wt_panic"},{"b":"400000","o":"176421F","s":"__wt_block_extlist_read"},{"b":"400000","o":"176474B","s":"__wt_block_extlist_read_avail"},{"b":"400000","o":"17613E8","s":"__wt_block_checkpoint_load"},{"b":"400000","o":"1765827"},{"b":"400000","o":"1783008","s":"__wt_btree_open"},{"b":"400000","o":"17B77D0","s":"__wt_conn_btree_open"},{"b":"400000","o":"1835FCB","s":"__wt_session_get_btree"},{"b":"400000","o":"1836560","s":"__wt_session_get_btree"},{"b":"400000","o":"18366DC","s":"__wt_session_get_btree_ckpt"},{"b":"400000","o":"17C6BB8","s":"__wt_curfile_open"},{"b":"400000","o":"1833208"},{"b":"400000","o":"17FC56F","s":"__wt_metadata_cursor_open"},{"b":"400000","o":"17FC62C","s":"__wt_metadata_cursor"},{"b":"400000","o":"17B3F2A","s":"wiredtiger_open"},{"b":"400000","o":"E2CBAE","s":"_ZN5mongo18WiredTigerKVEngineC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_S8_mbbb"},{"b":"400000","o":"E25F85"},{"b":"400000","o":"D21898","s":"_ZN5mongo20ServiceContextMongoD29initializeGlobalStorageEngineEv"},{"b":"400000","o":"5C0070","s":"_ZN5mongo13initAndListenEi"},{"b":"400000","o":"575802","s":"main"},{"b":"7F0048363000","o":"20830","s":"__libc_start_main"},{"b":"400000","o":"5BA699","s":"_start"}],"processInfo":{ "mongodbVersion" : "3.2.9", "gitVersion" : "22ec9e93b40c85fc7cae7d56e7d6a02fd811088c", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "4.4.0-22-generic", "version" : "#40-Ubuntu SMP Thu May 12 22:03:46 UTC 2016", "machine" : "x86_64" }, "somap" : [ { "elfType" : 2, "b" : "400000", "buildId" : "1F47133C759EFF79F90A6EBF640DC97B744D70C1" }, { "b" : "7FFD695E0000", "elfType" : 3, "buildId" : "0BFAB3B46E2221A329F60F59D05CBFBED1FECA7C" }, { "b" : "7F00496CF000", "path" : "/lib/x86_64-linux-gnu/libssl.so.1.0.0", "elfType" : 3, "buildId" : "822754695E4B31AE82937258BDFF3D52EFA0BA36" }, { "b" : "7F0049274000", "path" : "/lib/x86_64-linux-gnu/libcrypto.so.1.0.0", "elfType" : 3, "buildId" : "B75A96C59BE1B5B54FBF1A91ED722BEC9406288E" }, { "b" : "7F004906C000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "A779DBCB3A477DC0C8D09B60FAC7335D396C19DF" }, { "b" : "7F0048E68000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "A2ADF3615338D49C702C41EB83A99AB743D2B574" }, { "b" : "7F0048B5F000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "5C4078C04888A418F3DB0868702ECFDB35B3AD8B" }, { "b" : "7F0048949000", "path" : "/lib/x86_64-linux-gnu/libgcc_s.so.1", "elfType" : 3, "buildId" : "68220AE2C65D65C1B6AAA12FA6765A6EC2F5F434" }, { "b" : "7F004872C000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "B77847CC9CACBCA3B5753D0D25A32E5795AFE75B" }, { "b" : "7F0048363000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "369DE0E1D833CAA693AF17F17C83BA937F0A4DAD" }, { "b" : "7F0049938000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "EDFA6D46E00CA97F349FDD3333D88493D442932C" } ] }} mongod(_ZN5mongo15printStackTraceERSo+0x32) [0x1521902] mongod(+0x11208A9) [0x15208a9] mongod(+0x1121112) [0x1521112] libpthread.so.0(+0x113D0) [0x7f004873d3d0] libc.so.6(gsignal+0x38) [0x7f0048398418] libc.so.6(abort+0x16A) [0x7f004839a01a] mongod(_ZN5mongo13fassertFailedEi+0x93) [0x14a3713] mongod(+0xE480F2) [0x12480f2] mongod(__wt_eventv+0x4F7) [0x1c371b7] mongod(__wt_err+0x8E) [0x1c375ae] mongod(__wt_panic+0x24) [0x1c379b4] mongod(__wt_block_extlist_read+0x6F) [0x1b6421f] mongod(__wt_block_extlist_read_avail+0x2B) [0x1b6474b] mongod(__wt_block_checkpoint_load+0x1F8) [0x1b613e8] mongod(+0x1765827) [0x1b65827] mongod(__wt_btree_open+0xBA8) [0x1b83008] mongod(__wt_conn_btree_open+0x140) [0x1bb77d0] mongod(__wt_session_get_btree+0xEB) [0x1c35fcb] mongod(__wt_session_get_btree+0x680) [0x1c36560] mongod(__wt_session_get_btree_ckpt+0x12C) [0x1c366dc] mongod(__wt_curfile_open+0x218) [0x1bc6bb8] mongod(+0x1833208) [0x1c33208] mongod(__wt_metadata_cursor_open+0x5F) [0x1bfc56f] mongod(__wt_metadata_cursor+0x3C) [0x1bfc62c] mongod(wiredtiger_open+0x158A) [0x1bb3f2a] mongod(_ZN5mongo18WiredTigerKVEngineC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_S8_mbbb+0xBCE) [0x122cbae] mongod(+0xE25F85) [0x1225f85] mongod(_ZN5mongo20ServiceContextMongoD29initializeGlobalStorageEngineEv+0x3E8) [0x1121898] mongod(_ZN5mongo13initAndListenEi+0x480) [0x9c0070] mongod(main+0x712) [0x975802] libc.so.6(__libc_start_main+0xF0) [0x7f0048383830] mongod(_start+0x29) [0x9ba699] ----- END BACKTRACE ----- mongorepair.sh: line 5: 31896 Aborted mongod --dbpath /var/lib/mongodb --repair --storageEngine wiredTiger Mongo Repair Complete Starting Mongo Server I tried to revover data using http://www.alexbevi.com/blog/2016/02/10/recovering-a-wiredtiger-collection-from-a-corrupt-mongodb-installation/ But this is also giving me error while recovery Unable to find source-code formatter for language: shell. Available languages are: actionscript, ada, applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html, java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r, rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml root@vps306768:~/wiredtiger-2.9.1# ./wt -v -h ../backup/mongodb -C "extensions=[./ext/compressors/snappy/.libs/libwiredtiger_snappy.so]" -R salvage collection-8--2945606112955784657.wt [1484199812:302917][32152:0x7f757c003700], file:WiredTiger.wt, connection: read checksum error for 4096B block at offset 53248: block header checksum of 2750094563 doesn't match expected checksum of 1661035712 [1484199812:303046][32152:0x7f757c003700], file:WiredTiger.wt, connection: WiredTiger.wt: encountered an illegal file format or internal value [1484199812:303094][32152:0x7f757c003700], file:WiredTiger.wt, connection: the process must exit and restart: WT_PANIC: WiredTiger library panic lt-wt: WT_PANIC: WiredTiger library panic Attached : WiredTiger.wt and WiredTiger.turtle

Top User Comments

mark.agarunov commented on Thu, 12 Jan 2017 23:19:57 +0000: Hello saurabh73, I'm glad to hear the repaired files fixed your issue. Please note that SERVER project is for reporting bugs or feature suggestions for the MongoDB server. For MongoDB-related support discussion please post on the mongodb-user group or Stack Overflow with the mongodb tag. A question like this involving more discussion would be best posted on the mongodb-user group. Thanks, Mark saurabh73 commented on Thu, 12 Jan 2017 19:55:51 +0000: Just need one quick suggestion. My mongod service get's terminated by server instance repeatedly. Any suggestion how to fix that? saurabh73 commented on Thu, 12 Jan 2017 19:53:36 +0000: Thanks a lot for such a quick response. The given fix is working and I am able to restore my data with given fix. Thanks a ton again. mark.agarunov commented on Thu, 12 Jan 2017 18:57:21 +0000: Hello saurabh73, I've attached a repair attempt of the files you've provided. Would you please extract these files and replace them in your $dbpath and let us know if it resolves the issue? Please note that the method described in the linked blog post is not supported and may lead to further corruption in some circumstances. Thanks, Mark

Steps to Reproduce