Info

After unclean shutdown my WiredTiger.wt file is corrupted. When I'm try to --repair I got this stacktrace: 2017-06-19T19:32:57.073+0300 I CONTROL [initandlisten] options: { repair: true, storage: { dbPath: ".", engine: "wiredTiger" } } 2017-06-19T19:32:57.095+0300 W - [initandlisten] Detected unclean shutdown - ./mongod.lock is not empty. 2017-06-19T19:32:57.095+0300 W STORAGE [initandlisten] Recovering data from the last clean checkpoint. 2017-06-19T19:32:57.095+0300 I STORAGE [initandlisten] Detected WT journal files. Running recovery from last checkpoint. 2017-06-19T19:32:57.095+0300 I STORAGE [initandlisten] journal to nojournal transition config: create,cache_size=1G,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0), 2017-06-19T19:32:57.102+0300 E STORAGE [initandlisten] WiredTiger (0) [1497889977:102005][2433:0x7f1b31cc0c80], file:WiredTiger.wt, connection: read checksum error for 4096B block at offset 12288: block header checksum of 3831956594 doesn't match expected checksum of 861916107 2017-06-19T19:32:57.102+0300 E STORAGE [initandlisten] WiredTiger (0) [1497889977:102056][2433:0x7f1b31cc0c80], file:WiredTiger.wt, connection: WiredTiger.wt: encountered an illegal file format or internal value 2017-06-19T19:32:57.102+0300 E STORAGE [initandlisten] WiredTiger (-31804) [1497889977:102066][2433:0x7f1b31cc0c80], file:WiredTiger.wt, connection: the process must exit and restart: WT_PANIC: WiredTiger library panic 2017-06-19T19:32:57.102+0300 I - [initandlisten] Fatal Assertion 28558 2017-06-19T19:32:57.102+0300 I - [initandlisten] ***aborting after fassert() failure 2017-06-19T19:32:57.118+0300 F - [initandlisten] Got signal: 6 (Aborted). 0x15147a2 0x1513749 0x1513fb2 0x7f1b308c6390 0x7f1b30521428 0x7f1b3052302a 0x149c0d3 0x1241992 0x1c23417 0x1c2384e 0x1c23c54 0x1b550af 0x1b5561b 0x1b52558 0x1b56617 0x1b73608 0x1ba75d0 0x1c2201b 0x1c225b0 0x1c2272c 0x1bb6278 0x1c1f258 0x1be94ef 0x1be95ac 0x1ba42cc 0x1226eb0 0x1220495 0x111da78 0x9be8c0 0x974322 0x7f1b3050c830 0x9b8bf9 ----- BEGIN BACKTRACE ----- {"backtrace":[{"b":"400000","o":"11147A2","s":"_ZN5mongo15printStackTraceERSo"},{"b":"400000","o":"1113749"},{"b":"400000","o":"1113FB2"},{"b":"7F1B308B5000","o":"11390"},{"b":"7F1B304EC000","o":"35428","s":"gsignal"},{"b":"7F1B304EC000","o":"3702A","s":"abort"},{"b":"400000","o":"109C0D3","s":"_ZN5mongo13fassertFailedEi"},{"b":"400000","o":"E41992"},{"b":"400000","o":"1823417","s":"__wt_eventv"},{"b":"400000","o":"182384E","s":"__wt_err"},{"b":"400000","o":"1823C54","s":"__wt_panic"},{"b":"400000","o":"17550AF","s":"__wt_block_extlist_read"},{"b":"400000","o":"175561B","s":"__wt_block_extlist_read_avail"},{"b":"400000","o":"1752558","s":"__wt_block_checkpoint_load"},{"b":"400000","o":"1756617"},{"b":"400000","o":"1773608","s":"__wt_btree_open"},{"b":"400000","o":"17A75D0","s":"__wt_conn_btree_open"},{"b":"400000","o":"182201B","s":"__wt_session_get_btree"},{"b":"400000","o":"18225B0","s":"__wt_session_get_btree"},{"b":"400000","o":"182272C","s":"__wt_session_get_btree_ckpt"},{"b":"400000","o":"17B6278","s":"__wt_curfile_open"},{"b":"400000","o":"181F258"},{"b":"400000","o":"17E94EF","s":"__wt_metadata_cursor_open"},{"b":"400000","o":"17E95AC","s":"__wt_metadata_cursor"},{"b":"400000","o":"17A42CC","s":"wiredtiger_open"},{"b":"400000","o":"E26EB0","s":"_ZN5mongo18WiredTigerKVEngineC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_S8_mbbb"},{"b":"400000","o":"E20495"},{"b":"400000","o":"D1DA78","s":"_ZN5mongo20ServiceContextMongoD29initializeGlobalStorageEngineEv"},{"b":"400000","o":"5BE8C0","s":"_ZN5mongo13initAndListenEi"},{"b":"400000","o":"574322","s":"main"},{"b":"7F1B304EC000","o":"20830","s":"__libc_start_main"},{"b":"400000","o":"5B8BF9","s":"_start"}],"processInfo":{ "mongodbVersion" : "3.2.7", "gitVersion" : "4249c1d2b5999ebbf1fdf3bc0e0e3b3ff5c0aaf2", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "4.4.0-79-generic", "version" : "#100-Ubuntu SMP Wed May 17 19:58:14 UTC 2017", "machine" : "x86_64" }, "somap" : [ { "elfType" : 2, "b" : "400000", "buildId" : "B025CB0CF792AB652E442FF480378B48601044BE" }, { "b" : "7FFFFD1F1000", "elfType" : 3, "buildId" : "9446C0DD55430C6F14109300BF1799D05C90D64D" }, { "b" : "7F1B31841000", "path" : "/lib/x86_64-linux-gnu/libssl.so.1.0.0", "elfType" : 3, "buildId" : "7F514146540382F59AD705BA8C913A75204C6858" }, { "b" : "7F1B313FD000", "path" : "/lib/x86_64-linux-gnu/libcrypto.so.1.0.0", "elfType" : 3, "buildId" : "E6D4D2E4A048992CD5501E5985094E6CEC6C5D4F" }, { "b" : "7F1B311F5000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "0DBB8C21FC5D977098CA718BA2BFD6C4C21172E9" }, { "b" : "7F1B30FF1000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "C0C5B7F18348654040534B050B110D32A19EA38D" }, { "b" : "7F1B30CE8000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "05451CB4D66C321691F64F253880B7CE5B8812A6" }, { "b" : "7F1B30AD2000", "path" : "/lib/x86_64-linux-gnu/libgcc_s.so.1", "elfType" : 3, "buildId" : "68220AE2C65D65C1B6AAA12FA6765A6EC2F5F434" }, { "b" : "7F1B308B5000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "84538E3C6CFCD5D4E3C0D2B6C3373F802915A498" }, { "b" : "7F1B304EC000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "CBFA941A8EB7A11E4F90E81B66FCD5A820995D7C" }, { "b" : "7F1B31AAA000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "A7D5A820B802049276B1FC26C8E845A3E194EB6B" } ] }} mongod(_ZN5mongo15printStackTraceERSo+0x32) [0x15147a2] mongod(+0x1113749) [0x1513749] mongod(+0x1113FB2) [0x1513fb2] libpthread.so.0(+0x11390) [0x7f1b308c6390] libc.so.6(gsignal+0x38) [0x7f1b30521428] libc.so.6(abort+0x16A) [0x7f1b3052302a] mongod(_ZN5mongo13fassertFailedEi+0x93) [0x149c0d3] mongod(+0xE41992) [0x1241992] mongod(__wt_eventv+0x4F7) [0x1c23417] mongod(__wt_err+0x8E) [0x1c2384e] mongod(__wt_panic+0x24) [0x1c23c54] mongod(__wt_block_extlist_read+0x6F) [0x1b550af] mongod(__wt_block_extlist_read_avail+0x2B) [0x1b5561b] mongod(__wt_block_checkpoint_load+0x1F8) [0x1b52558] mongod(+0x1756617) [0x1b56617] mongod(__wt_btree_open+0xBB8) [0x1b73608] mongod(__wt_conn_btree_open+0x140) [0x1ba75d0] mongod(__wt_session_get_btree+0xEB) [0x1c2201b] mongod(__wt_session_get_btree+0x680) [0x1c225b0] mongod(__wt_session_get_btree_ckpt+0x12C) [0x1c2272c] mongod(__wt_curfile_open+0x218) [0x1bb6278] mongod(+0x181F258) [0x1c1f258] mongod(__wt_metadata_cursor_open+0x5F) [0x1be94ef] mongod(__wt_metadata_cursor+0x3C) [0x1be95ac] mongod(wiredtiger_open+0x188C) [0x1ba42cc] mongod(_ZN5mongo18WiredTigerKVEngineC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_S8_mbbb+0x820) [0x1226eb0] mongod(+0xE20495) [0x1220495] mongod(_ZN5mongo20ServiceContextMongoD29initializeGlobalStorageEngineEv+0x3E8) [0x111da78] mongod(_ZN5mongo13initAndListenEi+0x480) [0x9be8c0] mongod(main+0x752) [0x974322] libc.so.6(__libc_start_main+0xF0) [0x7f1b3050c830] mongod(_start+0x29) [0x9b8bf9] ----- END BACKTRACE ----- My mongodb version: v3.2.7 (4249c1d2b5999ebbf1fdf3bc0e0e3b3ff5c0aaf2) I'm attached WiredTiger files. Please, could you help?

Top User Comments

a.maslov commented on Tue, 20 Jun 2017 08:04:57 +0000: Hi Thomas! All works perfectly, thank you! thomas.schubert commented on Mon, 19 Jun 2017 17:46:28 +0000: Hi a.maslov, Thank you for the report. I've attached a repair attempt of the files you've provided. Would you please extract these files and replace them in your $dbpath and let us know if it resolves the issue? If you are still seeing errors after replacing these files, please provide the complete logs from mongod so that we can further investigate. Kind regards, Thomas

Steps to Reproduce