Launching soon: The world's first vendor agnostic bug scrubLearn more & join waitlist
OPERATIONAL DEFECT DATABASE
...
...
We have mongodb installed on Unix (FreeBSD12.1) VM. When hard reboot occurs due to vSphere HA or host issue, mongodb is crashing. When we are doing repair it gives attached error. It is found that WiredTiger.turtle file is empty after crash, which causing problem in repair. How to solve this issue? Thanks and regards, Akash T
dmitry.agranat commented on Wed, 13 Jan 2021 08:59:18 +0000: Hi taloleakash@gmail.com, We haven’t heard back from you for some time, so I’m going to close this ticket. If this is still an issue for you, please provide additional information and we will reopen the ticket. Regards, Dima dmitry.agranat commented on Tue, 29 Dec 2020 11:35:56 +0000: taloleakash@gmail.com, can you try using --repair with the latest MongoDB version, which is 4.4.3 after significantly increasing your node memory? In the backtrace I've noticed this: _ZN5mongo29reportOutOfMemoryErrorAndExitEv JIRAUSER1257765 commented on Mon, 28 Dec 2020 13:38:31 +0000: Hi @dmitry.agranat MongoDB server version is 3.6.14. check mongo_recover_script.txtwhich i used for restoring backup. beforecrash_WiredTiger.turtleand after crashed it is becaming empty so unable to upload. crashed_mongodb_logs.txtsame logs are coming when i ran mongo repair command mongod --repair --dbpath /var/db/mongodb --port 9091 dmitry.agranat commented on Mon, 28 Dec 2020 12:11:30 +0000: Thanks taloleakash@gmail.com, did you try to --repair it with the latest MongoDB version? The attached screenshot does not show what version was used and the --repair log lines & output. In my previous comment, I've requested: The wiredTiger.wt and wiredTiger.turtle files (before a repair on these files was executed) But I could not find the wiredTiger.turtle after the crash. Please note that both files are needed after the crash but before your repair attempt. JIRAUSER1257765 commented on Wed, 23 Dec 2020 12:07:16 +0000: Hi Dmitry, This case is for standalone mongodb deployment. We have hourly backup of application configuration not whole database backup of high size. This is UFS filesystem. We are using local datastore of VMFS5 on vsphere. Check attached log files and mongodb files for before and after crash of mongodb. WiredTiger.turtle is empty after crash. aftercrash_sizeStorer.wtbeforecrash_sizeStorer.wtworking_mongodb_logs.txtcrashed_mongodb_logs.txtaftercrash_WiredTiger.wtbeforecrash_WiredTiger.wtaftercrash_WiredTigerLAS.wtbeforecrash_WiredTigerLAS.wt beforecrash_WiredTiger.turtle JIRAUSER1257765 commented on Mon, 21 Dec 2020 04:58:36 +0000: Hi Dmitry, I will share additional information today. dmitry.agranat commented on Sun, 20 Dec 2020 07:19:44 +0000: Hi taloleakash@gmail.com, We still need additional information to diagnose the problem. If this is still an issue for you, would you please provide the data mentioned in my last comment? Thanks, Dima dmitry.agranat commented on Tue, 8 Dec 2020 09:31:43 +0000: Hi taloleakash@gmail.com, The scenario you've described and the outcome should not have happened but we do not have enough information for the event. For example, you have not mentioned what filesystem is being used. MongoDB requires a filesystem that supports fsync() on directories. You can also try mongod --repair using the latest version of MongoDB. The ideal resolution is to perform a clean resync from an unaffected node. If the above fails, please provide: The logs for the affected node, including before, leading up to, and after the first sign of corruption. As much of syslog and dmesg content leading up to the first sign of corruption as possible. A description of the underlying storage mechanism in use, including details like: What file system and/or volume management system is in use? Is data storage locally attached or network-attached? Are disks RAIDed and if so how? Are disks SSDs or HDDs? A description of your backup method, if any. A description of your disks have been recently checked for integrity? A history of the deployment, including: a timeline of version changes a timeline of hardware upgrade/downgrade cycles or configuration changes a timeline of disaster recovery or backup restoration activities a timeline of any manipulations of the underlying database files, including copies or moves, and information about whether mongod was running during each manipulation. The wiredTiger.wt and wiredTiger.turtle files (before a repair on these files was executed) Thanks, Dima
Run mongodb on FreeBSD12.1 VM Hard reboot or poweroff VM from host start vm and check mongodb service try to repair WiredTiger.turtle not found error.