Launching soon: The world's first vendor agnostic bug scrubLearn more & join waitlist
OPERATIONAL DEFECT DATABASE
...
...
The WT bootstrap code double-opens the .turtle file, which causes validation exception and process termination in Windows debug builds. The second open happens at this line and the file has previously been opened here. With this problem debug builds on Windows are unusable and evergreen does not catch it because we build with --dbg=on --opt=off, which includes the release runtime. This is the call stack: > msvcr120.dll!_invoke_watson(const wchar_t * pszExpression=0x000001d20000000b, const wchar_t * pszFunction=0x00007ffcb1d67adf, const wchar_t * pszFile=0x0000001d00000000, unsigned int nLine=1722168464, unsigned __int64 pReserved=0) Line 132 C++ msvcr120.dll!_invalid_parameter(const wchar_t * pszExpression=0x0000000000000000, const wchar_t * pszFunction=0x0000000000000000, const wchar_t * pszFile=0x0000000000000000, unsigned int nLine=0, unsigned __int64 pReserved=0) Line 85 C++ msvcr120.dll!_invalid_parameter_noinfo() Line 97 C++ msvcr120.dll!_fdopen(int filedes, const char * mode) Line 55 C mongod.exe!__win_handle_open(__wt_session_impl * session=0x000001d2f3760000, __wt_fh * fh=0x000001d2f333f100, const char * name=0x000001d2f333d270, unsigned int file_type=16, unsigned int flags=72) Line 634 C mongod.exe!__wt_open(__wt_session_impl * session=0x000001d2f3760000, const char * name=0x00007ff755c0edb8, unsigned int file_type=16, unsigned int flags=72, __wt_fh * * fhp=0x0000001d9373c9b8) Line 204 C mongod.exe!__wt_turtle_read(__wt_session_impl * session=0x000001d2f3760000, const char * key=0x000001d2f33411c0, char * * valuep=0x0000001d9373cbc8) Line 261 C mongod.exe!__wt_metadata_search(__wt_session_impl * session=0x000001d2f3760000, const char * key=0x000001d2f33411c0, char * * valuep=0x0000001d9373cbc8) Line 275 C mongod.exe!__conn_btree_config_set(__wt_session_impl * session=0x000001d2f3760000) Line 250 C mongod.exe!__wt_conn_btree_open(__wt_session_impl * session=0x000001d2f3760000, const char * * cfg=0x0000001d9373cfc8, unsigned int flags=0) Line 318 C mongod.exe!__wt_session_get_btree(__wt_session_impl * session=0x000001d2f3760000, const char * uri=0x00007ff755c0aba0, const char * checkpoint=0x0000000000000000, const char * * cfg=0x0000001d9373cfc8, unsigned int flags=0) Line 529 C mongod.exe!__wt_session_get_btree(__wt_session_impl * session=0x000001d2f3760000, const char * uri=0x00007ff755c0aba0, const char * checkpoint=0x0000000000000000, const char * * cfg=0x0000001d9373cfc8, unsigned int flags=0) Line 522 C mongod.exe!__wt_session_get_btree_ckpt(__wt_session_impl * session=0x000001d2f3760000, const char * uri=0x00007ff755c0aba0, const char * * cfg=0x0000001d9373cfc8, unsigned int flags=0) Line 339 C mongod.exe!__wt_curfile_open(__wt_session_impl * session=0x000001d2f3760000, const char * uri=0x00007ff755c0aba0, __wt_cursor * owner=0x0000000000000000, const char * * cfg=0x0000001d9373cfc8, __wt_cursor * * cursorp=0x0000001d9373d048) Line 553 C mongod.exe!__session_open_cursor_int(__wt_session_impl * session=0x000001d2f3760000, const char * uri=0x00007ff755c0aba0, __wt_cursor * owner=0x0000000000000000, __wt_cursor * other=0x0000000000000000, const char * * cfg=0x0000001d9373cfc8, __wt_cursor * * cursorp=0x0000001d9373d048) Line 329 C mongod.exe!__wt_open_cursor(__wt_session_impl * session=0x000001d2f3760000, const char * uri=0x00007ff755c0aba0, __wt_cursor * owner=0x0000000000000000, const char * * cfg=0x0000001d9373cfc8, __wt_cursor * * cursorp=0x0000001d9373d048) Line 383 C mongod.exe!__wt_metadata_cursor_open(__wt_session_impl * session=0x000001d2f3760000, const char * config=0x0000000000000000, __wt_cursor * * cursorp=0x0000001d9373d048) Line 59 C mongod.exe!__wt_metadata_cursor(__wt_session_impl * session=0x000001d2f3760000, __wt_cursor * * cursorp=0x0000000000000000) Line 103 C mongod.exe!wiredtiger_open(const char * home=0x00007ff755d4b158, __wt_event_handler * event_handler=0x000001d2f26ad260, const char * config=0x000001d2f2518740, __wt_connection * * wt_connp=0x000001d2f26ad258) Line 2238 C mongod.exe!mongo::WiredTigerKVEngine::WiredTigerKVEngine(const std::basic_string,std::allocator > & canonicalName={...}, const std::basic_string,std::allocator > & path={...}, const std::basic_string,std::allocator > & extraOpenOptions={...}, unsigned __int64 cacheSizeGB=18, bool durable=true, bool ephemeral=false, bool repair=false, bool readOnly=false) Line 248 C++ mongod.exe!mongo::`anonymous namespace'::WiredTigerFactory::create(const mongo::StorageGlobalParams & params={...}, const mongo::StorageEngineLockFile * lockFile=0x000001d2f26b3070) Line 86 C++ mongod.exe!mongo::ServiceContextMongoD::initializeGlobalStorageEngine() Line 187 C++ mongod.exe!mongo::_initAndListen(int listenPort=27017) Line 568 C++ mongod.exe!mongo::initAndListen(int listenPort=27017) Line 757 C++ mongod.exe!mongoDbMain(int argc=3, char * * argv=0x000001d2f2501840, char * * envp=0x000001d2f2505e10) Line 1061 C++ mongod.exe!wmain(int argc=3, wchar_t * * argvW=0x000001d2f0abe7a0, wchar_t * * envpW=0x000001d2f0ad78c0) Line 795 C++ This is the error: Unhandled exception at 0x00007FFC66AADD27 (msvcr120.dll) in mongod.exe: An invalid parameter was passed to a function that considers invalid parameters fatal.
xgen-internal-githook commented on Fri, 8 Apr 2016 06:49:03 +0000: Author: {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexg@wiredtiger.com'} Message: SERVER-23588 Stop using _open_osfhandle on Windows. It introduced some odd failures on debug MongoDB builds. Switch to using fopen instead. Branch: mongodb-3.2 https://github.com/wiredtiger/wiredtiger/commit/e0fc972c47872ad517cd44c9c517eba01242bf8e xgen-internal-githook commented on Fri, 8 Apr 2016 06:46:10 +0000: Author: {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexg@wiredtiger.com'} Message: Import wiredtiger-wiredtiger-2.8.0-140-g7bcf6fc.tar.gz from wiredtiger branch mongodb-3.4 ref: 94c171d..7bcf6fc SERVER-23588 mongod with WiredTiger won't start on Windows when built with --dbg=on --opt=off SERVER-23517 WiredTiger changes for MongoDB 3.3.5 Branch: master https://github.com/mongodb/mongo/commit/58fe2bfb350939a4b8c6e8ddd9735a49dc8d2db3 xgen-internal-githook commented on Fri, 8 Apr 2016 06:41:14 +0000: Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: Merge pull request #2638 from wiredtiger/server-23588-win-fopen SERVER-23588 Stop using _open_osfhandle on Windows. Branch: mongodb-3.4 https://github.com/wiredtiger/wiredtiger/commit/5206656dfbb725a031be35203fc05f3f3beedc6f xgen-internal-githook commented on Fri, 8 Apr 2016 06:41:12 +0000: Author: {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexg@wiredtiger.com'} Message: SERVER-23588 Stop using _open_osfhandle on Windows. It introduced some odd failures on debug MongoDB builds. Switch to using fopen instead. Branch: mongodb-3.4 https://github.com/wiredtiger/wiredtiger/commit/e0fc972c47872ad517cd44c9c517eba01242bf8e xgen-internal-githook commented on Fri, 8 Apr 2016 05:02:26 +0000: Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: Merge pull request #2638 from wiredtiger/server-23588-win-fopen SERVER-23588 Stop using _open_osfhandle on Windows. Branch: develop https://github.com/wiredtiger/wiredtiger/commit/5206656dfbb725a031be35203fc05f3f3beedc6f xgen-internal-githook commented on Fri, 8 Apr 2016 05:02:24 +0000: Author: {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexg@wiredtiger.com'} Message: SERVER-23588 Stop using _open_osfhandle on Windows. It introduced some odd failures on debug MongoDB builds. Switch to using fopen instead. Branch: develop https://github.com/wiredtiger/wiredtiger/commit/e0fc972c47872ad517cd44c9c517eba01242bf8e alexander.gorrod commented on Fri, 8 Apr 2016 04:34:01 +0000: Thanks kaloian.manassiev I was able to reproduce when I included the enterprise modules. I didn't have much luck figuring out where we were going wrong with the API usage exactly, but I ended up switching out our usage of _os_openfhandle and fdopen to using fopen instead. With that change, our code became simpler and the problem was resolved. I've pushed that change into a pull request (https://github.com/wiredtiger/wiredtiger/pull/2638). kaloian.manassiev commented on Thu, 7 Apr 2016 18:14:45 +0000: Thanks, Alex. I looked into this a little bit more and I noticed that it only happens when enterprise mongod.exe is built with --dbg=on --opt=off, but does not happen when only building the free version. This is the command line I used, which should apply for spawnhost: scons -j 2 --dbg=on --opt=off --ssl --dynamic-windows --win-version-min=ws08r2 CPPPATH="c:/openssl/include c:/sasl/include c:/snmp/include" LIBPATH="c:/openssl/lib c:/sasl/lib c:/snmp/lib" --mute mongod.exe As far as msvcr120.dll is involved, I am not sure why this gets linked when using these parameters. mark.benvenuto, do you have any idea? alexander.gorrod commented on Thu, 7 Apr 2016 07:36:53 +0000: Another thing I noticed is that the stack trace you posted is using msvcr120.dll, but you mention building against the debug version which would be msvcr120d.dll. alexander.gorrod commented on Thu, 7 Apr 2016 02:25:47 +0000: kaloian.manassiev I've attempted to reproduce the failure you mention without success. I've also reviewed the code, and it appears reasonable to me. It follows the pattern outlined on StackOverflow here: http://stackoverflow.com/questions/7369445/is-there-a-windows-equivalent-to-fdopen-for-handles, which appears to comply with the Windows documentation for the functionality. Could you provide some additional information about the failure? I'm using an Evergreen Windows VS 2013 spawn host. I created a build as per your instructions in the ticket description. I started mongod from both a clean dbpath and a dbpath with an existing database (with and without content). I notice that you use --dbg=on --opt=off in both the build steps, and the reason why Evergreen doesn't encounter the problem. Do I need to do something else to link against debug runtime libraries on Windows?
Build mongod.exe with the following command line: scons -j 12 --dbg=on --opt=off --dynamic-windows --win-version-min=ws08r2 mongod.exe