Launching soon: The world's first vendor agnostic bug scrubLearn more & join waitlist
OPERATIONAL DEFECT DATABASE
...
...
1. In the /var/log/syslog, you may see log messages similar to the log lines below. 2023-06-30T01:34:55.506234+00:00 nos-wld-nsxtmn02.vcf.netone.local kernel - - - [6869346.074509] sd 2:0:1:0: [sdb] tag#1 CDB: Write(10) 2a 00 04 af de e0 00 02 78 00 2023-06-30T01:34:55.506238+00:00 nos-wld-nsxtmn02.vcf.netone.local kernel - - - [6869346.074512] print_req_error: 1 callbacks suppressed 2023-06-30T01:34:55.506240+00:00 nos-wld-nsxtmn02.vcf.netone.local kernel - - - [6869346.074516] print_req_error: I/O error, dev sdb, sector 78634720 2023-06-30T01:34:55.513497+00:00 nos-wld-nsxtmn02.vcf.netone.local kernel - - - [6869346.075123] EXT4-fs warning: 3 callbacks suppressed 2023-06-30T01:34:55.513521+00:00 nos-wld-nsxtmn02.vcf.netone.local kernel - - - [6869346.075127] EXT4-fs warning (device dm-8): ext4_end_bio:323: I/O error 10 writing to inode 4194321 (offset 85286912 size 872448 starting block 9828828) 2. Filesystem goes into read only mode nvpapi/api_server.log , you may see log messages similar to the log lines below. 2023-07-13T03:11:02.145Z napi.root.node.services.utils.node-mgmt ERROR File system(s) found to be in read-only mode: /dev/sdb1 -> /mnt 3. "mount" command show file system used by appliance in a readonly mode.4. Any of the filesystem used by appliance if full.5. Not able to login into appliance or repetitive login failure message.
It may cause because of following reason 1. File System Corruption: When the file system is corrupted, Manager appliance OS might choose to mount it as read-only to prevent further damage.2. Improper System Shutdown: Abrupt power loss or an unclean system shutdown can lead to a read-only file system.3. Hardware Issues: Faulty hard drives or damaged sectors can cause the file system to switch to read-only mode.4. Incorrect File System Configuration: Mistakes in the /etc/fstab file or other configuration files can result in a read-only file system.
If there is a file system corruption, the OS level file system may be repaired using disk utilities, however, the VMware components such as corfu is not repaired using these tools.In such cases where file system corruption may have occurred and the system is in read only mode, it is advised to restore the appliances from back, to a point before the outage occurred.For details on restoring from backup, please review the VMware NSX Administration Guide.
Please note, backups are a requirement in production environments.If you believe you have encountered this issue and are unable to restore from backup, please open a support request with VMware NSX-T GSS and refer to this KB article. For more information, see How to Submit a Support Request.