Launching soon: The world's first vendor agnostic bug scrubLearn more & join waitlist
OPERATIONAL DEFECT DATABASE
...
...
Upon upgrading to Workspace ONE UEM version 2209, servers hosted on Windows server 2012 R2 may experience the following symptoms: Web services running on IIS, such as Console, API or Device Services, fails to loadEnrollments and device communication with UEM may failREST API endpoints are inaccessible Error Observed when reaching Health check end-points:HTTP Error 500 - Internal Server ErrorError on device enrollmentsJSON could not be serialized because of error : The data couldn't be read because it isn't in the correct format. For the concerned endpoint, IIS log will record HTTP status code as 500 and sub-status code as 19 ("500 19"):GET /AirWatch/default.aspx - 443 - 192.x.x.x Mozilla/5.0 - 500 19 13 4933 691 110 - Version Identified Workspace ONE UEM 2209
Enhancements have been made to remove IIS Server version information from HTTP responses. These changes are not compatible with IIS version 8.5 and lower.
This may result in the following impact: Console is not accessibleEnrollments fail and devices cannot communicate with WS1 UEMREST API is inaccessibleHealth check endpoint does not load Console – GET to https://<ConsoleURL>/airwatch/awhealth/v1 Device Services – GET to https://<DeviceServicesURL>/deviceservices/awhealth/v1 Device Management – GET to https://<DeviceServicesURL>/devicemanagement/awhealth/v1 MDM API – GET to https://<RESTApiURL>/api/mdm/hcSystem API – GET to https://<RESTApiURL>/api/system/hcMEM API – GET to https://<RESTApiURL>/api/mem/hcMAM API – GET to https://<RESTApiURL>/api/mam/hc This affects environments hosted on Windows Server 2012 R2, running WS1 UEM 2209 and above
For customers on Workspace ONE UEM 22.9.0.11+ deployed on Windows Server 2012 R2, VMware has released a custom patch to help ease the complexity of the provided workaround while maintaining security posture. The following custom patch will provide a method to remove the vulnerable server header via a modified programmatic method in addition to removing the incompatible IIS configurations found in the various Web.Config files. Additional deployment steps can be found in the custom DLL package. https://resources.workspaceone.com/view/znbcwbr3mphy2y484hqd/. Please note that this is the only method to enable this communication flow and outside of this script execution the associated DLL will never be enabled. Additionally, this patch is intended to be a temporary resolution for Windows Server 2012 R2 on Workspace ONE UEM 2209. The changes contained within this patch will not be merged into the main code base and the expectation will be that customers upgrade their Windows OS's at the latest with Workspace ONE UEM v2302. For information on our recently published update to our OS supportability model please see:https://techzone.vmware.com/blog/upcoming-changes-workspace-one-uem-windows-and-sql-supported-versionshttps://kb.vmware.com/s/article/90455
You may leverage any one of options noted below to mitigate the problem: Upgrade windows server to 2016 or above.Post upgrade to UEM 2209, make manual changes to below files which have been identified as leading to page load failures. Files : Console Server {WS1 UEM Install Directory}:\AirWatch\Default Website\web.config {WS1 UEM Install Directory}:\AirWatch\AirWatch XXXX\Websites\WanderingWiFi.AirWatch.Console.Web\web.config Device Services Server {WS1 UEM Install Directory}:\AirWatch\Default Website\web.config {WS1 UEM Install Directory}:\AirWatch\AirWatch XXXX\Websites\WanderingWiFi.AirWatch.DeviceServices\web.config API Server {WS1 UEM Install Directory}:\AirWatch\Default Website\web.config {WS1 UEM Install Directory}:\AirWatch\AirWatch XXXX\Websites\AirWatchApi\System\web.config{WS1 UEM Install Directory}:\AirWatch\AirWatch XXXX\Websites\AirWatchApi\Mdm\web.config{WS1 UEM Install Directory}:\AirWatch\AirWatch XXXX\Websites\AirWatchApi\Mam\web.config{WS1 UEM Install Directory}:\AirWatch\AirWatch XXXX\Websites\AirWatchApi\Mcm\web.config{WS1 UEM Install Directory}:\AirWatch\AirWatch XXXX\Websites\AirWatchApi\Mem\web.config Change required :The following change is done to a text file in XML format.Remove or comment out the XML attribute of removeServerHeader="true" from the <requestFiltering> element.Note there shouldn't be any attributes called "removeServerHeader" as the result, regardless it has a "true" or "false" value. For example:Before change:After change: Restart IIS after updating the web.config file(s) Note: The aforementioned alteration needs to adhere to the syntax regulations of XML. An XML syntax mistake will result in the identical '500 19' status code within the IIS log.You can use a XML formatter such as Edge or IE browser to validate whether modified file conforms to the structure of well-formed XML: Create a copy of the modified web.config, and add the ".xml" extension to the copied file. For example, "web.config.xml" Launch Edge or Internet Explorer browser, use File→Open (Control + O) to open the copied XML file When there is no XML syntax error, the browser will display formatted XML. For example:When there is any syntax error, the browser will tell which line and column the problem lies.For example:Repeat the edit/test cycle until there is no syntax error. Apply caution to: Make sure to update the web.config file with the tested XML content as the final step Since this needs to be done to multiple web.config files, make sure not to mix up one web.config with another one.
Change Log Dec 21, 2022 - Updated title to reflect correct issue identifier (CRSVC-33876)Dec 21, 2022 - Provide DLL Custom Patch ResolutionJan 20, 2023 - Updated Resource Linked to an updated script which handles additional web.config scenariosAug 23, 2023 - Updated Symptoms and Workarounds