Launching soon: The world's first vendor agnostic bug scrubLearn more & join waitlist
OPERATIONAL DEFECT DATABASE
...
...
Document Version Release Date Details 3 06/10/2020 Updated information about PHSS_44778 in Description and updated Resolution. 2 11/26/2018 Updated formatting in the Description section. 1 11/21/2018 Original Document Release. Use of restricted words in the HPE Serviceguard cluster may result in the cluster failing. The use of restricted keywords will result in cmcld causing a core dump with SIGSEGV. Such use refers to having some string-based attribute value in the cluster database (CDB) which starts with one of the following keywords. acps address applied_configuration_version capacities cluster cm condition config criteria dependencies detachednodes environment generic_resources gms groups ip lan module_versions networks nodes peers pkgs polling_targets prefixes quorum_servers resources script_log_file security services storage_groups subnet weights weight_defaults Common examples of string-based attribute values in the CDB include the cluster name, a nodename and a package name. The full list of impacted CDB attributes are as follows: Cluster attributes: "CLUSTER_NAME", "NODE_NAME", "USER_NAME", "USER_HOST", "HPVM_VOLUME_GROUP", "VOLUME_GROUP", "OPS_VOLUME_GROUP", "GMS_LOCATON", "CAPACITY_NAME", "WEIGHT_NAME", Package attributes: (Names in lower cases for modular packages and upper cases for legacy ones) "package_name" or "PACKAGE_NAME" "node_name" or "NODE_NAME" "script_log_file" or "SCRIPT_LOG_FILE" "service_name" or "SERVICE_NAME" "resource_name" or "RESOURCE_NAME" "dependency_name" or "DEPENDENCY_NAME" "weight_name" or "WEIGHT_NAME" "generic_resource_name" or "GENERIC_RESOURCE_NAME" "ip_subnet_node" or "IP_SUBNET_NODE" "service_cmd" or "SERVICE_CMD" "generic_resource_name" or "GENERIC_RESOURCE_NAME" "vg" or "VG" "vxvm_dg" or "VXVM_DG" "user_name" or "USER_NAME" "user_host" or "USER_HOST" For clusters with one or more of the above attributes having a value starting with a restricted keyword, unexpected behaviors may occur when trying to check and/or update the CDB through commands like cmcheckconf, cmapplyconf and cmdeleteconf. A patch fix was previously released, as described in the patch text of PHSS_44549 described below. Defect: QXCR1001473542 When the cluster is up, cmapplyconf of a package configuration file having package_name starting with the word "config" fails because cmcld is unable to recognize the package while scanning the CDB. A word starting with "config" is treated by the code as a reserved word. When the package name "config_pkg" is received by cmcld, it assumes that this is a reserved key-value pair and this assumption finally leads to the failure. Resolution: With this fix, cmcheckconf / cmapplyconf will report that a package name cannot begin with the word "config". The patch containing this fix will not install on a system which contains a package with the package name starting with the word "config" until Note that the above fix in PHSS_44549 attempted to address the issue only with keyword "config". A later fix introduced in Serviceguard 11.20 patch PHSS_44778 attempted to address ALL identified keywords. Another known unexpected behavior involves an illegal package name starting with a restricted keyword. Running cmapplyconf to apply the package configuration, cmapplyconf should fail with an error. But when the cluster is stopped, cmapplyconf does not complain and allows the illegal package name to be applied. And if such a package is later removed with cmdeleteconf, the removal of the package will leave stale references in the CDB, which can result in cmcld aborting on next cluster reformation. In one reported case, cmcld aborted with SIGSEGV and a stack trace like the following when cluster reformation took place as a result of a cmhaltnode operation. pe_setup_pkg_result () at pe/pe_eval_placement.c:2380 pe_setup_pkg_result_all () at pe/pe_eval_placement.c:2332 pe_eval_placement () at pe/pe_eval_placement.c:169 pm_place_all () at pkg/pkg_pe_support.c:1673 pm_sync_done_event () at pkg/pkg_reconfig.c:739 pm_sync_event () at pkg/pkg_reconfig.c:320 pm_event_handler () at pkg/pkg.c:820 deliver_events_on_list_of_queues () at utils/cl_event.c:402 cl_event_loop () at utils/cl_event.c:520 Given the possibility of cmcld abort due to stale references in the CDB, a node can go down (a Serviceguard TOC) or even the entire cluster can go down at cluster reformation. It is therefore important to get rid of any attribute value in the CDB that starts with a restricted keyword. The advisory resolution given below is to detail the steps for the cleanup.
Any HPE system running HP-UX 11i v3 Serviceguard version A.11.20.00 (or later).
The original resolution proposed installing PHSS_44778 as the final fix addressing QXCR1001667763 as shown below. 9. Symptom: QXCR1001667763 On one node, cmcld core-dump'ed with SIGSEGV and later resulted in a SG-TOC in cluster reformation after deleting a package which name starts with one of the restricted keywords, which is used by Serviceguard internally for CDB. core.ORG: ELF-32 core file - IA64 from 'cmcld' - received SIGSEGV $(gdb) bt #0 0x4234170:0 in pe_setup_pkg_result () at pe/pe_eval_placement.c:2380 #1 0x422bc20:0 in pe_setup_pkg_result_all () at pe/pe_eval_placement.c:2332 #2 0x422ac00:0 in pe_eval_placement () at pe/pe_eval_placement.c:169 #3 0x42d6230:0 in pm_place_all () at pkg/pkg_pe_support.c:1673 #4 0x42d9e20:0 in pm_sync_done_event () at pkg/pkg_reconfig.c:739 #5 0x42d8740:0 in pm_sync_event () at pkg/pkg_reconfig.c:320 #6 0x42497b0:0 in pm_event_handler () at pkg/pkg.c:820 #7 0x439b3b0:0 in deliver_events_on_list_of_queues () at utils/cl_event.c:402 #8 0x439b910:0 in cl_event_loop () at utils/cl_event.c:520 9. Defect: QXCR1001667763 Serviceguard uses some restricted keywords in cluster database (CDB) for internal purpose. If an attribute value starts with one of these restricted keywords, Serviceguard incorrectly interprets the CDB value as attribute. Due to this, sometimes online validation or configuration of cluster or package fails with unknown reasons. There will also be node failures and cluster failures in next cluster database configuration change due to caching of these restricted keywords in cmcld memory. Resolution: cmcheckconf and cmapplyconf have been enhanced with additional checks to disallow package or cluster configuration using Serviceguard restricted keywords which are used internally by the cluster database. Unfortunately, the fix in PHSS_44778 exhibited a regression causing an installation failure. The following patch warning was posted. 20/03/02 - This Non-Critical Warning has been issued by HPE. PHSS_44778 fails to install on Serviceguard nodes where the access control parameter "user_host" or "USER_HOST" has been set to "cluster_member_node". Users should take one of the following steps: 1) Do not install the patch. Install the replacement patch to be released in April 2020 time frame. 2) Set "user_host" or "USER_HOST" to "CLUSTER_MEMBER_NODE" or "Cluster_member_node" to install patch PHSS_44778 anyway. PHSS_44821 was released in April 2020 to address the above regression. 2. Defect: QXCR1001727597 PHSS_44778, in addressing QXCR1001667763, adds checking to disallow attribute values starting with keywords reserved for CDB's internal use. But some predefined values, like 'cluster_member_node' for the 'user_host' attribute should be allowed. The patch installation fails because the checking imposed by the patch does not cater for legitimate predefined values that happen to start with a restricted keyword. Resolution: This fix will allow the values for attributes which have predefined values even those they are reserve strings. At the same time, PHSS_44821 now also bans attribute values starting with restricted keywords in uppercase letters because they could also expose QXCR1001667763. Previously, PHSS_44778 only bans attribute values starting with restricted keywords in lowercase letters. Note : Attribute values starting with restricted keywords in a mixture of uppercase and lowercase letters are allowed. They will not expose QXCR1001667763. IMPORTANT : The (revised) final resolution is to install PHSS_44821. All Serviceguard/HP-UX clusters should install PHSS_44821. During the patch installation, the checkinstall script of PHSS_44821 will scan the CDB to check for the presence of any cluster/package attributes starting with a restricted keyword in either all uppercase or all lowercase letters. If exists, the patch installation will fail. In addition, HPE recommend also performing a manual inspection of "cmviewcl -v -fline" output to look for restricted keywords in the configurations. Please ignore the checks only for cluster and package attribute name "USER_HOST/user_host", which might contain a predefined values that starts with restricted keyword. If other cluster or package attributes in the cluster start with a restricted keyword perform the following steps: Halt the cluster: # cmhaltcl Removal of cluster and package configuration with illegal attribute values starting with a restricted keyword. A. If the illegal attribute is a cluster attribute, the whole cluster needs to be deleted. Ensure the existing cluster and package configuration has a backup before deleting the cluster. B. If the illegal attribute is a package attribute, the entire package must be deleted. Ensure that the existing package configuration has a backup and delete the package. C. If the illegal attribute is a cluster member's node name, the member node must be removed from the cluster. Refer to "Managing Serviceguard A.11.20" manual for steps to remove a node from the cluster. Install PHSS_44821 on all nodes after performing all of 2A, 2B, 2C above. The preinstall script should complete the check successfully. Re-apply the impacted cluster and packages after correction in all attributes of cluster and packages configuration files. If the node was deleted as part of Step 2C, refer to the "Managing Serviceguard A.11.20" manual for steps to add a new node to the cluster. After all the illegal attributes in cluster and packages configuration files are renamed, run the cluster: # cmruncl RECEIVE PROACTIVE UPDATES : Receive support alerts (such as Customer Advisories), as well as updates on drivers, software, firmware, and customer replaceable components, proactively via e-mail through HPE Subscriber's Choice. Sign up for Subscriber's Choice at the following URL: Proactive Updates Subscription Form. NAVIGATION TIP : For hints on navigating HPE.com to locate the latest drivers, patches, and other support software downloads for ProLiant servers and Options, refer to the Navigation Tips document . SEARCH TIP : For hints on locating similar documents on HPE.com, refer to the Search Tips Document . To search for additional advisories related to HP-UX 11i v3 Serviceguard, use the following search string: +Advisory +ProLiant -"Software and Drivers" +HP-UX 11i v3 Serviceguard