Symptoms
After upgrading Avamar server and client to 7.3 or higher, some backups going to Data Domain fail with error "the user has insufficient access rights".Sample backup log:2017-01-17 15:08:02 avtar Info : - Establishing a connection via token to the Data Domain system with encryption (Connection mode: A:3 E:2).2017-01-17 15:08:12 avtar Warning : Calling DDR_OPEN_VIA_TOKEN returned result code:5075 message:the user has insufficient access rights2017-01-17 15:08:12 avtar Error : Data Domain server "DDserver.local" open failed DDR result code: 5075, desc: the user has insufficient access rights2017-01-17 15:08:12 avtar Error : Problem logging into the DDR server:'', only GSAN communication was enabled.2017-01-17 15:08:12 avtar FATAL : Backup is incomplete because file "/ddr_files.xml" is missing2017-01-17 15:08:12 avtar Info : DDR errors caused the backup to not be posted, errors=0, fatals=02017-01-17 15:08:12 avtar Info : Backup was not committed to the DDR.2017-01-17 15:08:12 avtar FATAL : Fatal server connection problem, aborting initialization. Verify correct server address and login credentials.
Cause
The client's hostname does not match the --hostname flag in avagent.cmd file. Avamar 7.3 uses token based authentication as the default authentication method to DD. The token is requested by Avamar server on behalf of the client, and it is assigned by DD. When the client uses this token to connect to DD, DD finds out that the owner of the token does not have the same name as the hostname in the token. Therefore DD considers the token as invalid for this client, and backup fails.Sample error that is present in ddfs.info about the invalid token:(tid 0x7fe816905c20): ost_validate_token: Invalid Token, Client hostname avamar.local not found in Token with list(avamar.com).Avamar server issues the token via ddrmaint commands. The request can be found in /data01/avamar/var/ddrmaintlogs/ddrmaint.log:ddrmaint.bin[10688]: Info: request-token:body:service avamar.local.
Resolution
There are two workarounds possible:1. Remove --hostname flag in avagent.cmd file, and re-register the client with the actual hostname.2. Disable token authorization in MCS
a. Edit /usr/local/avamar/var/mc/server_data/prefs/mcserver.xml configuration fileb. Replace the following entry:
use_ddr_auth_token = true
with
use_ddr_auth_token = false
c. Restart MCS:
dpnctl stop mcsdpnctl start mcs, sched
