...
TCP sessions stuck in FINWAIT for long time show tcp brief | inc :49 Wed Dec 9 16:21:28.442 PST 0x1042bc88 0x60000000 0 0 xx.xx.236.36:37530 xx.xx.32.26:49 FINWAIT1 0x104a2064 0x60000000 0 0 xx.xx.236.36:41629 xx.xx.32.26:49 ESTAB 0x102c1eb0 0x60000000 0 0 xx.xx.236.36:33446 xx.xx.32.26:49 FINWAIT1 0x10277ec4 0x60000000 0 0 xx.xx.236.36:37546 xx.xx.32.26:49 FINWAIT1 0x10630614 0x60000000 0 0 xx.xx.236.36:19118 xx.xx.32.26:49 FINWAIT1 - continuous AAA transaction failures. - transactions will back to normal in between - Again go back to failure state.
When "aaa accounting system default start-stop group tacacs+" is reconfigured or configured Seen on 5.2.21 and 5.3.0 release (other release might be affected as well) AAA accoutning transactions configured to use TACACS, - Issue is associate with "aaa accounting system" configuration - This configuration triggers this issue. If "aaa accounting system" is configured to use "tacacs" as a method. in case of transaction failure, "locald" will keep retrying the message. This retry will force tacacsd to open multiple sockets. - In case of transaction failure tacacsd cleans up socket after 5 minutes. - the opened sockets count keep accumulating for 5 minutes and creates issue for new transaction.
use "aaa accounting system default start-stop group tacacs+ none" need to restart "locald_DSC" process always configure system accounting with a "none" failover method. - If system accounting fails after initial default retry count. - with a "none" key work system will mark the feature as done (not as success) - This will avoid infinite retry configuration: aaa accounting system default start-stop group tacacs none
Recovering after hitting this issue: 1) configure aaa accounting system default start-stop group tacacs none 2) restart "locald_DSC" process. 3) wait for 15 minutes 4) after 15 minutes all open socket will get cleared (both in FIN wait, EST). 5) after with there will be a very few sockets opened (ESTstate) fromasr9k.