Symptoms
A database held in hardware (TCAM), shared between tenants, has a limit that is exceeded by software in tenants that adds and manages entries in the database.
Symptomatic logs on tenant:
in /var/log/ltm, repeating logs are recorded, following is an example:
err tmm[635]: 01010331:3: Neuron client neuron_client_pva_hwl failed with rule request submit(client connection is busy (has outstanding requests))
in /var/log/tmm, cycles of following group of logs are recorded:
notice neuron_client_negotiate: Neuron client connection established
notice [DDOS Neuron]Neuron daemon started
notice hudproxy_neuron_client_closed_cb: Neuron client connection terminated
notice [DDOS Neuron]Neuron daemon stopped
For F5OS host, in partition /var/F5/partitionX/log/velos.log repeating logs are recorded, following is an example:
tcam-manager[41]: priority="Err" version=1.0 msgid=0x6b01000000000007 msg="ERROR" MSG="TCAM processing Error(-5) executing:TCAM_INSERT for ruleno:0x20000000937"
In the log message, the msgid and ruleno can vary, but the Error(-5) is an indication of this issue.
Impact
The neuron client software will restart and log repeatedly.
Inefficient use of TCAM database.
Conditions
The BIG-IP system with a rSeries r5xxx, r10xxx, r12xxx or has VELOS blades such as BX110.
The rSeries 2xxx, 4xxx and iSeries platforms are not affected.
Large configurations, on the order of high hundreds of virtual servers, are more likely to encounter issue.
