Symptoms
N/ARequesting metering information about a namespace along with bucket level details can take 50 to 60 seconds to complete.HA Proxy Load Balancer was used in this environment.Error seen:
504 Gateway Time-out
The server didn't respond in time.
Cause
The operation on the ECS takes longer than the Time-out value of the Load Balancer (LB).
Resolution
Using curl to obtain the metering data for a namespace and they also want bucket level details.The LB in this case is HA Proxy and the Time-out value was set to 60 seconds, but the command was constantly taking 50+ seconds.Based on that we can see how the LB was giving up on the command.Here is an example:
# curl -H "$TOK" https:///object/billing///info.json?include_bucket_detail=true
504 Gateway Time-out
The server didn't respond in time.
We then substitute the LB name with the public IP of one of the ECS nodes and we get a response (details not shown; customer data) but we can see it takes > 50 seconds to complete. When you factor in that other things need to happen we can see how this would exceed 60 and the LB would fail the request.
# time curl -H "$TOK" https://10.xxx.xxx.xxx/object/billing/namespace//info.json?include_bucket_detail=true
real 0m54.314s
user 0m0.004s
sys 0m0.008s
From the time command output above, we can see that the command actually return the time, but it takes a long time in response.
Improvement/enhancement has been released in ECS code version 3.0 and above, in order to have this metering request run faster when the user also requests to include bucket details.
