Symptoms
iControl REST queries may fail against specific versions of BIG-IP in AWS.
When this issue is encountered, all queries fail for the entirety of the BIG-IP uptime.
An error message mentioning "RestWorkerUriNotFoundException" will be returned.
For instance, this basic query will always return 404:
curl -k -u admin:ADMINPASSWORD -sv -X GET https://1.2.3.4/mgmt/tm/ltm
*
Trying 1.2.3.4...
* Connected to 1.2.3.4 (1.2.3.4) port 443 (#0)
* TLS 1.2 connection using TLS_DHE_RSA_WITH_AES_256_CBC_SHA256
* Server certificate: localhost.localdomain
* Server auth using Basic with user 'admin'
> GET /mgmt/tm/ltm HTTP/1.1
> Host: 1.2.3.4
> Authorization: Basic ....
> User-Agent: curl/7.43.0
> Accept: */*
>
< HTTP/1.1 404 Not Found
< Date: 20 Jun 2016 17:49:39 UTC
< Server: com.f5.rest.common.RestRequestSender
...
{ [1093 bytes data]
* Connection #0 to host 1.2.3.4 left intact
{
"errorStack" : [
"com.f5.rest.common.RestWorkerUriNotFoundException: http://localhost:8100/mgmt/tm/ltm",
"at com.f5.rest.workers.ForwarderPassThroughWorker.cloneAndForwardRequest(ForwarderPassThroughWorker.java:293)",
"at com.f5.rest.workers.ForwarderPassThroughWorker.onForward(ForwarderPassThroughWorker.java:211)",
"at com.f5.rest.workers.ForwarderPassThroughWorker.onGet(ForwarderPassThroughWorker.java:370)",
"at com.f5.rest.common.RestWorker.callDerivedRestMethod(RestWorker.java:1009)",
"at com.f5.rest.common.RestWorker.callRestMethodHandler(RestWorker.java:976)",
"at com.f5.rest.common.RestServer.processQueuedRequests(RestServer.java:850)",
"at com.f5.rest.common.RestServer.access$000(RestServer.java:43)",
"at com.f5.rest.common.RestServer$1.run(RestServer.java:147)",
"at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)",
"at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)",
"at java.lang.Thread.run(Thread.java:722)\n"
],
"restOperationId" : 8827,
"code" : 404,
"referer" : "4.3.2.1",
"message" : "http://localhost:8100/mgmt/tm/ltm"
}
Impact
All iControl REST queries (GETs, PUTs, POSTs, DELETEs) will fail always until the BIG-IP is restarted.
Conditions
It is not known what triggers this, it intermittently affects new BIG-IP instances running in Amazon Web Services (AWS EC2) cloud environments.
