Web-GUI not working after Reboot

Hi,

after rebooting CoaL i'm not able to login via web-GUI. I'm getting the following error:

"Services required for authentication are currently unavailable. Signin will be re-enabled when service is restored"

It seems to be kind of a timing issue: after reboot adminui checks if authentication service is available - but not all zones are finished booting up. So adminui needs to check again after some time if authentication is available now.

I can fix the problem via "svcadm restart adminui" when i'm logged in to the adminui zone.

Logfile gives me the following:

[2016-04-02T15:37:35.054Z]  INFO: adminui/audit/15389 on 002ef8fe-783c-4a1c-b2fd-977ade7eed20: handled: 503 (req_id=d263dfd0-f8e8-11e5-a021-b7edb026542f, route=getapiauth, audit=true, remoteAddress=192.168.2.130, remotePort=60730, latency=2, _audit=true)
    GET /api/auth HTTP/1.1
    host: 192.168.9.101
    accept-language: de-de
    accept-encoding: gzip, deflate
    connection: keep-alive
    accept: */*
    user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/601.5.17 (KHTML, like Gecko) Version/9.1 Safari/601.5.17
    x-adminui-token: cd20668f3d5592a8934026d69cdc2a5dfd15d294
    referer: https://192.168.9.101/
    x-requested-with: XMLHttpRequest
    --
    HTTP/1.1 503 Service Unavailable
    content-encoding: gzip
    content-type: application/json

    {
      "code": "ServiceUnavailableError",
      "message": "service unavailable (moray)"
    }
    --
     ServiceUnavailableError: service unavailable (moray)
        at Server.requireAuth (/opt/smartdc/adminui/lib/auth.js:76:21)
        at next (/opt/smartdc/adminui/node_modules/restify/lib/server.js:745:30)
        at f (/opt/smartdc/adminui/node_modules/restify/node_modules/once/once.js:17:25)
        at attachOtherClients (/opt/smartdc/adminui/lib/adminui.js:180:16)
        at next (/opt/smartdc/adminui/node_modules/restify/lib/server.js:745:30)
        at f (/opt/smartdc/adminui/node_modules/restify/node_modules/once/once.js:17:25)
        at Server.attachSdcClients (/opt/smartdc/adminui/lib/sdc-clients.js:36:9)
        at next (/opt/smartdc/adminui/node_modules/restify/lib/server.js:745:30)
        at f (/opt/smartdc/adminui/node_modules/restify/node_modules/once/once.js:17:25)
        at Server.<anonymous> (/opt/smartdc/adminui/lib/trace-event.js:24:13)
    --
    req.timers: {
      "bunyan": 42,
      "parseQueryString": 18,
      "gzip": 162,
      "parseAccept": 59,
      "handler-4": 18,
      "attachSdcClients": 10,
      "handler-6": 11,
      "requireAuth": 522
    }

In Addition, i'm seeing an unhandled expection right after the services has started (not quite shure if it is related to my problem):

[2016-04-02T15:25:11.851Z]  INFO: adminui/15389 on 002ef8fe-783c-4a1c-b2fd-977ade7eed20: HTTP Server started on 0.0.0.0:80
[2016-04-02T15:25:11.852Z]  INFO: adminui/15389 on 002ef8fe-783c-4a1c-b2fd-977ade7eed20: HTTPS Server started on 0.0.0.0:443
[ Apr  2 15:25:12 Rereading configuration. ]
[ Apr  2 15:25:12 No 'refresh' method defined.  Treating as :true. ]
[2016-04-02T15:26:11.928Z] FATAL: adminui/15389 on 002ef8fe-783c-4a1c-b2fd-977ade7eed20: Uncaught Exception
    AssertionError: connection state change after close
        at Object.module.exports.(anonymous function) [as notEqual] (/opt/smartdc/adminui/node_modules/assert-plus/assert.js:242:35)
        at Client._onConnection (/opt/smartdc/adminui/node_modules/moray/node_modules/fast/lib/client.js:430:12)
        at /opt/smartdc/adminui/node_modules/moray/node_modules/fast/lib/client.js:196:14
        at FunctionCall.doCallback_ (/opt/smartdc/adminui/node_modules/moray/node_modules/backoff/lib/function_call.js:162:20)
        at FunctionCall.abort (/opt/smartdc/adminui/node_modules/moray/node_modules/backoff/lib/function_call.js:121:10)
        at Client.close (/opt/smartdc/adminui/node_modules/moray/node_modules/fast/lib/client.js:180:30)
        at /opt/smartdc/adminui/node_modules/moray/lib/connection_pool.js:134:16
        at Array.forEach (native)
        at ConnectionPool.removeHost (/opt/smartdc/adminui/node_modules/moray/lib/connection_pool.js:132:19)
        at /opt/smartdc/adminui/node_modules/moray/lib/connection_pool.js:76:14

Thanks,
Kilian

Start with lib/auth.js at requireAuth and trace startup client attachment through lib/adminui.js and lib/sdc-clients.js. Reproduce the reboot sequence and inspect the Moray connection errors in the provided logs; done means authentication becomes available after dependent services finish starting without requiring an adminui restart, and the uncaught exception is addressed if related.