It sounds to me like you want to use APM and use Kerberos Auth (SPNEGO) which has the client get a ticket from the domain server directly and then send that ticket to the APM. Then you can put your webserver in a pool behind the Kerberos auth APM. No popup will be shown as long as your browser is configured to send Kerberos auth on a 401 (the method of configuring this is different in IE or Firefox, but you can likely push out the IE config via group policy).
Here is the guide for the APM configuration part:
https://support.f5.com/kb/en-us/products/big-ip_apm/manuals/product/apm-aaa-auth-config-11-2-0/4.html