To allow just SMTP, DNS, and TLS, you can use the above data group method (preferred for complex filtering) or this simple iRule:
when CLIENT_ACCEPTED {
if { not ( [TCP::local_port] eq "53" or [TCP::local_port] eq "443" or [TCP::local_port] eq "25" ) } {
drop
}
}
If you literally meant "DNS UDP", then you could expand the evaluation like this:
not ( ( [TCP::local_port] eq "53" and [IP::protocol] eq "17" ) or [TCP::local_port] eq "443" or [TCP::local_port] eq "25" )
and if you literally meant "TLS" versus SSL, you could add something like the following:
not ( ( [TCP::local_port] eq "53" and [IP::protocol] eq "17" ) or ( [TCP::local_port] eq "443" and [SSL::cipher version] eq "TLSv1" ) or [TCP::local_port] eq "25" )
You should also be able to configure BIG-IP packet filters rules to accommodate your needs.