I would do this by applying a virtual 0.0.0.0/0 against ONLY the internal vlan, with a rule applied (below). I am not sure I understand your requirements as a couple of them seem conflicting. Here's a start for you, and If I am misunderstanding, please post back. Also, if you use AH or NAT-T at all, you'll also need to allow for protocol 51 and udp/4500 (respectively)
when CLIENT_ACCEPTED {
if { [IP::protocol] == 50 } {
pool isp-gateways member ISP1
} elseif { [UDP::local_port] == 500 } {
pool isp-gateways member ISP1
} else {
pool isp-gateways
}
}
Of course, you'd need to make sure you have a forwarder for your internal vlan as well applied ONLY to the public-facing vlan.