Thanks everyone for the replies. I did finally get this working. All of the VLANs, interface numbers, and MAC addresses (virtual nics in vsphere and nics in BigIP) all did line up correctly. I did have all of the self IPs set to "allow all" for port lockdown, and I was using VMxnet3 for the virtual NICs.
The first problem was configuration problems on the virtual distributed switches in our virtualization environment, although I can't provide any detail about what was changed. I confirmed the virtual distributed switches in vcenter were configured for the correct VLANs. Next, I ran into a problem where enabling one of the virtual NICs wasn't working due a bug, whose fix was to shut down the VM, delete the NIC, and add a new NIC. Next, I had to use untagged VLANs, whereas our production Big IPs use tagged VLANs because of trunking.
Amazingly, after I got the one virtual NIC re-added and configured, my ping to its gateway started working. I asked our vmware guy to hold off on the changes to the 'Promiscuous Mode' and 'Forget Transmits' settings.
I added back the self IPs and VLANs for the other 2 interfaces, a default route, and one static route, confirmed the NICs were 'connected' in vcenter, and these new BigIP interfaces started working correctly too, pinging their gateways and reaching other networks.
So, I wish I could provide one simple reason for the problems, but it was at least 3 different factors: use untagged vlans, ensure virtual NICs connected/up, ensure virtual switches set correctly.
Now I can start testing the iApp for Websense, in hopes of figuring out why it doesn't work in our production environment.