Hi Kirit,
If you have a very limited number of destination hosts and/or ports you want to allow access to, you could configure a single forwarding virtual server per destination host. You could then use source VLAN restrictions, an iRule or packet filters to restrict which clients can access the VIP.
A more flexible option would be to configure a network forwarding VIP (destination 0.0.0.0:0) and then use an iRule to restrict which source hosts/networks can access which destination hosts/networks. Here are two examples of such an iRule:
Access Control Based on IP
http://devcentral.f5.com/wiki/default.aspx/iRules/AccessControlBasedOnIP.html
Access Control Based on Network or Host
http://devcentral.f5.com/wiki/default.aspx/iRules/AccessControlBasedOnNetworkOrHost.html
Aaron