Due to security policy, my department is being asked to use a reverse proxy to protect those SMTP servers.
if you just want to separate client-side (between client and bigip) and server-side (between bigip and server) connections, standard virtual server type already does.
sol8082: Overview of TCP connection setup for BIG-IP LTM virtual server types
https://support.f5.com/kb/en-us/solutions/public/8000/000/sol8082.html
if you want bigip to filter some smtp traffic, there is smtp profile in asm module. otherwise, you may have to write irule to filter traffic you do not want.
Manual Chapter: Securing SMTP Traffic Using the Default Configuration
https://support.f5.com/kb/en-us/products/big-ip_asm/manuals/product/asm-implementations-11-5-0/49.html