Hi Brian,
You have a lot of interesting and exciting questions. You're hitting all the correct points in regards to SAP and high availability. My approach to the architecture of the NetWeaver platform is to simplify, offload and accelerate, I think this will be clear as I go into some more detail.
First, I see you're referring to both the SAP Messaging server and Web Dispatcher, if you are in a decision making point between the two, let me start off by saying that SAP, "...recommends that you use the SAP Web Dispatcher as the entrance point for your Web queries. This is then used as the access point for your network and also executes load balancing for HTTP requests." (1) (Link below) Please also see SAP Note 1040325 which describes exactly what decision process you should use to decided between Web Dispatcher and SAP Message Server. (2) (Link below)
To summarize, and please refer to the SAP documentation I've linked to for specific and exact wording, Web Dispatcher is the better software solution in the cases were you are doing redirects, when you're running applications through the portal and when your applications are state-full. SAP recommends a hardware solution such as BIG-IP LTM for offload, security and acceleration.
The SAP Landscape can be simplified and offloaded by using BIG-IP LTM instead of the Web Dispatcher. You've already mentioned many of the reasons, including SSL Off-Load, the use of iRules and there are many others which revolve around our full proxy architecture.
Getting more specific about your questions, in regards to balancing on load, we do have one solution with using Dynamic Ratio Load Balancing with SNMP (3) (link below). However, this does not go to the level of checking Java thread usage or SAP work process to the same extent of the Messaging Server. As F5 becomes more integrated with SAP this may certainly happen in the future.
In regards to logon groups, again, LTM does not have the same level of introspection that Web Dispatcher does. We can initiate persistence through cookies (inserted by LTM or by the server) which will insure that a user returns to the same Java instance, which improves Java performance.
Finally, in regards to your port redirection questions. On the first part, what you're describing is a very easy setup for LTM. With our full proxy architecture, you would setup the Virtual Server on an IP Address and port 80 and have the pools mapped to the instance of the Java on whichever port they are on. The client would only connect to port 80 and be seamlessly mapped to the proper port on the back. There is no redirect in this setup, but a full proxy communication.
In regards to the switch from http to https, it sounds like the iRule you already have could be tweaked to provide the URI to the client without the the port number visible, unless of course the HTTPS port is something under than the default 443 (in which case it needs to be there for the client to know where to send the request). I would have to look at the iRule to comment further. There are many creative solutions to such problems with Stream Profiles and iRules.
I've included the links for the items I was reference above:
-----------------------------------------
(1) SAP NetWeaver Library: "HTTP Load Distribution Using SAP Message Server" - http://help.sap.com/saphelp_nw2004s/helpdata/en/40/c235c15ab7468bb31599cc759179ef/content.htm
(2) SAP Service Note 1040325 https://service.sap.com/~form/handler?_APP=01100107900000000342&_EVENT=REDIR&_NNUM=1040325
(3) Implementing monitors for Dynamic Ratio load balancing: https://support.f5.com/kb/en-us/products/big-ip_ltm/manuals/product/LTM_config_guide_943/ltm_AppendixA.html1185064.