Replacing the WebSphere Apache Plugin with iRules

Problem Definition

"We’re having a bit of difficulty configuring the LTM to handle all the redirects that this WebSphere application does. We’ve tried streaming profiles and iRules, but every method seems to break one component while fixing another. The main trick seems to be trying to deal with the default WebSphere ports of 9081 and 9444 for HTTP and HTTPS, respectively. We ideally want to hide these odd-number ports from the end-user. Normally this is a fairly simple procedure, but it’s proved pretty challenging. The issue may lie on the server and/or in the application code, but we’d like to be able to flex the muscle of the F5, if we could, and solve the problem there. One of the main stumbling blocks seems to be pop-up windows for viewing documents (PDF and Word). Word docs instantiate a Java applet, and we’ve had some success rewriting the requests there, but it’s the Adobe file transfer / view that has been the most confounding.

The real puzzler is that IBM provides an Apache-based load-balancer with a WebSphere plugin that works really well in hiding the odd port numbers behind standard 80/443. Unfortunately, it’s poorly documented (if at all), so I’m not sure there will be any opportunity to reverse-engineer it and map it to LTM.

So, if anyone has any direct experience with WebSphere, or more specifically the IBM SCORE application and can pass on any insights, it would be appreciated."

hmmmm, I think we might be able to do something here...

The Apache Plugin

The "Apache webserver plugin" used by WebSphere is an XML file that defines the server clusters, the services they provide, and the URI's which should be forwarded to each cluster. The items in the plugin file that interest us are the UriGroup, ServerCluster and Route definitions. UriGroup statements group selected URIs together so that Route statements may be used to direct requests to a specific ServerCluster based on the URIs requested.

URI Groups

Since the functionality we wish to replace is the determination of which service will receive requests for specific URI's, we will start with the definition of the groups of URI's -- the "UriGroup" definitions in the XML file:

   <UriGroup Name="default_host_WebSphere_Portal_URIs">
      <Uri AffinityCookie="JSESSIONID" AffinityURLIdentifier="jsessionid" Name="/wps/PA_1_0_6D/*"/>
      <Uri AffinityCookie="JSESSIONID" AffinityURLIdentifier="jsessionid" Name="/wps/PA_1_0_6E/*"/>
      <Uri AffinityCookie="JSESSIONID" AffinityURLIdentifier="jsessionid" Name="/wps/PA_1_0_6C/*"/>
      <Uri AffinityCookie="JSESSIONID" AffinityURLIdentifier="jsessionid" Name="/wps/*"/>
      <Uri AffinityCookie="JSESSIONID" AffinityURLIdentifier="jsessionid" Name="/wsrp/*"/>
      <Uri AffinityCookie="JSESSIONID" AffinityURLIdentifier="jsessionid" Name="/wps/content/*"/>
      <Uri AffinityCookie="JSESSIONID" AffinityURLIdentifier="jsessionid" Name="/wps/pdm/*"/>
      ...
   </UriGroup>
...
   <UriGroup Name="default_host_Server_Cluster_URIs">
      <Uri AffinityCookie="JSESSIONID" AffinityURLIdentifier="jsessionid" Name="/snoop/*"/>
      <Uri AffinityCookie="JSESSIONID" AffinityURLIdentifier="jsessionid" Name="/hello"/>
      <Uri AffinityCookie="JSESSIONID" AffinityURLIdentifier="jsessionid" Name="/hitcount"/>
      <Uri AffinityCookie="JSESSIONID" AffinityURLIdentifier="jsessionid" Name="*.jsp"/>
      <Uri AffinityCookie="JSESSIONID" AffinityURLIdentifier="jsessionid" Name="*.jsv"/>
      <Uri AffinityCookie="JSESSIONID" AffinityURLIdentifier="jsessionid" Name="*.jsw"/>
      <Uri AffinityCookie="JSESSIONID" AffinityURLIdentifier="jsessionid" Name="/j_security_check"/>
      <Uri AffinityCookie="JSESSIONID" AffinityURLIdentifier="jsessionid" Name="/ibm_security_logout"/>
      <Uri AffinityCookie="JSESSIONID" AffinityURLIdentifier="jsessionid" Name="/servlet/*"/>
      <Uri AffinityCookie="JSESSIONID" AffinityURLIdentifier="jsessionid" Name="/ivt/*"/>
      ...
   </UriGroup>

The UriGroup definition contains URI strings with glob-style pattern matching (which will come in handy later). All URIs within each group are intended to use the same ServerCluster.

Server Clusters

Four application services are defined by the named ServerCluster definitions. Each physical node has two different services, and each service has two ports (one for http and one for https). Looking at the bolded items in the XML file snip below, you can see the service WebSphere_Portal is defined to run on ports 9081 (http) and 9444 (https) on server1.domain.com and server2.domain.com, and the service Server_Cluster is defined on ports 9080 (http) and 9443 (https) on both nodes:

   <ServerCluster CloneSeparatorChange="false" LoadBalance="Round Robin" Name="WebSphere_Portal"...
      <Server CloneID="12xx2868r" ConnectTimeout="0" ExtendedHandshake="false" LoadBalanceWeight="2" ...
         <Transport Hostname="server1.domain.com" Port="9081" Protocol="http"/>
         <Transport Hostname="server1.domain.com" Port="9444" Protocol="https">
            <Property Name="keyring" Value="D:\IBM\WebSphere\AppServer\etc\plugin-key.kdb"/>
            <Property Name="stashfile" Value="D:\IBM\WebSphere\AppServer\etc\plugin-key.sth"/>
         </Transport>
      </Server>
      <Server CloneID="12vxx4xx3" ConnectTimeout="0" ExtendedHandshake="false" LoadBalanceWeight="2" ...
         <Transport Hostname="server2.domain.com" Port="9081" Protocol="http"/>
         <Transport Hostname="server2.domain.com" Port="9444" Protocol="https">
            <Property Name="keyring" Value="D:\IBM\WebSphere\DM\etc\plugin-key.kdb"/>
            <Property Name="stashfile" Value="D:\IBM\WebSphere\DM\etc\plugin-key.sth"/>
         </Transport>
      </Server>
      <PrimaryServers>
         <Server Name="WebSphere_Portal_1"/>
         <Server Name="WebSphere_Portal_2"/>
      </PrimaryServers>
   </ServerCluster> 

   <ServerCluster CloneSeparatorChange="false" LoadBalance="Round Robin" Name="Server_Cluster" ...
      <Server ConnectTimeout="0" ExtendedHandshake="false" MaxConnections="-1" Name="server01" ...
         <Transport Hostname="server1.domain.com" Port="9080" Protocol="http"/>
         <Transport Hostname="server1.domain.com" Port="9443" Protocol="https">
            <Property Name="keyring" Value="D:\IBM\WebSphere\DM\etc\plugin-key.kdb"/>
            <Property Name="stashfile" Value="D:\IBM\WebSphere\DM\etc\plugin-key.sth"/>
         </Transport>
      </Server>
      <Server ConnectTimeout="0" ExtendedHandshake="false" MaxConnections="-1" Name="server2" ...
         <Transport Hostname="server2.domain.com" Port="9080" Protocol="http"/>
         <Transport Hostname="server2.domain.com" Port="9443" Protocol="https">
            <Property Name="keyring" Value="D:\IBM\WebSphere\DM\etc\plugin-key.kdb"/>
            <Property Name="stashfile" Value="D:\IBM\WebSphere\DM\etc\plugin-key.sth"/>
         </Transport>
      </Server>
      <PrimaryServers>
         <Server Name="Server_Cluster_1"/>
         <Server Name="Server_Cluster_2"/>
      </PrimaryServers>
   </ServerCluster>

The physical nodes (Transport definitions) are added as FQDNs (server1.domain.com and server2.domain.com), so you will need to resolve names to the actual IP addresses to create your server pools.

Route Statements

Route statements correlate a UriGroup with the corresponding ServerCluster:

<Route ServerCluster="Server_Cluster" UriGroup="default_host_Server_Cluster_URIs" VirtualHostGroup="default_host"/>
   ...
   <Route ServerCluster="WebSphere_Portal" UriGroup="default_host_WebSphere_Portal_URIs" VirtualHostGroup="default_host"/>

In this case, any request for a URI in the group default_host_Server_Cluster_URIs will be routed to the Server_Cluster pool, and requests for those URI's in the group default_host_WebSphere_Portal_URIs will be routed to the WebSphere_Portal pool.

The LTM Configuration

Now that we have a better understanding of what the plugin XML file defines, we can build the corresponding LTM configuration:

server pools
the iRule that selects them
persistence, ssl and http profiles
and the virtual servers that tie them all together to accept HTTP and HTTPS requests

Pools

Look up the FQDNs provided in the XML file to create the required server pools with pool members on the indicated IP addresses and ports. In most cases, HTTPS traffic will be decrypted at LTM and forwarded to the servers over HTTP, so only the 2 HTTP pools will be required. Assuming the hostnames server1.domain.com and server2.domain.com resolve to 192.168.100.1 and 192.168.100.2, we would create the following pools:

pool Server_Cluster_http {
   member 192.168.100.1:9080
   member 192.168.100.2:9080
}
pool WebSphere_Portal_http {
   member 192.168.100.1:9081
   member 192.168.100.2:9081
}

If traffic will be re-encrypted, create the HTTPS pools as well:

pool Server_Cluster_https {
   member 192.168.100.1:9443
   member 192.168.100.2:9443
}
pool WebSphere_Portal_https {
   member 192.168.100.1:9444
   member 192.168.100.2:9444
}

Examine the Server definitions in the XML file for other pool member settings that might be relevant, such as ratio (LoadBalanceWeight in the XML file) and connection limits.

SSL Profile

LTM must decrypt HTTP requests to manage them under this configuration. You can either offload SSL to LTM completely, or decrypt and re-encrypt HTTPS requests. In either case, create a clientssl profile containing a certificate & key pair for the virtual server hostname. If you are offloading SSL (HTTPS traffic will be decrypted at LTM and forwarded to the servers over HTTP), that's all you need for SSL. If instead you will be re-encrypting, create a serverssl profile to handle the re-encryption task.

HTTP Profile

If you are offloading SSL, create a custom HTTP profile with the "Rewrite Redirects" option set to "All", allowing the system to rewrite any self-referencing server-set redirects to the proper protocol scheme.

Persistence

Default cookie persistence is the simplest option you can choose here. Noting the references in the XML file to the AffinityCookie named JSESSIONID, you can alternatively enable JSESSIONID persistence with another simple iRule found in the DevCentral codeshare: JSESSIONID Persistence

iRule

We will use a switch statement in an iRule to replicate the actions that the Route statements define to the Apache webserver. The switch statement will contain the mappings of the UriGroup definitions to the pools defined in the corresponding Route statements. Using a switch statement in favor of a Data Group List provides the same capabilty for partial glob-style URI matching as that used in the UriGroup definitions. So consider the following UriGroup and Route definitions:

   <UriGroup Name="default_host_WebSphere_Portal_URIs">
      <Uri AffinityCookie="JSESSIONID" AffinityURLIdentifier="jsessionid" Name="/wps/*"/>
      <Uri AffinityCookie="JSESSIONID" AffinityURLIdentifier="jsessionid" Name="/wsrp/*"/>
      <Uri AffinityCookie="JSESSIONID" AffinityURLIdentifier="jsessionid" Name="/wps/content/*"/>
      <Uri AffinityCookie="JSESSIONID" AffinityURLIdentifier="jsessionid" Name="/wps/pdm/*"/>...
   <Route ServerCluster="WebSphere_Portal" UriGroup="default_host_WebSphere_Portal_URIs" VirtualHostGroup="default_host"/>
...
   <UriGroup Name="default_host_Server_Cluster_URIs">
      <Uri AffinityCookie="JSESSIONID" AffinityURLIdentifier="jsessionid" Name="/snoop/*"/>
      <Uri AffinityCookie="JSESSIONID" AffinityURLIdentifier="jsessionid" Name="/hello"/>
      <Uri AffinityCookie="JSESSIONID" AffinityURLIdentifier="jsessionid" Name="/hitcount"/>
      <Uri AffinityCookie="JSESSIONID" AffinityURLIdentifier="jsessionid" Name="*.jsp"/>
      <Uri AffinityCookie="JSESSIONID" AffinityURLIdentifier="jsessionid" Name="*.jsv"/>
      <Uri AffinityCookie="JSESSIONID" AffinityURLIdentifier="jsessionid" Name="*.jsw"/>
...
   <Route ServerCluster="Server_Cluster" UriGroup="default_host_Server_Cluster_URIs" VirtualHostGroup="default_host"/>

Here is an iRule that replicates this definition to handle HTTP requests, mapping all of the URI strings for HTTP requests, including the embedded wildcards, to the corresponding HTTP pool:

when HTTP_REQUEST {
   switch -glob [string tolower [HTTP::uri]] {
      "/wsrp/*" -
      "/wps/content/*" -
      "/wps/pdm/*" -
      "/wps/*" { pool WebSphere_Portal_http }

      "/snoop/*" -
      "/hello" -
      "/hitcount" -
      "*.jsp" -
      "*.jsv" -
      "*.jsw" { pool Server_Cluster_http }
   }
}

The "-" after a URI means to execute the next defined script body. The blank lines between groups are added for readability, and are not required. Adding a new URI is as simple as duplicating a line in the appropriate group and changing the URI string. Since the switch command will fall out on the first match, more specific matches must be listed before more general ones matching the same patterns with different script bodies, so thorough testing and some experimentation may be required if you have different patterns that match in both groups. (For instance if you had the URI /wps/randompath/myscript.jsp, it would match both /wps/* and *.jsp, but would be sent to WebSphere_Portal_http since it matched first.)

If LTM is offloading encryption, the iRule above would work for both HTTP and HTTPS requests, since the decision is based only on the URI. If LTM is re-encrypting HTTPS requests to the backend servers, we will need a way to send HTTP requests to the HTTP pools, and HTTPS requests to the corresponding HTTPS pools after re-encrypting. The iRule above can be enhanced to check the destination port of the request to see if it was HTTP or HTTPS, then select the appropriate pool based on URI and protocol scheme:

when HTTP_REQUEST {
   switch -glob [string tolower [HTTP::uri]] {
      "/wsrp/*" -
      "/wps/content/*" -
      "/wps/pdm/*" 
      "/wps/*" { if [TCP::local_port == 80] }{ pool WebSphere_Portal_http } else { pool WebSphere_Portal_https }

      "/snoop/*" -
      "/hello" -
      "/hitcount" -
      "*.jsp" -
      "*.jsv" -
      "*.jsw" { if [TCP::local_port == 80] }{ pool Server_Cluster_https } else { pool Server_Cluster_https }
   }
}

Alternatively, you could duplicate the HTTP iRule for the HTTPS virtual server and simply replace the pool selections with those appropriate for HTTPS. The resulting iRule is slightly more efficient, since the destination port test isnot required, but it does require maintaining 2 spearate versions of essentially the same iRule.

Virtual Servers

Finally, define a virtual server for HTTP on port 80 and another for HTTPS on port 443. To each, apply the persistence profile and the appropriate routing iRule. To the HTTPS virtual, apply also the clientssl profile and the custom http profile. (Do NOT apply either to the HTTP virtual server or traffic will not flow as expected.) In the ServerCluster definition, we see the load balancing method is RoundRobin, so we will choose that method here. Examine the ServerCluster definition for other virtual server settings that might be relevant. Once you have associated all the objects with the virtual server, you are ready to test the application without the Apache webserver plugin.

Get the Flash Player to see this player.

20080729-ReplacingWebSphereApachePlugin.mp3

Published Jul 29, 2008

Version 1.0

Was this article helpful?

12 Comments

giltjr
Nimbostratus
Jul 31, 2008
Our company is new to F5 and last April we started a project to use the F5 instead of the plugin. It took me two months to figure out all of this stuff and basically what you have it was I did. Of course since we were new to the F5, I had to learn how to do things on the F5 and learn iRules (I had never seen TCL before). I'm glad to see that what I did was the right way to do it.

One point though. There is a problem with using the iRule to use the JSESSION cookie for session persistence in the case where you have multiple pools. An fairly easy modification to that iRule is instead of using the full cookie as the persist key is to only use the 1st 26 positions, all the way up to the ":".

The issue is that the cookie has the jessionid that is unique to this session and then a unique identifier is added to the end for each WebSphere server you end up using. With two pools you have two servers. So as you get switched to another server the cookie value changes.

We have four pools, so you could get sent to four different servers. Each server tags the end of the JSESSION cookie with their unique id and thus the cookie value gets changed.

This all assumes that the WebSphere Servers are setup to share the JSESSION cookie. If they are not, then you get into real trouble.
Deb_Allen_18
Historic F5 Account
Jul 31, 2008
Good points, thanks for the comment.

Looks like you could use the logic from the ASPSessionID Persistence iRule to build that solution for JSessionID: http://devcentral.f5.com/wiki/default.aspx/iRules/ASP_SessionID_Persistence.html
Simon_83666
Nimbostratus
Aug 07, 2008
I'm a WebSphere admin and I've been experimenting with replacing the WAS plugin with some config on LTM. I also noticed the following issues:

1) for WebSphere V6, if you only intend to use port 80 on the Virtual you would need an iRule similar to the following to stop the client request being redirected to the Websphere port - 908x , otherwise, a virtual with the corresponding 908x port would need to be created and that doesn't look very nice on the client's browser.

when HTTP_RESPONSE {

if { [HTTP::is_redirect] } {

log local0. "Request redirected. Original value: [HTTP::header value Location], updated value: [string map -nocase {9081 80 9082 80} [HTTP::header value Location]]"

Do the update, replacing port 9080 with 80

HTTP::header replace Location [string map -nocase {9081 80 9082 80} [HTTP::header value Location]]

}

}

2) For WebSpehre V5, one would need to add additional virtual host alias on the Websphere setup so it accepts traffic on the 908x port with the real IP of the WebSphere server. I also had to add a stream profile to correct any reference to the 908x port on the HTTP response ( the above rule for V6 redirect does not work). I'm not sure whether this is application specific problem or an issue across all Websphere V5 installation.
giltjr
Nimbostratus
Aug 07, 2008
deb:

Thanks for pointing me to the ASP iRule and that is basically what I did.

sh710:

The redirect issue is not isolated to WebSphere. It is actually an issue anytime the backend server does not listen on port 80.

With your WebSphere V5 issue it is most likely an issue in any WebSphere or J2EE/Java Server.

My guess is your applciations is using dynamically generated HTML using system varaibles. So the HTML has the host name and port that WebSphere is using, but that is not the host name that is used to get to the F5. So, as you did, you need to create a stream profile that replaces what WebSphere puts into the html code with what is needed to get back to F5.

I know I had this issue where we had a page that was dynamically created that had variables that were passed to a Java applet. Since the host name and port in the web page did not match the host name in the browsers URL, the applet did not work. So, one stream profile later and everything worked.
Simon_83666
Nimbostratus
Aug 07, 2008
giltjr,

Thanks for the comments and the confirmation for the use of stream profile. However, I cannot 100% agree with you on the port redirect - we also tested using the LTM to distribute traffic among two IBM HTTP Server (Apache) where they're listening on ports such as 908x , and the redirect rule was not needed to setup the virtual to listen on 80.

Simon
giltjr
Nimbostratus
Aug 08, 2008
I guess I was not clear. Anytime the backend server, of anytime, issues redirects and the pages are dynamically built you will need the redirect rule. If there is no redirects or the pages are static and do not refer to the real port that the backend server is listening on, you do not need the redirect rule.
Chris_Schaerli_
Nimbostratus
Sep 10, 2008
Couple questions for those who have set this up before. What kind of overhead will something like this add to the LTM? Are there any drawbacks you have found to this kind of setup?
Simon_83666
Nimbostratus
Sep 12, 2008
We've explored this option but at the end we decided to use a local IHS on the WAS box. the main reason for that is the ease of administration on the plugin if you have hundreds of websites with different context roots.
lcarico5_53817
Nimbostratus
Nov 25, 2009
There's an easier way to do this. Go ahead and set up the pool directly to the java nodes (IP/port). Depending upon your install this could be a large pool as there are generally several nodes per physical server.

We're almost done!

Next, go into the Websphere admin console and check the "trusthostheadersport".

Finished.
giltjr
Nimbostratus
Nov 25, 2009
Do what?