Learn F5 Technologies, Get Answers & Share Community Solutions Join DevCentral

Filter by:
  • Solution
  • Technology
Clear all filters
Answers

Error processing HTTP request header

I'm running into an issue on my apache tomcat 8 servers when receiving traffic routed through the F5. We have enabled TLS on the application servers and have setup an SSL Server profile to send the traffic through over HTTPS. The pool is configured to send the traffic to the servers on ports 8443 and 8444. However we are getting the following error from the servers.

2019-03-20 05:26:32,938+0000 INFO org.apache.coyote.http11.Http11Processor service - Error  parsing HTTP request header
Note: further occurrences of HTTP header parsing errors will be logged at DEBUG level.
java.lang.IllegalArgumentException: Invalid character found in method name. HTTP method names must be tokens
    at org.apache.coyote.http11.Http11InputBuffer.parseRequestLine(Http11InputBuffer.java:462)
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:667)
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:802)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1410)
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.lang.Thread.run(Thread.java:748)

I am not certain what to look at on the F5 for how it is formatting/encrypting the traffic before it sends it onto the servers. Any advice on the direction I should take this would be very much appreciated.

0
Rate this Question
Comments on this Question
Comment made 1 month ago by brad9iner 58

I forgot to mention these application servers are on Oracle Linux 7.6.

0

Answers to this Question

placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER

It looks like this error occurs when you send an HTTPS request to a Tomcat server running an HTTP listener.

I got a similar error message from a sample Tomcat Docker image (docker run -it --rm -p 8888:8080 tomcat:alpine) when I sent it an HTTPS request (curl -Ivk https://localhost:8888/).

21-Mar-2019 00:36:25.559 INFO [http-nio-8080-exec-8] org.apache.coyote.http11.Http11Processor.service Error parsing HTTP request header
 Note: further occurrences of HTTP header parsing errors will be logged at DEBUG level.
 java.lang.IllegalArgumentException: Invalid character found in method name. HTTP method names must be tokens
    at org.apache.coyote.http11.Http11InputBuffer.parseRequestLine(Http11InputBuffer.java:428)
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:684)
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:806)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1498)
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.lang.Thread.run(Thread.java:748)

You could confirm this by removing the serverssl profile from the VIP in question.

You could also try connecting to ports 8443 and 8444 on your Tomcat hosts to verify that they are in fact configured to accept HTTPS connections.

If neither of these resolves the issue, you could try sharing a sanitized version of your VIP configuration, along with confirmation that the Tomcat servers are accepting HTTPS connections.

1
Comments on this Answer
Comment made 1 month ago by brad9iner 58

Here is the config we have setup on our tomcat servers:

<?xml version='1.0' encoding='utf-8'?>
`</pre>

<server port="-1" shutdown="SHUTDOWN"></server>

<listener classname="org.apache.catalina.core.AprLifecycleListener" sslengine="on"></listener>
  <listener classname="org.apache.catalina.core.JreMemoryLeakPreventionListener"></listener>
  <listener classname="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener"></listener>
  <listener classname="org.apache.catalina.core.ThreadLocalLeakPreventionListener"></listener>

<service name="Catalina"></service>

<pre>`<!-- http -->
<Connector
    protocol="org.apache.coyote.http11.Http11NioProtocol"
    connectionTimeout="20000"
    URIEncoding="utf-8"
    acceptorThreadCount="2"
    maxThreads="40"
    maxKeepAliveRequests="-1"
    keepAliveTimeout="30000"
    enableLookups="false"
    scheme="https"
    keystorePass="password"
    keystoreFile="~cernercentral_home/keystore.pkcs12"
    keystoreType="PKCS12"
    keyAlias="1"
    proxyPort="443"
    clientAuth="false"
    sslProtocol="TLS"
    SSLEnabled="true"
    secure="true"
    port="8443"
    server="cernercentral_home1"
/>

Which is the same execpt 8444 for our other JVM. We have 2 nodes in this cluster with the same configs. I don't have direct access to the F5 our VIP is configured on. But I had my resource do some testing. When they disabled 8444 in the pool in question for both nodes we are able to get successful responses. But when they disable 8443 and leave 8444 enabled then that is when we start getting "empty reply from server" messages as well as the stack trace in the description.

0
Comment made 1 month ago by brad9iner 58

new update on this, looking further into the F5 with my resource that has access we do have an iRule applied

when SERVER_CONNECTED {
    if {[LB::server port] != 8443 && [LB::server port] != 443} {
        SSL::disable
    }
   }

What would I need to edit on that to make it also look at ports 8444 and 8445? I can't quite find the "or" equivalent for this kind of iRule.

0
Comment made 1 month ago by DennisJann 213

Given your existing iRule, you would enclose the "or" conditions in ()'s like this:

when SERVER_CONNECTED {
   if { ([LB::server port] != 8443 or [LB::server port] != 8444 or [LB::server port] != 8445) && [LB::server port] != 443 } {
      SSL::disable
   }
}

I don't know your specific requirement, but I've generally used SSL::disable in the HTTP_REQUEST event like this:

when HTTP_REQUEST {
   if { [HTTP::path] starts_with "/plaintext-path" } {
      SSL::disable
      pool http_pool
   } else {
      pool https_pool
   }
}

Hope this helps.

1