BigSuds - bigsuds.ConnectionError: BadStatusLine: ''

This is a new one on me. I can't reproduce with the usual bad password, bad interface parameter and so on. This looks like an exception being raised by httplib, not bigsuds. According to the docs it is because the server responded with a status code that httplib cannot understand.

Do other calls work, such as b.Networking.Interfaces.get_list() ?

Hi mimlo

Agree, it does seem to be httplib getting a bad respone. Most commands work, (Networking.interfaces.get_list() does, as a matter of fact that's where the list of interfaces come from)

A the moment I am getting the same error from Networking.Interfaces.get_statistics and LocalLB.Pool.get_active_member_count

The version of F5 is 11.4.0 (Build 2419.0)

Confirmed working with python2.7 as well, so unless something is wrong in 11.4.0 I have no idea. Below is what I tested with in ipython

b = bigsuds.BIGIP(hostname=hostname, username=username, password=password)
interfaces = b.Networking.Interfaces.get_list()
b.Networking.Interfaces.get_statistics(interfaces)

This returned the expected interface statistics

pools = b.LocalLB.Pool.get_list()
b.LocalLB.Pool.get_active_member_count(pools)

This returned a list of active pool members for each pool supplied.

Anything else different? Any partitions/folders involved?

Some feedback, just tested on another F5 with same software revision and it works fine, so not related to F5 version, but rather to content returned.

To add, I guess this rules out the Python version as well. This works fine on 5 other systems, hmmm

Have you tried restarting the httpd process on the problem box?

Ah, looking for your debug output it seems you are using my F5 graphite collector tool -- https://github.com/mhite/graphite-collectors

I'm looking at the problem myself, too. It appears to be transient. It also seems this is a problem with the API itself returning an empty result code. Not sure what I can do to address this in my code other than retrying a few times.

Also, for what it's worth, I see it thrown by both bigsuds and httplib:

httplib:

Traceback (most recent call last): 
File "/usr/local/bin/f5-agent", line 5, in  
main() 
File "/usr/local/lib/python2.6/dist-packages/graphitecollectors/f5.py", line 768, in main 
args.no_virtual_server, args.no_pool) 
File "/usr/local/lib/python2.6/dist-packages/graphitecollectors/f5.py", line 219, in gather_f5_metrics 
b = b.with_session_id() 
File "/usr/local/lib/python2.6/dist-packages/bigsuds.py", line 118, in with_session_id 
session_id = self.System.Session.get_session_identifier() 
File "/usr/local/lib/python2.6/dist-packages/bigsuds.py", line 313, in __getattr__ 
client = self._client_creator('%s.%s' % (self._name, attr)) 
File "/usr/local/lib/python2.6/dist-packages/bigsuds.py", line 136, in _create_client 
self._password, self._cachedir) 
File "/usr/local/lib/python2.6/dist-packages/bigsuds.py", line 226, in get_client 
cache=cachedir) 
File "/usr/local/lib/python2.6/dist-packages/suds/client.py", line 112, in __init__ 
self.wsdl = reader.open(url) 
File "/usr/local/lib/python2.6/dist-packages/suds/reader.py", line 152, in open 
d = self.fn(url, self.options) 
File "/usr/local/lib/python2.6/dist-packages/suds/wsdl.py", line 136, in __init__ 
d = reader.open(url) 
File "/usr/local/lib/python2.6/dist-packages/suds/reader.py", line 79, in open 
d = self.download(url) 
File "/usr/local/lib/python2.6/dist-packages/suds/reader.py", line 95, in download 
fp = self.options.transport.open(Request(url)) 
File "/usr/local/lib/python2.6/dist-packages/suds/transport/https.py", line 60, in open 
return HttpTransport.open(self, request) 
File "/usr/local/lib/python2.6/dist-packages/suds/transport/http.py", line 62, in open 
return self.u2open(u2request) 
File "/usr/local/lib/python2.6/dist-packages/suds/transport/http.py", line 118, in u2open 
return url.open(u2request, timeout=tm) 
File "/usr/lib/python2.6/urllib2.py", line 397, in open 
response = meth(req, response) 
File "/usr/lib/python2.6/urllib2.py", line 510, in http_response 
'http', request, response, code, msg, hdrs) 
File "/usr/lib/python2.6/urllib2.py", line 429, in error 
result = self._call_chain(*args) 
File "/usr/lib/python2.6/urllib2.py", line 369, in _call_chain 
result = func(*args) 
File "/usr/lib/python2.6/urllib2.py", line 864, in http_error_401 
url, req, headers) 
File "/usr/lib/python2.6/urllib2.py", line 842, in http_error_auth_reqed 
return self.retry_http_basic_auth(host, req, realm) 
File "/usr/lib/python2.6/urllib2.py", line 852, in retry_http_basic_auth 
return self.parent.open(req, timeout=req.timeout) 
File "/usr/lib/python2.6/urllib2.py", line 391, in open 
response = self._open(req, data) 
File "/usr/lib/python2.6/urllib2.py", line 409, in _open 
'_open', req) 
File "/usr/lib/python2.6/urllib2.py", line 369, in _call_chain 
result = func(*args) 
File "/usr/lib/python2.6/urllib2.py", line 1178, in https_open 
return self.do_open(httplib.HTTPSConnection, req) 
File "/usr/lib/python2.6/urllib2.py", line 1143, in do_open 
r = h.getresponse() 
File "/usr/lib/python2.6/httplib.py", line 990, in getresponse 
response.begin() 
File "/usr/lib/python2.6/httplib.py", line 391, in begin 
version, status, reason = self._read_status() 
File "/usr/lib/python2.6/httplib.py", line 355, in _read_status 
raise BadStatusLine(line) 
httplib.BadStatusLine 

This particular exception is getting thrown right after creating the connection handle to the load balancer via bigsuds.

try:
    logging.info("Connecting to BIG-IP and pulling statistics...")
    b = bigsuds.BIGIP(hostname=ltm_host, username=user, password=password)
    logging.info("Requesting session...")
    b = b.with_session_id()
except bigsuds.Connecti , detail:
    logging.critical("Unable to connect to BIG-IP. Details: %s" % pformat(detail))
    sys.exit(1)

You can see I am catching bigsuds.Connecti but I guess I should also catch httplib.Connecti also and retry.

bigsuds:

Traceback (most recent call last): 
File "/usr/local/bin/f5-agent", line 5, in  
main() 
File "/usr/local/lib/python2.6/dist-packages/graphitecollectors/f5.py", line 768, in main 
args.no_virtual_server, args.no_pool) 
File "/usr/local/lib/python2.6/dist-packages/graphitecollectors/f5.py", line 349, in gather_f5_metrics 
tcp_stats = b.System.Statistics.get_tcp_statistics() 
File "/usr/local/lib/python2.6/dist-packages/bigsuds.py", line 430, in wrapped_method 
raise Connecti ('BadStatusLine: %s' % e) 
bigsuds.Connecti : BadStatusLine: 

So in this exception, it is actually being caught by bigsuds and rethrown as a wrapped exception. It is interesting to note that the place where this crashes -- get_tcp_statistics() -- is pretty deep into the metric gathering.

Hey gang,

I've created a temporary branch if you want to test:

https://github.com/mhite/graphite-collectors/tree/connection_error

This just implements some retry logic when a bigsuds Connecti or BadStatusLine from httplib exception is raised. It's not perfect (ie. if you have bad credentials, it will retry since it's the same exception) but I want to see if we can smooth over whatever problem the F5 is having with a simple retry.

Also, for those of you hitting this problem, I am wondering if you are using a local account vs. a remote (ie. LDAP) account for iControl authentication?