Demystifying iControl REST Part 6: Token-Based Authentication
iControl REST. It’s iControl SOAP’s baby, brother, introduced back in TMOS version 11.4 as an early access feature but released fully in version 11.5. Several articles on basic usage have been written on iControl REST so the intent here isn’t basic use, but rather to demystify some of the finer details of using the API. This article will cover the details on how to retrieve and use an authentication token from the BIG-IP using iControl REST and the python programming language. This token is used in place of basic authentication on API calls, which is a requirement for external authentication. Note that for configuration changes, version 12.0 or higher is required as earlier versions will trigger an un-authorized error. The tacacs config in this article is dependent on a version that I am no longer able to get installed on a modern linux flavor. Instead, try this Dockerized tacacs+ server for your testing. The Fine Print The details of the token provider are here in the wiki. We’ll focus on a provider not listed there: tmos. This provider instructs the API interface to use the provider that is configured in tmos. For this article, I’ve configured a tacacs server and the BIG-IP with custom remote roles as shown below to show BIG-IP version 12’s iControl REST support for remote authentication and authorization. Details for how this configuration works can be found in the tacacs+ article I wrote a while back. BIG-IP tacacs+ configuration auth remote-role { role-info { adm { attribute F5-LTM-User-Info-1=adm console %F5-LTM-User-Console line-order 1 role %F5-LTM-User-Role user-partition %F5-LTM-User-Partition } mgr { attribute F5-LTM-User-Info-1=mgr console %F5-LTM-User-Console line-order 2 role %F5-LTM-User-Role user-partition %F5-LTM-User-Partition } } } auth remote-user { } auth source { type tacacs } auth tacacs system-auth { debug enabled protocol ip secret $M$Zq$T2SNeIqxi29CAfShLLqw8Q== servers { 172.16.44.20 } service ppp } Tacacs+ Server configuration id = tac_plus { debug = PACKET AUTHEN AUTHOR access log = /var/log/access.log accounting log = /var/log/acct.log host = world { address = ::/0 prompt = "\nAuthorized Access Only!\nTACACS+ Login\n" key = devcentral } group = adm { service = ppp { protocol = ip { set F5-LTM-User-Info-1 = adm set F5-LTM-User-Console = 1 set F5-LTM-User-Role = 0 set F5-LTM-User-Partition = all } } } group = mgr { service = ppp { protocol = ip { set F5-LTM-User-Info-1 = mgr set F5-LTM-User-Console = 1 set F5-LTM-User-Role = 100 set F5-LTM-User-Partition = all } } } user = user_admin { password = clear letmein00 member = adm } user = user_mgr { password = clear letmein00 member = mgr } } Basic Requirements Before we look at code, however, let’s take a look at the json payload requirements, followed by response data from a query using Chrome’s Advanced REST Client plugin. First, since we are sending json payload, we need to add the Content-Type: application/json header to the query. The payload we are sending with the post looks like this: { "username": "remote_auth_user", "password": "remote_auth_password", "loginProviderName": "tmos" } You submit the same remote authentication credentials in the initial basic authentication as well, no need to have access to the default admin account credentials. A successful query for a token returns data like this: { username: "user_admin" loginReference: { link: "https://localhost/mgmt/cm/system/authn/providers/tmos/1f44a60e-11a7-3c51-a49f-82983026b41b/login" }- token: { uuid: "4d1bd79f-dca7-406b-8627-3ad262628f31" name: "5C0F982A0BF37CBE5DE2CB8313102A494A4759E5704371B77D7E35ADBE4AAC33184EB3C5117D94FAFA054B7DB7F02539F6550F8D4FA25C4BFF1145287E93F70D" token: "5C0F982A0BF37CBE5DE2CB8313102A494A4759E5704371B77D7E35ADBE4AAC33184EB3C5117D94FAFA054B7DB7F02539F6550F8D4FA25C4BFF1145287E93F70D" userName: "user_admin" user: { link: "https://localhost/mgmt/cm/system/authn/providers/tmos/1f44a60e-11a7-3c51-a49f-82983026b41b/users/34ba3932-bfa3-4738-9d55-c81a1c783619" }- groupReferences: [1] 0: { link: "https://localhost/mgmt/cm/system/authn/providers/tmos/1f44a60e-11a7-3c51-a49f-82983026b41b/user-groups/21232f29-7a57-35a7-8389-4a0e4a801fc3" }- - timeout: 1200 startTime: "2015-11-17T19:38:50.415-0800" address: "172.16.44.1" partition: "[All]" generation: 1 lastUpdateMicros: 1447817930414518 expirationMicros: 1447819130415000 kind: "shared:authz:tokens:authtokenitemstate" selfLink: "https://localhost/mgmt/shared/authz/tokens/4d1bd79f-dca7-406b-8627-3ad262628f31" }- generation: 0 lastUpdateMicros: 0 } Among many other fields, you can see the token field with a very long hexadecimal token. That’s what we need to authenticate future API calls. Requesting the token programmatically In order to request the token, you first have to supply basic auth credentials like normal. This is currently required to access the /mgmt/shared/authn/login API location. The basic workflow is as follows (with line numbers from the code below in parentheses): Make a POST request to BIG-IP with basic authentication header and json payload with username, password, and the login provider (9-16, 41-47) Remove the basic authentication (49) Add the token from the post response to the X-F5-Auth-Token header (50) Continue further requests like normal. In this example, we’ll create a pool to verify read/write privileges. (1-6, 52-53) And here’s the code (in python) to make that happen: def create_pool(bigip, url, pool): payload = {} payload['name'] = pool pool_config = bigip.post(url, json.dumps(payload)).json() return pool_config def get_token(bigip, url, creds): payload = {} payload['username'] = creds[0] payload['password'] = creds[1] payload['loginProviderName'] = 'tmos' token = bigip.post(url, json.dumps(payload)).json()['token']['token'] return token if __name__ == "__main__": import os, requests, json, argparse, getpass requests.packages.urllib3.disable_warnings() parser = argparse.ArgumentParser(description='Remote Authentication Test - Create Pool') parser.add_argument("host", help='BIG-IP IP or Hostname', ) parser.add_argument("username", help='BIG-IP Username') parser.add_argument("poolname", help='Key/Cert file names (include the path.)') args = vars(parser.parse_args()) hostname = args['host'] username = args['username'] poolname = args['poolname'] print "%s, enter your password: " % args['username'], password = getpass.getpass() url_base = 'https://%s/mgmt' % hostname url_auth = '%s/shared/authn/login' % url_base url_pool = '%s/tm/ltm/pool' % url_base b = requests.session() b.headers.update({'Content-Type':'application/json'}) b.auth = (username, password) b.verify = False token = get_token(b, url_auth, (username, password)) print '\nToken: %s\n' % token b.auth = None b.headers.update({'X-F5-Auth-Token': token}) response = create_pool(b, url_pool, poolname) print '\nNew Pool: %s\n' % response Running this script from the command line, we get the following response: FLD-ML-RAHM:scripts rahm$ python remoteauth.py 172.16.44.15 user_admin myNewestPool1 Password: user_admin, enter your password: Token: 2C61FE257C7A8B6E49C74864240E8C3D3592FDE9DA3007618CE11915F1183BF9FBAF00D09F61DE15FCE9CAB2DC2ACC165CBA3721362014807A9BF4DEA90BB09F New Pool: {u'generation': 453, u'minActiveMembers': 0, u'ipTosToServer': u'pass-through', u'loadBalancingMode': u'round-robin', u'allowNat': u'yes', u'queueDepthLimit': 0, u'membersReference': {u'isSubcollection': True, u'link': u'https://localhost/mgmt/tm/ltm/pool/~Common~myNewestPool1/members?ver=12.0.0'}, u'minUpMembers': 0, u'slowRampTime': 10, u'minUpMembersAction': u'failover', u'minUpMembersChecking': u'disabled', u'queueTimeLimit': 0, u'linkQosToServer': u'pass-through', u'queueOnConnectionLimit': u'disabled', u'fullPath': u'myNewestPool1', u'kind': u'tm:ltm:pool:poolstate', u'name': u'myNewestPool1', u'allowSnat': u'yes', u'ipTosToClient': u'pass-through', u'reselectTries': 0, u'selfLink': u'https://localhost/mgmt/tm/ltm/pool/myNewestPool1?ver=12.0.0', u'serviceDownAction': u'none', u'ignorePersistedWeight': u'disabled', u'linkQosToClient': u'pass-through'} You can test this out in the Chrome Advanced Rest Client plugin, or from the command line with curl or any other language supporting REST clients as well, I just use python for the examples well, because I like it. I hope you all are digging into iControl REST! What questions do you have? What else would you like clarity on? Drop a comment below.Parsing F5 BIG-IP LTM DNS profile statistics and extracting values with Python
Introduction Hello there! Arvin here from the F5 SIRT. A little while ago, I published F5BIG-IP Advanced Firewall Manager (AFM) DNS NXDOMAIN Query Attack Type Walkthroughpart one and two, where I went through the process of reviewing BIG-IP LTM DNS profile statistics and used it to set BIG-IP AFM DNS NXDOMAIN Query attack type detection and mitigation thresholds with the goal of mitigating DNS NXDOMAIN Floods. In this article, I continue to look atBIG-IP LTM DNS profile statistics, find ways of parsing it and to extract specific values of interest through Python. Python for Network Engineers Python has emerged as a go-to language for network engineers, providing a powerful and accessible toolset for managing and automating network tasks. Known for its simplicity and readability, Python enables network engineers to script routine operations, automate repetitive tasks, and interact with network devices through APIs. With extensive libraries and frameworks tailored to networking, Python empowers engineers to streamline configurations, troubleshoot issues, and enhance network efficiency. Its versatility makes it an invaluable asset for network automation, allowing engineers to adapt to evolving network requirements and efficiently manage complex infrastructures. Whether you're retrieving data, configuring devices, or optimizing network performance, Python simplifies the process for network engineers, making it an essential skill in the modern networking landscape. The Tools ChatGPT3.5 The "Python for Network Engineers" intro came from ChatGPT3.5 [:)]. Throughout this article, the python coding "bumps" avoidance and approaches came from ChatGPT3.5. Instead of googling, I asked ChatGPT "a lot" so I could get the python scripts to get the output I wanted. https://chat.openai.com/ Visual Studio Code UsingVisual Studio Code (VSCode) to build the scripts was very helpful, especially the tooltip / hints which tells me and help make sense of the available options for the modules used and describing the python data structures. Python 3.10 (From ChatGPT)Python 3.10, the latest version of the Python programming language, brings forth new features and optimizations that enhance the language's power and simplicity. With Python's commitment to readability and ease of use, version 3.10 introduces structural pattern matching, allowing developers to express complex logic more concisely. Other improvements include precise types, performance enhancements, and updates to syntax for cleaner code. Python 3.10 continues to be a versatile and accessible language, serving diverse needs from web development to data science and automation. Its vibrant community and extensive ecosystem of libraries make Python 3.10 a top choice for developers seeking both efficiency and clarity in their code. Python Script - extract DNS A requests value from LTM DNS profile statistics iControl REST output This python script will extract DNS A requests value from LTM DNS profile statistics iControl REST output. Python has many modules that can be used to simplify tasks. iControl REST output is in json format, so as expected, I used the json module. I wanted to format the output data in csv format so the extracted data can later be used in other tools that consume csv formatted data, thus, I used the csv module. I also used the os, time/datetime and tabulate modules for working with the filesystem (I used a Windows machine to run Python and VSCode) to write csv files. Create variables with date and time information that will be used in formatting the csv file name, keep track of the "A record requests" value at script execution, and present a tabulated output of the captured time and data when the script is executed. I also used BIGREST module to query/retrieve the "show ltm dns profile <DNS profile> statistics" instead of getting the output from iControl REST request sent through other methods. https://bigrest.readthedocs.io/introduction.html https://bigrest.readthedocs.io/bigip_show.html Here is the sample script output Here is the sample CSV-formatted data in a csv file with timestamp of the script run I created a github repository for the Python script and its sample script output and csv data see https://github.com/arvfopa/scripts/tree/main https://github.com/arvfopa/scripts/blob/main/extractAreq- Python Script "extractAreq" https://github.com/arvfopa/scripts/blob/main/extractAreq_output- "extractAreq" output Bumps along the way BIGREST module I initially encountered an error 'certificate verify failed: self signed certificate' when provided only the IP address and credentials used in the BIGIP class of the bigrest.bigip python module raise SSLError(e, request=request) requests.exceptions.SSLError: HTTPSConnectionPool(host='IP address', port=443): Max retries exceeded with url: /mgmt/shared/echo-query (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1007)'))) This is fixed by setting "session_verify" argument of the BIGIP class to "false" to disables SSL certificate validation device = BIGIP("<IP address>", "<username>", "<password>" , session_verify=False) https://bigrest.readthedocs.io/utils.html I also received this error "TypeError: Object of type RESTObject is not JSON serializable" raise TypeError(f'Object of type {o.__class__.__name__} ' TypeError: Object of type RESTObject is not JSON serializable I reread the BIGREST documentation and found that the output is a python dictionary and can is printed in json format.I rechecked the script and removed the json related syntax and module, and the script runs fine and still gets the same output. I updated the script on github with the simplified changes. https://bigrest.readthedocs.io/restobject.html Here's a sample of the RESTObject properties dictionary values. Plenty of data can be extracted. Example, "clientside.pktsIn" value, a virtual server statistic, can be observed and should detection and mitigation thresholds for AFM, say, UDP protocol DoS attack type, need to be set. This value can be monitored over time to understand how many packets a virtual server receives. ============== {'clientside.bitsIn': {'value': 0}, 'clientside.bitsOut': {'value': 0}, 'clientside.curConns': {'value': 0}, 'clientside.evictedConns': {'value': 0}, 'clientside.maxConns': {'value': 0}, 'clientside.pktsIn': {'value': 0}, 'clientside.pktsOut': {'value': 0}, 'clientside.slowKilled': {'value': 0}, 'clientside.totConns': {'value': 0}, 'cmpEnableMode': {'description': 'all-cpus'}, 'cmpEnabled': {'description': 'enabled'}, 'csMaxConnDur': {'value': 0}, 'csMeanConnDur': {'value': 0}, 'csMinConnDur': {'value': 0}, 'destination': {'description': '10.73.125.137:53'}, 'ephemeral.bitsIn': {'value': 0}, 'ephemeral.bitsOut': {'value': 0}, 'ephemeral.curConns': {'value': 0}, 'ephemeral.evictedConns': {'value': 0}, ============== CSV filename issue I encountered thiserror,"OSError: [Errno 22] Invalid argument: 'dns_stats_2023-12-07_18:01:11.csv". This is related to writing of the output csv file. I asked ChatGPT what this was about andwas providedwith thisanswer. ======================= The error you're encountering, "[Errno 22] Invalid argument," typically suggests an issue with the filename or file path. In this case, it seems to be related to the colon (':') character in the filename. In some operating systems (like Windows), certain characters are not allowed in filenames, and ":" is one of them. Since you're including a timestamp in the filename, it's common to replace such characters with alternatives. You can modify the timestamp format to use underscores or hyphens instead of colons. ==================== The timestamp variable in the script stores the value of the formatted timestamp that will be used in the filename. It initially used a colon (:) as the hour/min/sec separator. It was changed to dash (-) so it would not encounter this error. timestamp = datetime.now().strftime("%Y-%m-%d_%H-%M-%S") Checking the file csv file if it exists The function "write_to_csv" writes the time of collection (formatted_date) andextracted value of DNS A requests count (AReqsvalue). It is called every 10 seconds [time.sleep(10)] for a minute [end_time = time.time() + 60] and writes the output to a file in csv format. The "tabulate" function formats the output of the script. Getting the arrangement of the execution wrong would result in unexpected output. The "file_exists" check to write the "headers" was added to make sure that the "headers" are only written once. "write_to_csv" function ======================== def write_to_csv(formatted_date, AReqsvalue): current_datetime = datetime.now() formatted_date = current_datetime.strftime("%Y-%m-%d %H:%M:%S") csv_filename = f"dns_stats_{timestamp}.csv" headers = ["Date", "DNS A requests"] stats = [[formatted_date, AReqsvalue]] file_exists = os.path.exists(csv_filename) print(tabulate(stats, headers, tablefmt="fancy_grid")) with open(csv_filename, mode='a', newline='') as file: writer = csv.writer(file) if not file_exists: writer.writerow(headers) writer.writerows(stats) end_time = time.time() + 60 while time.time() < end_time: write_to_csv(formatted_date, AReqsvalue) time.sleep(10) ========================== Using ChatGPT In building this script, I usedChatGPT "a lot" and it helped to provide make more sense of the module options, errors and sample scripts. It has been a helpful tool. It tracks your conversation/questions to it and kind of understands the context/topic. "ChatGPT can make mistakes. Consider checking important information." is written at the bottom of the page. The data I used in this article are data from a lab environment.That said, when using public AI/ML systems, we should ensure we do not send any sensitive, proprietary information. Organizations have rolled out their own privacy policies when using AI/ML systems, be sure to follow your own organization's policies. Conclusion Using python to parse and extract values of interest from LTM profile statistics offers flexibility and hopefully simplifying observing and recording these data for further use. In particular, setting values for BIG-IP AFM DoS Detection and Mitigation thresholds will be easier if such data has been observed as it, in my opinion, is the "pulse" of the traffic the BIG-IP processes. As noted in the sample json data output, we can see many statistics that can be reviewed and observed to make configuration changes relevant, for example, mitigating a connection spike by setting a VS connection/rate limit. We can look at the "Conns" values and use the observed values to set a connection limit. Example: 'clientside.curConns': {'value': 0}, 'clientside.evictedConns': {'value': 0}, 'clientside.maxConns': {'value': 0}, 'clientside.totConns':{'value': 0} That's it for now. I hope this article has been educational. The F5 SIRT createssecurity-related content posted here in DevCentral, sharing the team's security mindset and knowledge. Feel free to view the articles that are tagged withthe following: F5 SIRT series-F5SIRT-this-week-in-security TWIS
iControl REST Python Requests module
While iControl REST is great and pretty robust given how much we use it. But I come to the forum today to see if anyone has any experience with the Python requests module and the underlying urllib3 module. When I do have problems with iControl its often with things like the following: Here I've increased the timeout to 5.0 seconds but still get read timeouts. HTTPSConnectionPool(host='redacted', port=443): Read timed out. (read timeout=5.0) I also often see this message and, no I don't have a proxy in front of the F5. However, I know the the F5 proxies the REST call (Caused by ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 503 Service Unavailable', Thanks!Demystifying iControl REST Part 1 - Understanding the request URI
iControl REST. It’s iControl SOAP’s baby brother, introduced back in TMOS version 11.4 as an early access feature but was released fully in version 11.5. Several articles on basic usage have been written on iControl REST (see the resources at the bottom of this article) so the intent here isn’t basic use, but rather to demystify some of the finer details of using the API. This article is the first of a four part series and will cover how the URI path plays a role in how the API functions. Examining the REST interface URI With iControl SOAP, all the interfaces and methods are defined in the WSDLs. With REST, the bulk of the interface and method is part of the URI. Take for example a request to get the list of iRules on a BIG-IP. https://x.x.x.x/mgmt/tm/ltm/rule With a simple get request with the appropriate headers and credentials (see below for curl and python examples,) this breaks down to the Base URI, the Module URI, and the Sub-Module URI. #python example >>> import requests >>> import json >>> b = requests.session() >>> b.auth = ('admin', 'admin') >>> b.verify = False >>> b.headers.update({'Content-Type':'application/json'}) >>> b.get('https://172.16.44.128/mgmt/tm/ltm/rule') #curl example curl -k -u admin:admin -H “Content-Type: application/json” -X GET https://172.16.44.128/mgmt/tm/ltm/rule Beyond the Sub-Module URI is the component URI. https://x.x.x.x/mgmt/tm/ltm/rule/testlog The component URI is the spot that snags most beginners. When creating a new configuration object on an F5, it is fairly obvious which URI to use for the REST call. If you were creating a new virtual server, it would be /mgmt/tm/ltm/virtual, while a new pool would be /mgmt/tm/ltm/pool. Thus a REST call to create a new pool on an LTM with the IP address 172.16.44.128 would look like the following: #python example >>> import requests >>> import json >>> b = requests.session() >>> b.auth = ('admin', 'admin') >>> b.verify = False >>> b.headers.update({'Content-Type':'application/json'}) >>> pm = ['192.168.25.32:80', '192.168.25.33:80'] >>> payload = { } >>> payload['kind'] = 'tm:ltm:pool:poolstate' >>> payload['name'] = 'tcb-pool' >>> payload['members'] = [ {'kind': 'ltm:pool:members', 'name': member } for member in pm] >>> b.post('https://172.16.44.128/mgmt/tm/ltm/pool', data=json.dumps(payload)) #curl example curl -k -u admin:admin -H "Content-Type: \ application/json" -X POST -d \ '{"name":"tcb-pool","members":[ \ {"name":"192.168.25.32:80","description":"first member”}, \ {"name":"192.168.25.33:80","description":"second member”} ] }' \ https://172.16.44.128/mgmt/tm/ltm/pool The call would create the pool “tcb-pool” with members at 192.168.25.32:80 and 192.168.25.33:80. All other aspects of the pool would be the defaults. Thus this pool would be created in the Common partition, have the Round Robin load balancing method, and no monitor.At first glance, some programmers would then modify the pool “tcb-pool” with a command like the following (same payload in python, added the .text attribute to see the error response on the requests object): #python example >>> b.put('https://172.16.44.128/mgmt/tm/ltm/pool', data=json.dumps(payload)).text #return data u'{"code":403,"message":"Operation is not supported on component /ltm/pool.","errorStack":[]}' #curl example curl -k -u admin:admin -H "Content-Type: \ application/json" -X PUT -d \ '{"name":"tcb-pool","members":[ \ {"name":"192.168.25.32:80","description":"first member"} {"name":"192.168.25.33:80","description":"second member"} ] }' \ https://172.16.44.128/mgmt/tm/ltm/pool #return data {"code":403,"message":"Operation is not supported on component /ltm/pool.","errorStack":[]} You can see because they use the sub module URI used to create the pool this returns a 403 error. The command fails because one is trying to modify a specific pool at the generic pool level. Now that the pool exists, one must use the URI that specifies the pool. Thus, the correct command would be: #python example >>> b.put('https://172.16.44.128/mgmt/tm/ltm/pool/~Common~tcb-pool', data=json.dumps(payload)) #curl example curl -k -u admin:admin -H "Content-Type: \ application/json" -X PUT -d \ '{"members":[ \ {"name":"192.168.25.32:80","description":"first member"} {"name":"192.168.25.33:80","description":"second member"} ] }' \ https://172.16.44.128/mgmt/tm/ltm/pool/~Common~tcb-pool We add the ~Common~ in front of the pool name because it is in the Common partition. However, this would also work with https://172.16.44.128/mgmt/tm/ltm/pool/tcb-pool. It is just good practice to explicitly insert the partition name since not all configuration objects will be in the default Common partition. Because we specify the pool in the URI, it is no longer necessary to have the “name” key value pair. In practice, programmers usually correctly modify items such as virtual servers and pools. However, we encounter this confusion much more often in configuration items that are ifiles. This may be because the creation of configuration items that are ifiles is a 3-step process. For instance, in order to create an external data group, one would first scp the file to 172.16.44.128/config/filestore/data_mda_1, then issue 2 Rest commands: curl -sk -u admin:admin -H "Content-Type: application/json" -X POST -d '{"name":"data_mda_1","type":"string","source-path":"file:///config/filestore/data_mda_1"}' https://172.16.44.128/mgmt/tm/sys/file/data-group curl -sk -u admin:admin -H "Content-Type: application/json" -X POST -d '{"name":"dg_mda","external-file-name":"/Common/data_mda_1"}' https://172.16.44.128/mgmt/tm/ltm/data-group/external/ To update the external data group, many programmers first try something like the following: curl -sk -u admin:admin -H "Content-Type: application/json" -X POST -d '{"name":"data_mda_2","type":"string","source-path":"file:///config/filestore/data_mda_2”}’ https://172.16.44.128/mgmt/tm/sys/file/data-group curl -sk -u admin:admin -H "Content-Type: application/json" -X PUT -d '{"name":"dg_mda","external-file-name":"/Common/data_mda_2"}' https://172.16.44.128/mgmt/tm/ltm/data-group/external/ The first command works because we are creating a new ifile object. However, the second command fails because we are trying to modify a specific external data group at the generic external data group level. The proper command is: curl -sk -u admin:admin -H "Content-Type: application/json" -X PUT -d '{"external-file-name":"/Common/data_mda_2"}' https://172.16.44.128/mgmt/tm/ltm/data-group/external/dg_mda The python code gets a little more complex with the data-group examples, so I've uploaded it to the codeshare here. Much thanks to Pat Chang for the bulk of the content in this article. Stay tuned for part 2, where we'll cover sub collections and how to use them.Demystifying iControl REST Part 7 - Understanding Transactions
iControl REST. It’s iControl SOAP’s baby, brother, introduced back in TMOS version 11.4 as an early access feature but released fully in version 11.5. Several articles on basic usage have been written about the rest interface so the intent here isn’t basic use, but rather to demystify some of the finer details of using the API. A few months ago, a question in Q&A from community member spirrello asking how to update a tcp profile on a virtual. He was using bigsuds, the python wrapper for the soap interface. For the rest interface on this particular object, this is easy; just use the put method and supply the payload mapping the updated profile. But for soap, this requires a transaction. There are some changes to BIG-IP via the rest interface, however, like updating an ssl cert or key, that likewise will require a transaction to accomplish. In this article, I’ll show you how to use transactions with the rest interface. The Fine Print From the iControl REST user guide, the life cycle of a transaction progresses through three phases: Creation - This phase occurs when the transaction is created using a POST command. Modification - This phase occurs when commands are added to the transaction, or changes are made to the sequence of commands in the transaction. Commit - This phase occurs when iControl REST runs the transaction. To create a transaction, post to /tm/transaction POST https://192.168.25.42/mgmt/tm/transaction {} Response: { "transId":1389812351, "state":"STARTED", "timeoutSeconds":30, "kind":"tm:transactionstate", "selfLink":"https://localhost/mgmt/tm/transaction/1389812351?ver=11.5.0" } Note the transId, the state, and the timeoutSeconds. You'll need the transId to add or re-sequence commands within the transaction, and the transaction will expire after 30 seconds if no commands are added. You can list all transactions, or the details of a specific transaction with a get request. GET https://192.168.25.42/mgmt/tm/transaction GET https://192.168.25.42/mgmt/tm/transaction/transId To add a command to the transaction, you use the normal method uris, but include the X-F5-REST-Coordination-Id header. This example creates a pool with a single member. POST https://192.168.25.42/mgmt/tm/ltm/pool X-F5-REST-Coordination-Id:1389812351 { "name":"tcb-xact-pool", "members": [ {"name":"192.168.25.32:80","description":"First pool for transactions"} ] } Not a great example because there is no need for a transaction here, but we'll roll with it! There are several other option methods for interrogating the transaction itself, see the user guide for details. Now we can commit the transaction. To do that, you reference the transaction id in the URI, remove the X-F5-REST-Coordination-Id header and use the patch method with payload key/value state: VALIDATING . PATCH https://localhost/mgmt/tm/transaction/1389812351 { "state":"VALIDATING" } That's all there is to it! Now that you've seen the nitty gritty details, let's take a look at some code samples. Roll Your Own In this example, I am needing to update and ssl key and certificate. If you try to update the cert or the key, it will complain that they do not match, so you need to update both at the same time. Assuming you are writing all your code from scratch, this is all it takes in python. Note on line 21 I post with an empty payload, and then on line 23, I add the header with the transaction id. I make my modifications and then in line 31, I remove the header, and finally on line 32, I patch to the transaction id with the appropriate payload. import json import requests btx = requests.session() btx.auth = (f5_user, f5_password) btx.verify = False btx.headers.update({'Content-Type':'application/json'}) urlb = 'https://{0}/mgmt/tm'.format(f5_host) domain = 'mydomain.local_sslobj' chain = 'mychain_sslobj try: key = btx.get('{0}/sys/file/ssl-key/~Common~{1}'.format(urlb, domain)) cert = btx.get('{0}/sys/file/ssl-cert/~Common~{1}'.format(urlb, domain)) chain = btx.get('{0}/sys/file/ssl-cert/~Common~{1}'.format(urlb, 'chain')) if (key.status_code == 200) and (cert.status_code == 200) and (chain.status_code == 200): # use a transaction txid = btx.post('{0}/transaction'.format(urlb), json.dumps({})).json()['transId'] # set the X-F5-REST-Coordination-Id header with the transaction id btx.headers.update({'X-F5-REST-Coordination-Id': txid}) # make modifications modkey = btx.put('{0}/sys/file/ssl-key/~Common~{1}'.format(urlb, domain), json.dumps(keyparams)) modcert = btx.put('{0}/sys/file/ssl-cert/~Common~{1}'.format(urlb, domain), json.dumps(certparams)) modchain = btx.put('{0}/sys/file/ssl-cert/~Common~{1}'.format(urlb, 'le-chain'), json.dumps(chainparams)) # remove header and patch to commit the transaction del btx.headers['X-F5-REST-Coordination-Id'] cresult = btx.patch('{0}/transaction/{1}'.format(urlb, txid), json.dumps({'state':'VALIDATING'})).json() A Little Help from a Friend The f5-common-python library was released a few months ago to relieve you of a lot of the busy work with building requests. This is great, especially for transactions. To simplify the above code just to the transaction steps, consider: # use a transaction txid = btx.post('{0}/transaction'.format(urlb), json.dumps({})).json()['transId'] # set the X-F5-REST-Coordination-Id header with the transaction id btx.headers.update({'X-F5-REST-Coordination-Id': txid}) # do stuff here # remove header and patch to commit the transaction del btx.headers['X-F5-REST-Coordination-Id'] cresult = btx.patch('{0}/transaction/{1}'.format(urlb, txid), json.dumps({'state':'VALIDATING'})).json() With the library, it's simplified to: tx = b.tm.transactions.transaction with TransactionContextManager(tx) as api: # do stuff here api.do_stuff Yep, it's that simple. So if you haven't checked out the f5-common-python library, I highly suggest you do! I'll be writing about how to get started using it next week, and perhaps a follow up on how to contribute to it as well, so stay tuned!Demystifying iControl REST Part 5: Transferring Files
iControl REST. It’s iControl SOAP’s baby, brother, introduced back in TMOS version 11.4 as an early access feature but released fully in version 11.5. Several articles on basic usage have been written on iControl REST so the intent here isn’t basic use, but rather to demystify some of the finer details of using the API. This article will cover the details on how to transfer files to/from the BIG-IP using iControl REST and the python programming language. (Note: this functionality requires 12.0+.) The REST File Transfer Worker The file transfer worker allows a client to transfer files through a series of GET operations for downloads and POST operations for uploads. The Content-Range header is used for both as a means to chunk the content. For downloads, the worker listens onthe following interfaces. Description Method URI File Location Download a File GET /mgmt/cm/autodeploy/software-image-downloads/ /shared/images/ Upload an Image File POST /mgmt/cm/autodeploy/software-image-uploads/ /shared/images/ Upload a File POST /mgmt/shared/file-transfer/uploads/ /var/config/rest/downloads/ Download a QKView GET /mgmt/shared/file-transfer/qkview-downloads/ /var/tmp/ Download a UCS GET /mgmt/shared/file-transfer/ucs-downloads/ /var/local/ucs/ Upload ASM Policy POST /mgmt/tm/asm/file-transfer/uploads/ /var/ts/var/rest/ Download ASM Policy GET /mgmt/tm/asm/file-transfer/downloads/ /var/ts/var/rest/ Binary and text files are supported. The magic in the transfer is the Content-Range header, which has the following format: Content-Range: start-end/filesize Where start/end are the chunk's delimiters in the file and filesize is well, the file size. Any file larger than 1M needs to be chunked with this header as that limit is enforced by the worker. This is done to avoid potential denial of service attacks and out of memory errors. There are benefits of chunking as well: Accurate progress bars Resuming interrupted downloads Random access to file content possible Uploading a File The function is shown below. Note that whereas normally with the REST API the Content-Type is application/json, with file transfers that changes to application/octet-stream. The workflow for the function works like this (line number in parentheses) : Set the Chunk Size (3) Set the Content-Type header (4-6) Open the file (7) Get the filename (apart from the path) from the absolute path (8) If the extension is an .iso file (image) put it in /shared/images, otherwise it’ll go in /var/config/rest/downloads (9-12) Disable ssl warnings requests (required with my version: 2.8.1. YMMV) (14) Set the total file size for use with the Content-Range header (15) Set the start variable to 0 (17) Begin loop to iterate through the file and upload in chunks (19) Read data from the file and if there is no more data, break the loop (20-22) set the current bytes read, if less than the chunk size, then this is the last chunk, so set the end to the size from step 7. Otherwise, add current bytes length to the start value and set that as the end. (24-28) Set the Content-Range header value and then add that to the header (30-31) Make the POST request, uploading the content chunk (32-36) Increment the start value by the current bytes content length (38) def _upload(host, creds, fp): chunk_size = 512 * 1024 headers = { 'Content-Type': 'application/octet-stream' } fileobj = open(fp, 'rb') filename = os.path.basename(fp) if os.path.splitext(filename)[-1] == '.iso': uri = 'https://%s/mgmt/cm/autodeploy/software-image-uploads/%s' % (host, filename) else: uri = 'https://%s/mgmt/shared/file-transfer/uploads/%s' % (host, filename) requests.packages.urllib3.disable_warnings() size = os.path.getsize(fp) start = 0 while True: file_slice = fileobj.read(chunk_size) if not file_slice: break current_bytes = len(file_slice) if current_bytes < chunk_size: end = size else: end = start + current_bytes content_range = "%s-%s/%s" % (start, end - 1, size) headers['Content-Range'] = content_range requests.post(uri, auth=creds, data=file_slice, headers=headers, verify=False) start += current_bytes Downloading a File Downloading is very similar but there are some differences. Here is the workflow that is different, followed by the code. Note that the local path where the file will be downloaded to is given as part of the filename. URI is set to downloads worker. The only supported download directory at this time is /shared/images. (8) Open the local file so received data can be written to it (11) Make the request (22-26) If response code is 200 and if size is greater than 0, increment the current bytes and write the data to file, otherwise exit the loop (28-40) Set the value of the returned Content-Range header to crange and if initial size (0), set the file size to the size variable (42-46) If the file is smaller than the chunk size, adjust the chunk size down to the total file size and continue (51-55) Do the math to get ready to download the next chunk (57-62) def _download(host, creds, fp): chunk_size = 512 * 1024 headers = { 'Content-Type': 'application/octet-stream' } filename = os.path.basename(fp) uri = 'https://%s/mgmt/cm/autodeploy/software-image-downloads/%s' % (host, filename) requests.packages.urllib3.disable_warnings() with open(fp, 'wb') as f: start = 0 end = chunk_size - 1 size = 0 current_bytes = 0 while True: content_range = "%s-%s/%s" % (start, end, size) headers['Content-Range'] = content_range #print headers resp = requests.get(uri, auth=creds, headers=headers, verify=False, stream=True) if resp.status_code == 200: # If the size is zero, then this is the first time through the # loop and we don't want to write data because we haven't yet # figured out the total size of the file. if size > 0: current_bytes += chunk_size for chunk in resp.iter_content(chunk_size): f.write(chunk) # Once we've downloaded the entire file, we can break out of # the loop if end == size: break crange = resp.headers['Content-Range'] # Determine the total number of bytes to read if size == 0: size = int(crange.split('/')[-1]) - 1 # If the file is smaller than the chunk size, BIG-IP will # return an HTTP 400. So adjust the chunk_size down to the # total file size... if chunk_size > size: end = size # ...and pass on the rest of the code continue start += chunk_size if (current_bytes + chunk_size) > size: end = size else: end = start + chunk_size - 1 Now you know how to upload and download files. Let’s do something with it! A Use Case - Upload Cert & Key to BIG-IP and Create a Clientssl Profile! This whole effort was sparked by a use case in Q&A, so I had to deliver the goods with more than just moving files around. The complete script is linked at the bottom, but there are a few steps required to get to a clientssl certificate: Upload the key & certificate Create the file object for key/cert Create the clientssl profile You know how to do step 1 now. Step 2 is to create the file object for the key and certificate. After a quick test to see which file is the certificate, you set both files, build the payload, then make the POST requests to bind the uploaded files to the file object. def create_cert_obj(bigip, b_url, files): f1 = os.path.basename(files[0]) f2 = os.path.basename(files[1]) if f1.endswith('.crt'): certfilename = f1 keyfilename = f2 else: keyfilename = f1 certfilename = f2 certname = f1.split('.')[0] payload = {} payload['command'] = 'install' payload['name'] = certname # Map Cert to File Object payload['from-local-file'] = '/var/config/rest/downloads/%s' % certfilename bigip.post('%s/sys/crypto/cert' % b_url, json.dumps(payload)) # Map Key to File Object payload['from-local-file'] = '/var/config/rest/downloads/%s' % keyfilename bigip.post('%s/sys/crypto/key' % b_url, json.dumps(payload)) return certfilename, keyfilename Notice we return the key/cert filenames so they can be used for step 3 to establish the clientssl profile. In this example, I name the file object and the clientssl profile to the name of the certfilename (minus the extension) but you can alter this to allow the objects names to be provided. To build the profile, just create the payload with the custom key/cert and make the POST request and you are done! def create_ssl_profile(bigip, b_url, certname, keyname): payload = {} payload['name'] = certname.split('.')[0] payload['cert'] = certname payload['key'] = keyname bigip.post('%s/ltm/profile/client-ssl' % b_url, json.dumps(payload)) Much thanks to Tim Rupp who helped me get across the finish line with some counting and rest worker errors we were troubleshooting on the download function. Get the Code Upload a File Download a File Upload Cert/Key & Build a Clientssl Profile
Python Bigsuds - License BIG-IP
Problem this snippet solves: This bigsuds script will license a BIGIP How to use this snippet: This bigsuds script will license a BIGIP It's a port of some of Erick's: https://devcentral.f5.com/wiki/icontrol.BigIpLicensingCommandLineTool.ashx and update of the pycontrol v2: pyControl v2 - License BIGIP - DevCentral Still missing the proxy functionality in Erick's though. Usage: license-bigip.py --bigip < IP|hostname> --username < username> --server < license_server_hostname> --reg_keys < regkeys> --license < license_file> --eula < eula_file> Ex. From CLI: (virt1)user1@desktop:python $./license-bigip.py --help Usage: license-bigip.py [options] Options: -h, --help show this help message and exit -b BIGIP, --bigip=BIGIP -u UNAME, --username=UNAME -s SERVER_HOSTNAME, --server=SERVER_HOSTNAME -r REG_KEYS_STRING, --reg_keys=REG_KEYS_STRING -l LOCAL_LICENSE_FILE_NAME, --license=LOCAL_LICENSE_FILE_NAME -e LOCAL_EULA_FILE_NAME, --eula=LOCAL_EULA_FILE_NAME Providing New Keys: (virt1)user1@desktop:python $./license-bigip.py --server activate.f5.com --bigip 10.11.50.201 --username admin --reg_keys "B6907-36850-30441-96521-8395199" Enter your password for username: admin Password: Attempting to get license online. License server requires you to submit EULA. reg keys provided ['B6907-36850-30441-96521-8395172'] Getting dossier using keys:['B6907-36850-30441-96521-8395199'] License Found. Attempting to installing License on BIGIP: License status = STATE_ENABLED Use Existing Keys: (virt1)user1@desktop:python $./license-bigip.py --server activate.f5.com --bigip 10.11.50.201 --username admin Enter your password for username: admin Password: Attempting to get license online. License server requires you to submit EULA. Reg Key list is empty, attempting to retrieve existing keys from the unit Getting dossier using keys:[B6907-36850-30441-96521-8395199, P459718-6102942, B273271-7144254, N123537-7304104] License Found. Attempting to installing License on BIGIP: License status = STATE_ENABLED Script license-bigip.py Code : #!/usr/bin/env python ''' ---------------------------------------------------------------------------- The contents of this file are subject to the "END USER LICENSE AGREEMENT FOR F5 Software Development Kit for iControl"; you may not use this file except in compliance with the License. The License is included in the iControl Software Development Kit. Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. The Original Code is iControl Code and related documentation distributed by F5. The Initial Developer of the Original Code is F5 Networks, Inc. Seattle, WA, USA. Portions created by F5 are Copyright (C) 1996-2004 F5 Networks, Inc. All Rights Reserved. iControl (TM) is a registered trademark of F5 Networks, Inc. Alternatively, the contents of this file may be used under the terms of the GNU General Public License (the "GPL"), in which case the provisions of GPL are applicable instead of those above. If you wish to allow use of your version of this file only under the terms of the GPL and not to allow others to use your version of this file under the License, indicate your decision by deleting the provisions above and replace them with the notice and other provisions required by the GPL. If you do not delete the provisions above, a recipient may use your version of this file under either the License or the GPL. ---------------------------------------------------------------------------- ''' def usage (): print "Usage:" print "%s --bigip --username --server \ --reg_keys --license --eula " % sys.argv[0] print "ex. " print " Will attempt to re-license with existing keys on unit using license server activate.f5.com" print " %s --bigip 192.168.1.245 --username admin" % sys.argv[0] print " Will attempt to re-license with provided reg_keys CSV string using license server activate.f5.com" print " %s --bigip 192.168.1.245 --username admin --reg_keys \"XXXX-XXXX-XXXX-XXXX,XXXX-XXXX,XXXX-XXXX\" " % sys.argv[0] def get_license_from_F5_License_Server ( server_hostname, dossier_string, eula_string, email, firstName, lastName, companyName, phone, jobTitle, address, city, stateProvince, postalCode, country ): try: license_string = "" # Unfortunately, F5 wsdl on license server references http but F5 only accepts https so as an ugly workaround need to # download wsdl, save to disk, replace links http with https, and have SUDS client reference local file instead #(eg. url = "file:///home/admin/f5wsdl.xml") download_url = "https://" + server_hostname + "/license/services/urn:com.f5.license.v5b.ActivationService?wsdl" # Check to see if there's a copy of wsdl file on disk first # Careful with this behavior if you switch server hostnames. local_wsdl_file_name = str(server_hostname) + '-f5wsdl-w-https.xml' wsdl_data = [] try: with open(local_wsdl_file_name, 'r') as fh_wsdl: wsdl_data = fh_wsdl.read() except: print "Can't find a locally stored WSDL file." if not wsdl_data: print "Attempting to fetch wsdl online." f5wsdl = urllib2.urlopen(download_url) newlines = [] for line in f5wsdl: # do the replacing here newlines.append(line.replace('http://' + server_hostname , 'https://' + server_hostname)) fh_local = open(local_wsdl_file_name,'w') fh_local.writelines(newlines) fh_local.close() # put url going to pass to client in file format url = "file:" + urllib.pathname2url(os.getcwd()) + "/" + local_wsdl_file_name #Now create client object using wsdl from disk instead of the interwebs. client = Client(url) # NOT using below as will just try actually licensing and fail then if needed # try: # # ping() method should return string containing date # print "Checking License Service Reachability..." # return_ping_date = client.service.ping() # except: # print "License SOAP service unreachable. Check network connectivity." # return transaction = client.factory.create('ns0:LicenseTransaction') # If eula isn't present on first call to getLicense, transaction will fail # but it will return a eula after first attempt transaction = client.service.getLicense( dossier = dossier_string, eula = eula_string, email = email, firstName = firstName , lastName = lastName, companyName = companyName, phone = phone, jobTitle = jobTitle, address = address, city = city, stateProvince = stateProvince, postalCode = postalCode, country = country, ) #Extract the eula offered from first try eula_string = transaction.eula if transaction.state == "EULA_REQUIRED": #Try again, this time with eula populated transaction = client.service.getLicense( dossier = dossier_string, eula = eula_string, email = email, firstName = firstName , lastName = lastName, companyName = companyName, phone = phone, jobTitle = jobTitle, address = address, city = city, stateProvince = stateProvince, postalCode = postalCode, country = country, ) if transaction.state == "LICENSE_RETURNED": license_string = transaction.license else: print "Can't retrieve license from Licensing server" print "License server returned error: Number:" + str(transaction.fault.faultNumber) + " Text: " + str(transaction.fault.faultText) return license_string except: print "Can't retrieve License from Server" traceback.print_exc(file=sys.stdout) def get_reg_keys(obj): try: reg_keys = [] reg_keys = obj.Management.LicenseAdministration.get_registration_keys() return reg_keys except: print "Get Reg Keys error. Check log." traceback.print_exc(file=sys.stdout) def get_dossier (obj, reg_keys ): try: dossier_string = obj.Management.LicenseAdministration.get_system_dossier ( reg_keys ) return dossier_string except: print "Get Dossier error. Check log." traceback.print_exc(file=sys.stdout) def get_eula_file (obj): try: eula_char_array = obj.Management.LicenseAdministration.get_eula_file( ) eula_string = base64.b64decode(eula_char_array) return eula_string except: print "Get eula_file. Check log." traceback.print_exc(file=sys.stdout) def install_license (obj, license_string ): try: license_char_array = base64.b64encode(license_string) obj.Management.LicenseAdministration.install_license ( license_file_data = license_char_array ) except: print "Install License error. Check log." traceback.print_exc(file=sys.stdout) def get_license_status (obj): try: license_status = obj.Management.LicenseAdministration.get_license_activation_status() return license_status except: print "Get License Status error. Check log." traceback.print_exc(file=sys.stdout) ### IMPORT MODULES ### import os import sys import time import traceback import base64 import urllib import urllib2 import getpass from suds.client import Client from optparse import OptionParser import bigsuds # from suds import WebFault # import logging # logging.getLogger('suds.client').setLevel(logging.DEBUG) # logging.getLogger('suds.metrics').setLevel(logging.DEBUG) # logging.getLogger('suds').setLevel(logging.DEBUG) #### SET CONFIG VARIABLES #### #Misc EULA Variables email = "example.icontrol@f5.com" firstName = "example" lastName = "iControl" companyName = "F5" phone = "2062725555" jobTitle = "DEV OPS" address = "111 EXAMPLE ICONTROL RD" city = "Seattle" stateProvince = "WA" postalCode = "98119" country = "United States" parser = OptionParser() parser.add_option("-b", "--bigip", action="store", type="string", dest="bigips", default="192.168.1.245") parser.add_option("-u", "--username", action="store", type="string", dest="uname", default="admin") parser.add_option("-s", "--server", action="store", type="string", dest="server_hostname", default="activate.f5.com" ) parser.add_option("-r", "--reg_keys", action="store", type="string", dest="reg_keys_string" ) parser.add_option("-l", "--license", action="store", type="string", dest="local_license_file_name") parser.add_option("-e", "--eula", action="store", type="string", dest="local_eula_file_name") (options, args) = parser.parse_args() ### INITIALIZE BIGIP OBJECT ### if options.bigip and options.uname: print "Enter your password for username: %s" % options.uname upass = getpass.getpass() else: usage() sys.exit() #hardcoded so don't have to provide any arguments. #print "Enter your password for username: %s" % options.uname #upass = getpass.getpass() # Can re-license a list of BIG-IPs IF you don't provide a list of regkeys on CLI. # As reg key split function does not accommodate list of keys for multiple devices. if options.bigips: bigip_list = options.bigips.split(",") for i in bigip_list: print "\nAttempting to License BIGIP " + i reg_keys = [] license_string = "" eula_string = "" reg_keys_string = "" local_license_file_name = "" local_eula_file_name = "" server_hostname = "authem.f5net.com" uname = "admin" b = bigsuds.BIGIP( hostname = i, username = uname, password = upass, ) ########## START MAIN LOGIC ###### if local_license_file_name: try: print "Attempting to retrive License from local disk ..." with open(local_license_file_name, 'r') as fh_license: license_string = fh_license.read() except: print "Can't Open license file named: \"" + local_license_file_name + "\" on disk." #sys.exit() print "No worries. Will attempt to retrieve one from online." if not license_string: print "Attempting to get license online." print "License server requires you to submit EULA." if options.local_eula_file_name: print "Attempting to retrive EULA from local disk first..." try: with open(options.local_eula_file_name, 'r') as fh_eula: eula_string = fh_eula.read() except: print "Can't find EULA file named : \"" + options.local_eula_file_name + "\" on disk." print "No worries. Will attempt to retrieve one during transaction with License Server." # Could also try seeing if Target BIG-IP has one stored # eula_file = get_eula_file(b) if options.reg_keys_string: reg_keys = options.reg_keys_string.split(",") print "reg keys provided" print reg_keys if len(reg_keys) < 1: print "Reg Key list is empty, attempting to retrieve existing keys from the unit" reg_keys = get_reg_keys(b) print "Getting dossier using keys:" + str(reg_keys) dossier_string = get_dossier( b, reg_keys ) # print "dossier = " + str(dossier_output) license_string = get_license_from_F5_License_Server( options.server_hostname, dossier_string, eula_string, email, firstName, lastName, companyName, phone, jobTitle, address, city, stateProvince, postalCode, country ) if license_string: print "License Found. Attempting to installing License on BIGIP:" install_license ( b, license_string ) else: print "Sorry. Could not retrieve License. Check your connection" license_status = get_license_status ( b ) print "License status = " + str(license_status)
Object path for list of partitions in python SDK
Hi all, I'm trying to get the list of partitions of a BigIP using the f5-Python-SDK. In tmsh, the objects are located under auth/partition, so e.g. "tmsh list auth partition" will list the according configuration. This is also reflected in iControl REST. Using curl, I can get that list of partitions by using the path /mgmt/tm/auth/partition as follows: $ curl -sk -u admin:xxxxxxxxxxxx https://my_bigip/mgmt/tm/auth/partition | python -mjson.tool { "items": [ { "defaultRouteDomain": 0, "description": "Repository for system objects and shared objects.", "fullPath": "Common", "generation": 1, "kind": "tm:auth:partition:partitionstate", "name": "Common", "selfLink": "https://localhost/mgmt/tm/auth/partition/Common?ver=11.6.0" }, { "defaultRouteDomain": 123, "fullPath": "my_partition1", "generation": 1, "kind": "tm:auth:partition:partitionstate", "name": "my_partition1", "selfLink": "https://localhost/mgmt/tm/auth/partition/my_partition1?ver=11.6.0" }, { "defaultRouteDomain": 456, "fullPath": "my_partition2", "generation": 1, "kind": "tm:auth:partition:partitionstate", "name": "my_partition2", "selfLink": "https://localhost/mgmt/tm/auth/partition/my_partition1?ver=11.6.0" }, ], "kind": "tm:auth:partition:partitioncollectionstate", "selfLink": "https://localhost/mgmt/tm/auth/partition?ver=11.6.0" } However, when trying to adopt that using the python SDK, I do not get the same results: >>> from f5.bigip import BigIP >>> session = BigIP('1.2.3.4', 'admin', 'xxxxxxxxxxxx') >>> for partition in session.auth.partitions.get_collection(): ... print partition.name ... Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python2.7/dist-packages/f5_sdk-0.1.5-py2.7.egg/f5/bigip/mixins.py", line 82, in __getattr__ raise AttributeError(error_message) AttributeError: '' object has no attribute 'auth' So, .auth.partitions (or .auth generally) does not seem to be the right entry point. Does anybody know: if there is a comprehensive list of mappings of tmsh / iControl REST / python SDK paths? whether the whole auth tree is still missing in the python modules and will be added later? what is the right way to get a list of partitions using the f5-python-SDK? Any help is appreciated. Many thanks in advance! Best regards Martin
Python module to post and retrieve IControl Rest JSON objects for AVR statistics
Problem this snippet solves: This module simplifies making Python dictionary objects that are converted to IControl rest AVR JSON objects. It also handles making AVR requests and retrieving results as well allowing multiple AVR requests to be queued, posted and retrieved. It also has some basis type checking for the elements of a AVR request. This module requires Bigip 12.1 on the target that statistics are retrieved. How to use this snippet: The main class is rest_avr.avr_req. It is a dictionary class that maps directly to an IControl Rest AVR JSON request as translated by json.dumps. Each dictionary element is an object derived from a customer class for each part of the request. The element classes have add() and clear() functions. if the element class only allows one entry the add() function will replace the existing entry, otherwise it will append the entry to the request element. The rest_avr.avr_req class also has functions to populate the HTTP host and authentication values for the target system. rest_avr.avr_req.post_and_response returns the Python representation of the JSON result of the query. rest_avr.avr_req.add_to_queue() adds the currently constructed request to a queue of requests to post. rest_avr.avr_req.post_and_response_queue() returns a python list of results of queued queries. The following code sample constructs, posts and returns results for an AVR statistics request for specific DNS records and a specificrecord type, then queues multiple quests and posts and returns results. #!/usr/bin/python import json import sys import time import rest_avr #print rest_avr.ShowAVRJsonApi #Populate the url avr_dns_req=rest_avr.avr_req() avr_dns_req.auth('admin','admin') avr_dns_req.url_base('10.10.2.113','dns') #Populate the json object avr_dns_req['analyticsModule'].add('dns') avr_dns_req['reportFeatures'].add('time-aggregated') avr_dns_req['entityFilters'].add('domain-name', 'OPERATOR_TYPE_EQUAL', ['test2.test1.com','test1.test1.com']) avr_dns_req['entityFilters'].add('query-type', 'OPERATOR_TYPE_EQUAL', ['a']) avr_dns_req['viewMetrics'].add('packets') avr_dns_req['viewDimensions'].add('domain-name') avr_dns_req['metricFilters'].add('packets', 'OPERATOR_TYPE_GREATER_THAN', 0) avr_dns_req['sortByMetrics'].add('packets', 'ascending') avr_dns_req['pagination'].add(20, 0) avr_dns_req['timeRange'].add(1461778251000000, None) #Post and retrieve results. result_py=avr_dns_req.post_and_response() if result_py != None: print ('\n' + result_py['results']['timeAggregated'][0]['dimensions'][0]['value'] + " " + result_py['results']['timeAggregated'][0]['metricValues'][0]['value'] + '\n') else: print result_py.error_layer print result_py.error_code print result_py.error_text # Now add multiple requests to a queue avr_dns_req.add_to_queue() avr_dns_req['entityFilters'].clear() avr_dns_req['entityFilters'].add('query-type', 'OPERATOR_TYPE_EQUAL', ['aaaa']) avr_dns_req.add_to_queue() #post and retrieve queued results result_py_q=avr_dns_req.post_and_response_queue() for result_py in result_py_q: if result_py != None: print ('\n' + result_py['results']['timeAggregated'][0]['dimensions'][0]['value'] + " " + result_py['results']['timeAggregated'][0]['metricValues'][0]['value'] + '\n') else: print result_py.error_layer print result_py.error_code print result_py.error_text Code : """ rest_avr provides a python interface to Bigip AVR statistics using the REST API. The main Python rest_avr.avr_req object is a Python dictionary that maps to a JSON object that can be processed with the json.dumps() function An IControl Rest AVR JSON request and response can be initiated with avr_req.post_and_response The simple description of the API can is available at avr_req.ShowJsonApi() Each of these modules has a method to add single or multiple elements as appropriate to the specific module. Once these elements are are populated a RestAPI request can be made with results returned as a python representation. avr_req.auth(user, passw) avr_req.url_base(host, module) avr_req['analyticsModule'].add(module) avr_req['analyticsModule'].clear() avr_req['reportFeatures'].add(metric_name, predicate, value) avr_req['reportFeatures'].clear() avr_req['entityFilters'].add(dimension_name, predicate, values) avr_req['entityFilters'].clear() avr_req['viewMetrics'].add(metric_name) avr_req['viewMetrics'].clear() avr_req['viewDimensions'].add(metric_name, order) avr_req['viewDimensions'].clear() avr_req['metricFilters'].add(metric_name, predicate, valu) avr_req['metricFilters'].clear() avr_req['sortByMetrics'].add(metric_name, orde) avr_req['sortByMetrics'].clear() avr_req['pagination'].add(num_results, skip_result) avr_req['pagination'].clear() avr_req['timeRange'].add(t_from, t_to) avr_req['timeRange'].clear() After a request in constructed a REST API call is initiated with initiated with: avr_req.post_and_response() The response is a python dictionary data structure of the results as processed by json.loads """ from copy import deepcopy import requests import json import sys import time import warnings __author__ = 'Mark Lloyd' __version__ = '1.0' # 05/24/2016 import json import requests import time class BadDictElement(Exception): def __init__(self, key, value, expl): Exception.__init__(self, '{0} {1} {2} '.format(key, value, expl)) class BadTime(Exception): def __init__(self, variable, value): Exception.__init__(self, '{0} {1} should be 16 char decimal in microseconds '.format('a', 'b')) class RequestFailure(Exception): def __init__(self, key, value): Exception.__init__(self, '{0} {1} '.format(key, value)) class analyticsModule(str): """ This class is tied to the structure of the parent class. parent() get's the parent object so we can make the string pseudo-mutable. accessed from within an avr request ['analyticsModule'].add(module) Adds a single string to analyticsModule element . If one exists it is replaced. ['analyticsModule'].clear() Send a null value to the analyticsModule element. See rest_avr.ShowAVRJsonApi for more details """ def parent(self, parent): self.parent = parent def add(self, module): """ avr_req.['analyticsModule'].add(module) Adds a single string to analyticsModule element . If one already exists it is replaced. This should be the same as the module string in avr_req.url_base. """ self.parent['analyticsModule'] = analyticsModule(module) self.parent['analyticsModule'].parent = self.parent def clear(self): """ avr_req.['analyticsModule'].add(module) replaces the analyticsModule mddule with a null string """ self.parent['analyticsModule'] = analyticsModule('') self.parent['analyticsModule'].parent = self.parent class metricFilters(list): """ avr_req.['metricFilters'].add(metric_name, predicate, value) metric name is a string, value is an integer Valid predicates strings are ['OPERATOR_TYPE_EQUAL', 'OPERATOR_TYPE_NOT_EQUAL', 'OPERATOR_TYPE_GREATER_THAN', OPERATOR_TYPE_LOWER_THAN','OPERATOR_TYPE_GREATER_THAN_OR_EQUAL', 'OPERATOR_TYPE_LOWER_THAN_OR_EQUAL']) avr_req['metricFilters'].clear() Clears metricFilters elements See rest_avr.ShowAVRJsonApi for more details. """ def __init__(self): self.append([]) self.valid_metric_predicate = ( ['OPERATOR_TYPE_EQUAL', 'OPERATOR_TYPE_NOT_EQUAL', 'OPERATOR_TYPE_GREATER_THAN', 'OPERATOR_TYPE_LOWER_THAN', 'OPERATOR_TYPE_GREATER_THAN_OR_EQUAL', 'OPERATOR_TYPE_LOWER_THAN_OR_EQUAL']) def add(self, metric_name, predicate, value): """ avr_req.['metricFilters'].add(metric_name, predicate, value) metric name is a string, value is an integer Valid predicates strings are ['OPERATOR_TYPE_EQUAL', 'OPERATOR_TYPE_NOT_EQUAL', 'OPERATOR_TYPE_GREATER_THAN', OPERATOR_TYPE_LOWER_THAN','OPERATOR_TYPE_GREATER_THAN_OR_EQUAL', 'OPERATOR_TYPE_LOWER_THAN_OR_EQUAL'] """ if type(value) is not int: raise BadDictElement(metric_name, value, 'value should be integer') if predicate in self.valid_metric_predicate: # first check if it is already there for metric in self[0]: if metric['metricName'] == metric_name: metric['predicate'] = predicate metric['value'] = value return 0 # if it is not there then just add it. self[0].append({'metricName': metric_name, 'predicate': predicate, 'value': value}) else: raise BadDictElement(metric_name, predicate, 'invalid predicate') def clear(self): """ avr_req['metricFilters'].clear() Clears metricFilters elements """ del self[0][:] class entityFilters(list): """ avr_req.['entityFilters'].add(dimension_name, predicate, values): All values are strings valid predicate is 'OPERATOR_TYPE_EQUAL' ['entityFilters'].clear() Clears the entityFilters element See rest_avr.ShowJsonApi for more details """ def __init__(self): self.append([]) def add(self, dimension_name, predicate, values): """ avr_req.['entityFilters'].add(dimension_name, predicate, values): All values are strings valid predicate is 'OPERATOR_TYPE_EQUAL' """ if predicate is 'OPERATOR_TYPE_EQUAL': # then loop throuth to see if the dimenson name already exists, if so replace for entity in self[0]: if entity['dimensionName'] == dimension_name: entity['predicate'] = predicate entity['values'] = values return 0 # if it is not there then just add it. self[0].append({'dimensionName': dimension_name, 'predicate': predicate, 'values': values}) else: raise BadDictElement(dimension_name, predicate, 'predicate must be OPERATOR_TYPE_EQUAL') def clear(self): """ ['entityFilters'].clear() Clears the entityFilters element """ del self[0][:] class reportFeatures(list): """ avr_req.['reportFeatures'].add( feature) adds report feature string. Multiple features are permitted. ['reportFeatures'].clear() Clears the analyticsModule element. See rest_avr.ShowAVRJsonApi for more details. """ def add(self, feature): """ avr_req.['reportFeatures'].add( feature) adds report feature string. Multiple features are permitted .""" if feature not in self: self.append(feature) def clear(self): """ ['reportFeatures'].clear() Clears the entityFilters element """ del self[:] class sortByMetrics(list): """ avr_req.['sortByMetrics'].add(metric_name, order) valid order names are 'ascending' and 'descending' sortByMetrics is optional in an AVR request. avr_req['sortByMetrics'].clear() Clears the sortByMetrics element. See rest_avr.ShowAVRJsonApi for more details. """ def __init__(self): self.metric_list = [] def add(self, metric_name, order): if metric_name not in self.metric_list: self.append({'metricName': metric_name, 'order': order}) self.metric_list.append(metric_name) def clear(self): """ ['sortByMetrics'].clear() Clears the sortByMetrics element """ del self[:] del self.metric_list[:] class viewDimensions(list): """ avr_req.['viewDimensions'].add(dimension_name): adds view dimension, only one dimension is allowed add will replace element if it already exists avr_req['viewDimensions'].clear() Clears the viewDimensions element. See rest_avr.ShowAVRJsonApi for more details. """ def __init__(self): self.append([]) self[0] = {} def add(self, dimension_name): """ avr_req.['viewDimensions'].add(dimension_name): adds view dimension string, only one dimension is allowed add will replace element if it already exists """ self[0]['dimensionName'] = dimension_name def clear(self, dimension_name): """ ['viewDimensions'].clear() Clears the viewDimensions element """ del self[0][:] class viewMetrics(list): """ avr_req.['viewMetrics'].add(metric_name): appends metric_name string to list. The specification allows multiple view metric elements avr_req['viewMetrics'].clear() Clears the viewMetrics elements See rest_avr.ShowAVRJsonApi for more details. """ def __init__(self): self.metric_list = [] def add(self, metric_name): """ avr_req.['viewMetrics'].add(metric_name): appends metric_name string to list. The specification allows multiple viewMetric elements """ if metric_name not in self.metric_list: self.append({'metricName': metric_name}) self.metric_list.append(metric_name) def clear(self): """ ['viewMetrics'].clear() Clears the viewMetrics elements """ del self[:] del self.metric_list[:] class timeRange(dict): """ avr_req.['timeRange'].add( t_from, t_to) both values are 16 digit numeric value in microseconds of unix/linux time. t_to is optional and can be replace by None timeRange is an optional. avr_req['timeRange'].clear() Clears the timeRange elements See rest_avr.ShowAVRJsonApi for more details. """ def add(self, t_from, t_to): """ avr_req.['timeRange'].add( t_from, t_to) both values are 16 digit numeric value in microseconds of unix/linux time. t_to is optional and can be replace by None timeRange is optional. """ if type(t_from) is long and len(str(t_from)) == 16: self['from'] = t_from else: raise BadTime(t_from + " is 16 digit numeric value in microseconds") if t_to != '' and t_to != 0 and t_to != None: if type(t_to) is long and len(str(t_from)) == 16: self['to'] = t_to else: raise BadTime(t_to + " is 16 digit numeric value in microseconds") else: if 'to' in self.keys(): del self['to'] def clear(self): """ ['timeRange'].clear() Clears the timeRange element """ del self[:] class pagination(dict): """ avr_req.['pagination'].add(num_results, skip_results) both are integer values. avr_req['pagination'].clear() Clears the pagination elements See rest_avr.ShowAVRJsonApi for more details. """ def add(self, num_results, skip_results): """ avr_req.['pagination'].add(num_results, skip_results) both arguments are integers. """ if type(num_results) is int: self['numberOfResults'] = num_results else: raise BadDictElement('number of Results ', num_results, 'must be integer') if type(skip_results) is int: self['skipResults'] = skip_results else: raise BadDictElement('skipResults ', skip_results, 'must be integer') def clear(self): """ ['pagination'].clear() Clears the pagination element """ del self[:] class avr_resp(dict): """ python response error is applicable. """ def __init__(self): self.error_layer = None self.error_code = None self.error_text = None class avr_req(dict): """ The main class for rest_avr. avr_req contains a dictionary that maps to the elements of a Icontrol REST AVR request along with capability of posting that request and receiving a response. The dictionary values are object instances of python classes that correspond to the the JSON values of the object's name/value pair. Each value has two public methods: avr_req.['objectName']add(): adds an element to the appropriate object with type checking. If an element allows more then one instance the add function will append the element If an element allows only one instance the add function will replace the element avr_req.['objectName'].clear()r: clears all elements in the object. printing rest_avr.ShowAVRJsonApi provides documentation for the AVR JASON elements. Further documentation is available on devcentral.f5.com To post an AVR Rest request there are two functions to populate the HTTP/HTTPS request. avr_req.auth(user, passw): provides the username and password avr_req.url_base(host, module) provides the host and the bigip module AVR queries to construct the URL to make the request. Then to post the request and return results in a python representation of the JSON response. avr_req.post_and_response() """ def __init__(self): self['analyticsModule'] = analyticsModule() self['analyticsModule'].parent = self self['pagination'] = pagination() self['metricFilters'] = metricFilters() self['entityFilters'] = entityFilters() self['reportFeatures'] = reportFeatures() self['sortByMetrics'] = sortByMetrics() self['viewDimensions'] = viewDimensions() self['viewMetrics'] = viewMetrics() self['timeRange'] = timeRange() self.avr_session = requests.session() self.avr_session.verify = False self.avr_session.headers.update({'Content-Type': 'application/json'}) # for multiple queued request handling. self.req_queue = [] self.generate_id = None self.done = None self.result = None self.num_requests = 0 self.res_queue = [] def post_and_response(self): """ returns a python representation of the json response to the request. failure returns array ['ERROR','component',error] """ warnings.filterwarnings("ignore") self.generate_request = self.avr_session.post(self.req_url_base + "/generate-report/", data=json.dumps(self)) self.generate_request_py = json.loads(self.generate_request.text) self.result_guid = self.generate_request_py['id'] self.results_status_url = self.req_url_base + "/generate-report/" + self.result_guid + "/?$select=status,reportResultsLink" self.results_url = self.req_url_base + "/report-results/" + self.result_guid self.sleeptime = .5 for i in range(5): time.sleep(self.sleeptime) self.sleeptime *= 2 # double backoff period each time. self.status_results_json = self.avr_session.get(self.results_status_url) self.status_results = json.loads(self.status_results_json.text) if self.status_results['status'] == 'FAILED': self.result = avr_resp() self.result_error_layer = 'REST' self.result_error_code = self.status_results['status'] self.result.error_text = self.status_results if self.status_results['status'] == 'FINISHED': self.raw_results_url = self.status_results['reportResultsLink'] self.results_url = self.raw_results_url.replace('localhost', self.host_name) self.results = self.avr_session.get(self.results_url) if self.results.status_code == 200: self.result = avr_resp() self.result.update(json.loads(self.results.text)) return self.result else: self.result = avr_resp() self.result.error_layer = 'HTTP' self.result.error_code = self.results.status_code self.result.error_text = self.results return self.result else: continue self.result = avr_resp() self.result.error_layer = 'REST_AVR' self.result.error_code = '408' self.result.error_text = 'TIMEOUT' def auth(self, user, passw): """ avr_req.auth(user, passw): username and password """ self.avr_session.auth = (user, passw) def url_base(self, host, module): """ avr_req.url_base(host, module) host and bigip module AVR queries to construct the URL to make the request. """ self.host_name = host self.req_url_base = 'https://%s/mgmt/tm/analytics/%s' % (host, module) self.module_py = {'analyticsModule': module} def add_to_queue(self): "adds request as currently constructed to queue" self.req_queue.append(deepcopy(self)) def clear_queue(self): """" clears request queue """ del self.req_queue[:] def post_and_response_queue(self): """ posts and sends response to from queue of requests. """ warnings.filterwarnings("ignore") for req in self.req_queue: req.generate_request = req.avr_session.post(req.req_url_base + "/generate-report/", data=json.dumps(req)) req.generate_request_py = json.loads(req.generate_request.text) req.generate_id = (req.generate_request_py['id']) req.results_status_url = self.req_url_base + "/generate-report/" + req.generate_id + "/?$select=status,reportResultsLink" self.sleeptime = .5 self.num_requests = len(self.req_queue) for i in range(5): for req in self.req_queue: if req.done is None: time.sleep(self.sleeptime) self.sleeptime *= 2 # double backoff period each time. req.status_results_json = req.avr_session.get(req.results_status_url) req.status_results = json.loads(req.status_results_json.text) if req.status_results['status'] == 'FAILED': req.result = avr_resp() req.result_error['layer'] = 'REST' req.result_error['error'] = req.status_results['status'] req.result_error['text'] = req.status_results if req.status_results['status'] == 'FINISHED': req.raw_results_url = req.status_results['reportResultsLink'] req.results_url = req.raw_results_url.replace('localhost', self.host_name) req.results = self.avr_session.get(req.results_url) if req.results.status_code == 200: req.result = avr_resp() req.result.update(json.loads(req.results.text)) req.done = True self.res_queue.append(req.result) self.num_requests -= 1 else: req.result = avr_resp() req.result_error.layer = 'HTTP' req.result_error.code = req.results.status_code req.result_error.text = req.results self.res_queue.append(req.result) if i == 5: if req.result == False: req.result = avr_resp() req.result.error_layer = 'REST_AVR' req.result.error_error = '408' req.result.error_text = 'TIMEOUT' if self.num_requests == 0: break return self.res_queue ShowAVRJsonApi = """ reportFeatures -------------- Specifies the kind of information that appears in a response from AVR. You may specify one or more of the following values: existing-entities time-aggregated time-series entities-count viewDimensions -------------- Specifies the dimensions for which to calculate a report, such as: {"dimensionName": "domain-name"} You may only specify a single dimension. You may omit this field in a report generation request. viewMetrics ----------- Specifies the list of metrics by which to sort results, such as: { "metricName": "average-tps" }, { "metricName": "transactions" } If you specify either time-aggregated or time-series features, you must specify one metric in a report generation request. sortByMetrics -------------- Specifies the list of metrics to sort by, such as: [{ metricName: "average-tps", order:"descending" } ] Valid values are ascending and descending. Sorting only applies to the time-aggregated feature. You do not need to specify this field in a report generation request. timeRange --------- Specifies the time range, in microseconds, for which to calculate a report, such as: {"from": 1410420888000000, "to": 1410424488000000 } You do not need to specify this field in a report generation request. entityFilters ============= Specifies the entities and values for which to calculate a report. You can specify a single entity with a second level of dimension filters that describe an aspect of the entity. If you specify multiple entity types, the results include only the entities that match all of the criteria. You do not need to specify this field in a report generation request. The following snippet contains two entities with corresponding values: [[{ "dimensionName" : "virtual", "predicate": "OPERATOR_TYPE_EQUAL", "values : ["phpAuction_VS_1"] }, { "dimensionName : "response-code", "predicate": "OPERATOR_TYPE_EQUAL", "values" : ["200"] } ]] metricFilters ------------- Specifies the metric filters for which to calculate a report, such as: [{ "metricName": "transactions", "predicate" : metricFilters "OPERATOR_TYPE_GREATER_THAN" "value": 100 }] You do not need to specify this field in a report generation request. For the existing-entities feature, AVR supports the OPERATOR_TYPE_LIKE predicate. AVR also supports the following predicates: OPERATOR_TYPE_EQUAL OPERATOR_TYPE_NOT_EQUAL OPERATOR_TYPE_GREATER_THAN OPERATOR_TYPE_LOWER_THAN OPERATOR_TYPE_GREATER_THAN_OR_EQUAL OPERATOR_TYPE_LOWER_THAN_OR_EQUAL pagination ---------- Specifies the number of results to return, and the number of results to skip, such as: { numberOfResults : 10, skipResults : 10} To see the second set of ten results, use the example shown here. AVR does not implement the OData query parameters top or skip. In order to see a specific set of results, you must set the number of results to return and then determine how many results to skip. You do not need to specify this field in a report generation request. """ Tested this on version: 12.0
BIG-IQ Central Management API automation and programmability - Workflow - Python
Summary In conjunction with the announcement of BIG-IQ 5.0 we are excited to bring the field greater flexibility when centrally managing BIG-IP devices utilizing well defined workflows and the BIG-IQ iControl REST API. We understand that automation and programmability is becoming more the norm these days as the network is evolving into a software defined application aware environment. This article will provide a basic example for first establishing trust while adding a device to inventory, second importing device configuration into BIG-IQ's configuration database and finally adding device to a license pool all utilizing the BIG-IQ rest api uri provided by the interface. Response is parsed in JSON format so there are dependencies required. This is intended to be an introduction to leveraging the API directly. We are in the process of developing a python object abstraction for the BIG-IQ CM REST API to provide even greater simplicity when leveraging automation for central management programmability. But we thought this level of detail may be appropriate to get the ball rolling. The BIG-IQ CM SDK will be structured similar to version 0.1.7 BIG-IP iControl REST API described here: BIGIP SDK - https://devcentral.f5.com/s/articles/f5-friday-python-sdk-for-big-ip-18233 BIGIP SDK Docs - https://f5-sdk.readthedocs.io/en/v0.1.7/ Let's get started ... READY ... SET ... GO ... Here are the API URIs' used as reference for both Python and Perl examples below: Trust:https://'bigiq-ip'/mgmt/cm/global/tasks/device-trust Discovery:https://'bigiq-ip'/mgmt/cm/global/tasks/device-discovery Import: ADC - https://'bigiq-ip'/mgmt/cm/adc-core/tasks/declare-mgmt-authority Import: Firewall - https://'bigiq-ip'/mgmt/cm/firewall/tasks/declare-mgmt-authority License: Pools - https://'bigiq-ip'/mgmt/cm/shared/licensing/pools License: Members: - https://'bigiq-ip'/mgmt/cm/shared/licensing/pools/'member_uuid'/members Using a python class abstraction (libraries can be located in /lib/) for Discovery, Import and Licensing, these tasks are a single call from one place which will provide the flexibility to create scripts defined by workflow. For example, lets - 1. Negotiate trust certificates for a newly deployed BIGIP. 2. Discover the BIGIP device and add to inventory. 3. Import ADC configuration and resolve all differences to BIGIQ’s management database. 4. Import Firewall configuration and resolve all differences to BIGIQ’s management database. 5. Add this newly discovered BIGIP to a provisioned license pool using the base-registration-key as a filter. Using the following python libraries, we can accomplish these tasks defined as methods in each class definition: ## Discovery ../lib/discovery.py: class Discover(object) def device_trust(dev_id): def ltm_discover(modules adc_core, firewall, asm, apm etc..): ## Import ../lib/import.py: class Import(object) def import_ltm(dev_id): def import_security(dev_id, afm=True): ## License ../lib/license.py : class License(object) def license_pool(config): Now it’s just a matter of putting the puzzle together in a script that will accomplish the workflow described above. BIG-IQ and BIG-IP device address and credentials are read in from a configuration file (../config/..) the script first and passed into def workflow for processing. Each method is called with argument <config> dictionary to perform the task required. After completion each will return True or False depending on result from rest response code and some evaluation. ## Import libraries to leverage methods described above from discovery import Discover from import_module import Import from license import License ## main def "workflow" def workflow (Discover, License, Import, config): ## Trust "establish certificate exchange" result_trust = Discover.device_trust(config) ## Discover "add a BIGIP to inventory" result = Discover.ltm_discover(config, result_trust) ## Import ADC "import adc configuration into database" result = Import.import_ltm(config, result_trust) ## Import AFM, ASM, APM "import module configuration into database" result = Import.import_sec(config, result_trust, afm=True) ## Add BIGIP member to license pool result = License.license_pool(config) ## Python main body if __name__ == '__main__': # read config file from ../config/.. config = {} with open (file) as infile: config[str(key)] = val.strip('\n') ## create an instance of each imported class defined in the libraries Discover = Discover(config) Import = Import(config) License = License(config) ## call the main program method "workflow" workflow (Discover, Import, License, config) If you are interested in this code for collaboration or solution, search on key words "bigiq" "api" "python" in code share section on dev central or you can refer to the reference link: Device trust, discovery and import using python requests - supported in Python version 2.7.9 and greater. https://devcentral.f5.com/s/articles/big-iq-trust-discovery-and-import-of-big-ip-using-rest-api-python-oo-973 Grants a license to a BIG-IP from an already-activated purchaced pool. https://devcentral.f5.com/s/articles/big-iq-licensing-of-big-ip-using-rest-api-python-oo-974 We will also be adding to github and will update this article once completed. So please come back and visit us soon and often for additional content.
