Learn F5 Technologies, Get Answers & Share Community Solutions Join DevCentral

Filter by:
  • Solution
  • Technology
Clear all filters
Answers

F5 BigIP: Python SDK & Exporting Importing ASM Policies

After a few hours of trying to make this work; I have to get some insight from the forum. I want to create a python script to export ASM policies every night from a PROD box and import/merge them into DR. I've spent a lot of time consuming the top searches for this and seem to run into some information that is dated. I'm so close but just can't finish putting it all together. I'll confess I'm not a seasoned Pythonista; but I do like the language and understand general structure of the syntax.

from f5.bigip import ManagementRoot
from f5.bigip.tm.asm import File_Transfer
from f5.bigip.tm.asm.tasks import Export_Policy
from f5.bigip.tm.asm.file_transfer import Downloads
from f5.bigip.tm.asm.file_transfer import Uploads

emgmt = ManagementRoot('bigip.PRODaddress.com', 'user', 'pass')
imgmt = ManagementRoot('bigip.DRaddress.com', 'user', 'pass')

policies = emgmt.tm.asm.policies_s.get_collection()

for policy in policies:
  #export policy; #
  policies.export_policy_xml(policy_name = policy.name, filename = policy.name+".xml")
  download_policy(export_server, policy+".xml", policy+".xml")
else:
    continue

After investigating many resources I realized I don't see anything in the SDK that refers to 'export_policy_xml' or 'download_policy'. However there is this f5.bigip.tm.asm.file_transfer module that has a Downloads and Uploads classes. I tried a few things in the interpreter to try to make this work and thought I had some progress.

>>> emgmt = ManagementRoot('bigip.PRODaddress.com', 'user', 'pass')
>>> policies = emgmt.tm.asm.policies_s.get_collection()
>>> export_object = Export_Policy(policies[3])
>>> export_object
<f5.bigip.tm.asm.tasks.Export_Policy object at 0x2c23d10>
>>> export_object.raw
{'_meta_data': {'reduction_forcing_pairs': [('enabled', 'disabled'), ('online', 'offline'), 
('vlansEnabled', 'vlansDisabled')], 'container': <f5.bigip.tm.asm.policies.Policy object at 0x2c2f190>, 
'minimum_additional_parameters': set(['inline', 'filename']), 'exclusive_attributes': ['filename', 
'inline'], 'allowed_commands': [], 'read_only_attributes': [], 'required_json_kind': 'tm:asm:tasks:export-
policy:export-policy-taskstate', 'bigip': <f5.bigip.ManagementRoot object at 0x7f486a304410>, 
'icontrol_version': '', 'required_command_parameters': set([]), 'icr_session': 
<icontrol.session.iControlRESTSession object at 0x7f486a304bd0>, 'required_load_parameters': set(['id']), 
'required_creation_parameters': set(['policyReference']), 'object_has_stats': False, 'minimum_version': 
'11.5.0'}}
>>> dir(export_object)
['Containers', '__class__', '__delattr__', '__dict__', '__doc__', '__format__', '__getattr__', 
'__getattribute__', '__hash__', '__init__', '__module__', '__new__', '__reduce__', '__reduce_ex__', 
'__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_activate_URI', 
'_assign_stats', '_build_final_uri', '_check_command_parameters', '_check_create_parameters', 
'_check_exclusive_parameters', '_check_for_boolean_pair_reduction', '_check_for_python_keywords', 
'_check_force_arg', '_check_generation', '_check_keys', '_check_load_parameters', 
'_check_supported_versions', '_create', '_delete', '_exists', '_format_collection_name', 
'_format_resource_name', '_get_base_uri', '_handle_requests_params', '_is_version_supported_method', 
'_iter_list_for_dicts', '_load', '_local_update', '_meta_data', '_minimum_one_is_missing', '_modify', 
'_prepare_put_or_patch', '_prepare_request_json', '_produce_instance', '_reduce_boolean_pair', '_refresh', 
'_set_meta_data_uri', '_stamp_out_core', '_to_dict', '_traverse', '_traverse_dict', '_update', 'attrs', 
'create', 'delete', 'exists', 'load', 'modify', 'raw', 'refresh', 'to_dict', 'transform_attr_names', 
'traversed', 'update']

If anyone could provide some insight into this I'd be very appreciative. I'm going to include a list of the resources I've already re-viewed so anyone who tries to poke at this won't think they found something new.

0
Rate this Question
Comments on this Question
Comment made 25-Jan-2018 by KajiMaster 11

Relevant (newer) research articles I've reviewed: | https://f5-sdk.readthedocs.io/en/latest/apidoc/f5.bigip.tm.asm.html?highlight=Download | https://f5-sdk.readthedocs.io/en/latest/userguide/file_transfers.html?highlight=Downloads | https://devcentral.f5.com/articles/getting-started-with-the-f5-common-python-sdk-27438

Things I think should have worked (I'm aware now this is prob just iControl and would require pycontrol?) | https://devcentral.f5.com/wiki/iControl.ASM__Policy__export_policy_xml.ashx

Some of the older articles from 2013 had solutions but they seem to use pycontrol. I played with this a bit and never could get ASM.Policy to populate a list: | https://devcentral.f5.com/questions/asm-policy-import-export

0

Answers to this Question

placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER

I took the liberty of writing this my way but it should give you an idea how to do this in your own:

from f5.bigip import ManagementRoot
from time import sleep

def task_checker(task):
    while True:
        task.refresh()
        if task.status in ['COMPLETED', 'FAILURE']:
            break
        sleep(1)
    assert task.status == 'COMPLETED'  # this will raise assertion error if export task fails
    return True

def download(mgmt, name, path=None):
    """This will download to the script directory,
       specify path to place it somewhere else
    """
    if path:
        filename = path+name
    else:
        filename = name
    mgmt.tm.asm.file_transfer.downloads.download_file(filename)

def iterate_policies(mgmt):
    policies = mgmt.tm.asm.policies_s.get_collection()
    for policy in policies:
        yield policy.name, policy.selfLink

def export(mgmt, filepath=None):
    path = filepath
    for name, link in iterate_policies(mgmt):
        task = mgmt.tm.asm.tasks.export_policy_s.export_policy.create(
            filename=name + '.xml',
            policyReference=link
        )
        if task_checker(task):
            download(mgmt, name, path=path)

emgmt = ManagementRoot('bigip.PRODaddress.com', 'user', 'pass')

export(emgmt, filepath='/usr/tmp/policies/')
0
Comments on this Answer
Comment made 01-Feb-2018 by KajiMaster 11

Wojciech,

Thanks so much for the assistance! I like how you incorporated the .create onto the export_policy. This was something I didn't think about earlier.

I tested out this code to see how it worked. Initially I got a 400 error due to what seems like a failure regarding the http call to the f5. Based on the details in the error I noticed the URI that the policy is assigned for the self.link makes an https://localhost reference. Since I'm running this script on a server with access to my F5s; I was thinking perhaps I need to do a replace on the link to sub out 'localhost' with the ip or FQDN of the target f5.

This also yielded the same error. Now i'm curious if the error is more specific with the Export Policy Task. Specificlly if i'm just not giving the task function enough information to work with.

>>> export(mgmt, filepath='/usr/tmp/policies/')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 6, in export
  File "/home/<user_name>/.virtualenvs/f5-policies/lib/python2.7/site-packages/f5/bigip/resource.py", line 1002, in create
    return self._create(**kwargs)
  File "/home/<user_name>/.virtualenvs/f5-policies/lib/python2.7/site-packages/f5/bigip/resource.py", line 968, in _create
    response = session.post(_create_uri, json=kwargs, **requests_params)
  File "/home/<user_name>/.virtualenvs/f5-policies/lib/python2.7/site-packages/icontrol/session.py", line 271, in wrapper
    raise iControlUnexpectedHTTPError(error_message, response=response)
icontrol.exceptions.iControlUnexpectedHTTPError: 400 Unexpected Error: Bad Request for uri: https://bigip.PRODaddress.com:443/mgmt/tm/asm/tasks/export-policy/
Text: u'{"code":400,"message":"Could not add the Export Policy Task \'Export Policy Task (UNNAMED)\'.  Failed validating value \'\' for policy: The value is empty","originalRequestBody":"{\\"policyReference\\": \\"https://localhost/mgmt/tm/asm/policies/Kdf832lkdsf8odfhjjiwuef?ver=13.1.0\\", \\"filename\\": \\"c_name.website.com_policy.xml\\"","referer":"<ip_where_script_will_live>","restOperationId":93635024,"kind":":resterrorresponse"}'

Items of interest to me:

"message":
"Could not add the Export Policy Task \
'Export Policy Task (UNNAMED)\
'.  Failed validating value \
'\' for policy: The value is empty"
0
Comment made 01-Feb-2018 by Wojciech Wypior

When I coded ASM into SDK I only tested up to 12.x which was the latest at the time perhaps export policy task requires something more in 13.1.x or it is a bug... I would need to try with curl

0
Comment made 02-Feb-2018 by KajiMaster 11

Interesting. Good to know. When I started working on this; a colleague had done some python scripting for communicating with the f5 using curl which worked without the SDK. But had to re-write a lot of the header to the f5 to get proper token for authentication. I was really impressed with ManagmentRoot call in the SDK and wanted to try avoiding re-writing if I could.

I would be willing leverage curl calls for the export to give me something to play around with. Do you think modifying this stanzas is the right approach:

 task = mgmt.tm.asm.tasks.export_policy_s.export_policy.create(
        filename=name + '.xml',
        policyReference=link
    )
    if task_checker(task):
        download(mgmt, name, path=path)

and

def download(mgmt, name, path=None):
    """This will download to the script directory,
       specify path to place it somewhere else
    """
    if path:
        filename = path+name
    else:
        filename = name
    mgmt.tm.asm.file_transfer.downloads.download_file(filename)

I'm a novice pythonista but i'm willing to spend the time poking if I can get pointed in the right direction. I appreciate your continued support!

0
placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER

Hi, I find the problem for the error 400. It's because in the code :

task = mgmt.tm.asm.tasks.export_policy_s.export_policy.create(
    filename=name + '.xml',
    policyReference=link
)

The link must be a dictionnary with this form : link_formated = {"link":link}

After this, it's ok

0