Learn F5 Technologies, Get Answers & Share Community Solutions Join DevCentral

Filter by:
  • Solution
  • Technology
Answers

Upload programatically external data group file?

Hi everyone,

I use external data group file contains more than 350K records. And now I would like to programmatically update (overwrite) the data group file for the automation process. I did some research and found something bigsuds library but the python version we used is not supported.

Does someone have any idea about uploading external data group file using python?

F5 version is 12.1.3.3 Python version is 3.7.0.

Thanks in advance.

--

Oguz

0
Rate this Question

Answers to this Question

placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER

Here's iControl REST curl examples. You should be able to find equivalent Python classes for these endpoints.

To upload a data-group file http://172.16.10.10/dg-testSw.txt from the remote http server to the BIG-IP and create the data-group file object (dg-testSw.txt), run

# curl -sku admin:<pass> https://<host>/mgmt/tm/sys/file/data-group \
  -X POST -H "Content-type: application/json" \
  -d '{"name":"dg-testSw.txt", "separator":":=", "type":"string", "sourcePath":"http://172.16.10.10/dg-testSw.txt"}'

which is equivalent to the tmsh command

# tmsh create sys file data-group dg-testSw.txt separator ":=" source-path http://172.16.10.10/dg-testSw.txt type string

The object dg-testSw.txt is created under /Common by default. You can specify partition: field to the above request for a different patition. The fields in the file dg-testSw.txt are separated by := and the type is string in this example: e.g.,

"darth" := "vader",
"luke" := "skywalker",
"han" := "solo",
"leia" := "organa",
"moff" := "tarkin",

Please refer to v11: iRules Data Group Updates for the data format.

To create the ltm data-group dg-testSw from the above file (dg-testSw.txt), run

# curl  -sku admin:<pass> https://<host>/mgmt/tm/ltm/data-group/external \
  -X POST -H "Content-type: application/json" \
  -d '{"name":"dg-testSw", "externalFileName":"dg-testSw.txt"}'

which is equivalent to the tmsh command

# tmsh create ltm data-group external dg-testSw external-file-name dg-testSw.txt

To delete the ltm data-group object dg-testSw

# curl -sku admin:<pass> https://<host>/mgmt/tm/ltm/data-group/external/dg-testSw -X DELETE

To delete the data-group file object dg-testSw.txt

# curl -sku admin:<pass> https://<host>/mgmt/tm/sys/file/data-group/dg-testSw.txt -X DELETE
0
Comments on this Answer
Comment made 08-Aug-2018 by oguzy 144

Hi Satoshi,

Thank you for your response and interest. After a deep research, I found different solutions and combined them.

Update External Datagroups

F5 Python SDK - File Transfers

0
placeholder+image
USER ACCEPTED ANSWER & F5 ACCEPTED ANSWER

Good to know that you found the solution.

Here's an alternative solution using F5 Python SDK for iControl REST. While the example below does not give you much advantage over the Jason's pure Python code (uses 'request' module) in this use-case, you may find the SDK useful in some cases.

Just like the curl examples above, the code relies on a http server for uploading the data-group file to BIG-IP, not via a separate scp or POST to the /mgmt/shared/file-transfer/uploads/file.txt endpoint. BTW, Jason Rahm's another post, "Demystifying iControl REST Part 5: Transferring Files" is a good read on the topic.

Here's the code (Lazy me; I should have refactored it instead of copy-pasting the routines).

from f5.bigip import ManagementRoot

# Get all the 'sys file data-group' files
def getDGFiles(top):
    try:
        dgFiles = top.sys.file.data_groups.get_collection()
        print('Got {} file objects.'.format(len(dgFiles)))
        for idx, f in enumerate(dgFiles):
            print('{}: {}'.format(idx, f.raw))
    except Exception as ex:
        print('Error: {}'.format(ex))

# Upload the external data-group file
def uploadDGFile(top, name, path, dataType):
    print('---- Uploading the file {} as {}'.format(path, name))
    try:
        dgFile = top.sys.file.data_groups.data_group.create(sourcePath=path, name=name, type=dataType)
        print(dgFile.raw)
    except Exception as ex:
        print('Error: {}'.format(ex))

# Delete the external data-group file
def deleteDGFile(top, folder, name):
    print('---- Deleting file object /{}/{}'.format(folder, name))
    try:
        dgObject = top.sys.file.data_groups.data_group.load(name=name, partition=folder)
        dgObject.delete()
        print(dgObject.raw)
    except Exception as ex:
        print('Error: {}'.format(ex))

# Get all the 'ltm data-group external' objects
def getDGObjects(top):
    try:
        dgObjects = top.ltm.data_group.externals.get_collection()
        print('Got {} file objects.'.format(len(dgObjects)))
        for idx, o in enumerate(dgObjects):
            print('{}: {}'.format(idx, o.raw))
    except Exception as ex:
        print('Error: {}'.format(ex))

# Add the external data-group object
def createDGObject(top, name, file):
    print('---- Creating DG object {} from the file {}'.format(name, file))
    try:
        dgObject = top.ltm.data_group.externals.external.create(name=name, externalFileName=file)
        print(dgObject.raw)
    except Exception as ex:
        print('Error: {}'.format(ex))

# Delete the external data-group object
def deleteDGObject(top, folder, name):
    print('---- Deleting DG object /{}/{}'.format(folder, name))
    try:
        dgObject = top.ltm.data_group.externals.external.load(name=name, partition=folder)
        dgObject.delete()
        print(dgObject.raw)
    except Exception as ex:
        print('Error: {}'.format(ex))


# ==== Main routines =====
top = ManagementRoot('192.168.184.30', 'admin', 'admin').tm
#getDGFiles(top)
uploadDGFile(top, 'dg-testSw.txt', 'http://172.16.10.10/dg-testSw.txt', 'string')
#getDGObjects(top)
createDGObject(top, 'sat2', 'dg-testSw.txt')
#deleteDGObject(top, 'Common', 'sat2')
#deleteDGFile(top, 'Common', 'dg-testSw.txt')
0