iControl 101 - #11 - Performance Graphs
The BIG-IP stores history of certain types of data for reporting purposes. The Performance item in the Overview tab shows the report graphs for this performance data. The management GUI gives you the options to customize the graph interval but it stops short of giving you access to the raw data behind the graphs. Never fear, the System.Statistics interface contains methods to query the report types and extract the CSV data behind them. You can even select the start and end times as well as the poll intervals. This article will discuss the performance graph methods and how you can query the data behind them and build a chart of your own.
Initialization
This article uses PowerShell and the iControl Cmdlets for PowerShell as the client environment for querying the data. The following setup will be required for the examples contained in this article.
PS> Add-PSSnapIn iControlSnapIn PS> Initialize-F5.iControl -Hostname bigip_address -Username bigip_user -Password bigip_pass PS> $SystemStats = (Get-F5.iControl).SystemStatistics
Now that that is taken care of, let's dive right in. In the System.Statistics interface, there are two methods that get you to the performance graph data. The first is the get_performance_graph_list() method. This method takes no parameters and returns a list of structures containing each graph name, title, and description.
PS> $SystemStats.get_performance_graph_list() graph_name graph_title graph_description ---------- ----------- ----------------- memory Memory Used Memory Used activecons Active Connections Active Connections newcons New Connections New Connections throughput Throughput Throughput httprequests HTTP Requests HTTP Requests ramcache RAM Cache Utilization RAM Cache Utilization detailactcons1 Active Connections Active Connections detailactcons2 Active PVA Connections Active PVA Connections detailactcons3 Active SSL Connections Active SSL Connections detailnewcons1 Total New Connections Total New Connections detailnewcons2 New PVA Connections New PVA Connections detailnewcons3 New ClientSSL Profile Connections New ClientSSL Profile Connections detailnewcons4 New Accepts/Connects New Accepts/Connects detailthroughput1 Client-side Throughput Client-side Throughput detailthroughput2 Server-side Throughput Server-side Throughput detailthroughput3 HTTP Compression Rate HTTP Compression Rate SSLTPSGraph SSL Transactions/Sec SSL Transactions/Sec GTMGraph GTM Performance GTM Requests and Resolutions GTMrequests GTM Requests GTM Requests GTMresolutions GTM Resolutions GTM Resolutions GTMpersisted GTM Resolutions Persisted GTM Resolutions Persisted GTMret2dns GTM Resolutions Returned to DNS GTM Resolutions Returned to DNS detailcpu0 CPU Utilization CPU Usage detailcpu1 CPU Utilization CPU Usage CPU CPU Utilization CPU Usage detailtmm0 TMM Utilization TMM Usage TMM TMM Utilization TMM CPU Utilization
Creating a Report
Ok, so you've now got the graph names, it's time to move on to accessing the data. The method you'll want is the get_performance_graph_csv_statistics() method. This method takes an array of PerformanceStatisticQuery structures containing the query parameters and returns an array of PerformanceGraphDataCSV structures, one for each input query. The following code illustrates how to make a simple query. The object_name corresponds to the graph_name in the get_performance_graph_list() method.
The start_time and end_time allow you to control what the data range is. Values of 0 (the default) will return the entire result set. If the user specifies a start_time, end_time, and interval that does not exactly match the corresponding value used within the database, the database will attempt to use to closest time or interval as requested. The actual values used will be returned to the user on output. For querying purposes, the start_time can be specified as: 0: in which case by default, it means 24 hours ago. N: where N represents the number of seconds since Jan 1, 1970. -N: where -N represents the number of seconds before now, for example: -3600 means 3600 seconds ago, or now - 3600 seconds. For querying purposes, the end_time can be specified as: 0: in which case by default, it means now. N: where N represents the number of seconds since Jan 1, 1970. -N: where -N represents the number of seconds before now, for example: -3600 means 3600 seconds ago, or now - 3600 seconds. The interval is the suggested sampling interval in seconds. The default of 0 uses the system default. The maximum_rows value allows you to limit the returned rows. Values are started at the start_time and if the number of rows exceeds the value of maximum_rows, then the data is truncated at the maximum_rows value. A value of 0 implies the default of all rows.
PS> # Allocate a new Query Object
PS> $Query = New-Object -TypeName iControl.SystemStatisticsPerformanceStatisticQuery
PS> $Query.object_name = "CPU"
PS> $Query.start_time = 0
PS> $Query.end_time = 0
PS> $Query.interval = 0
PS> $Query.maximum_rows = 0
PS> # Make method call passing in an array of size one with the specified query
PS> $ReportData = $SystemStats.get_performance_graph_csv_statistics( (,$Query) )
PS> # Look at the contents of the returned data.
PS> $ReportData
object_name : throughput
start_time : 1208354160
end_time : 1208440800
interval : 240
statistic_data : {116, 105, 109, 101...}
Processing the Data
The statistic_data, much like the ConfigSync's file transfer data, is transferred as a base64 encoded string, which translates to a byte array in .Net. We will need to convert this byte array into a string and that can be done with the System.Text.ASCIIEncoding class.
PS> # Allocate a new encoder and turn the byte array into a string PS> $ASCII = New-Object -TypeName System.Text.ASCIIEncoding PS> $csvdata = $ASCII.GetString($ReportData[0].statistic_data) PS> # Look at the resulting dataset PS> $csvdata timestamp,"CPU 0","CPU 1" 1208364000,4.3357230000e+00,0.0000000000e+00 1208364240,3.7098920000e+00,0.0000000000e+00 1208364480,3.7187980000e+00,0.0000000000e+00 1208364720,3.3311110000e+00,0.0000000000e+00 1208364960,3.5825310000e+00,0.0000000000e+00 1208365200,3.4826450000e+00,8.3330000000e-03 ...
Building a Chart
You will see the returned dataset is in the form of a comma separated value file. At this point you can take this data and import it into your favorite reporting package. But, if you want a quick and dirty way to see this visually, you can use PowerShell to control Excel into loading the data and generating a default report. The following code converts the csv format into a tab separated format, creates an instance of an Excel Application, loads the data, cleans it up, and inserts a default line graph based on the input data.
PS> # Replace commas with tabs in the report data and save to c:\temp\tabdata.txt
PS> $csvdata.Replace(",", "`t") > c:\temp\tabdata.txt
PS> # Allocate an Excel application object and make it visible.
PS> $e = New-Object -comobject "Excel.Application"
PS> $e.visible = $true
PS> # Load the tab delimited data into a workbook and get an instance of the worksheet it was inserted into.
PS> $wb = $e.Workbooks.Open("c:\temp\tabdata.txt")
PS> $ws = $wb.WorkSheets.Item(1)
PS> # Let's remove the first row of timestamps. Ideally you'll want this to be the
PS> # horizontal axis and I'll leave it up to you to figure that one out.
PS> $ws.Columns.Item(1).EntireColumn.Select()
PS> $ws.Columns.Item(1).Delete()
PS> # The last row of the data is filled with NaN to indicate the end of the result set. Let's delete that row.
PS> $ws.Rows.Item($ws.UsedRange.Rows.Count).Select()
PS> $ws.Rows.Item($ws.UsedRange.Rows.Count).Delete()
PS> # Select all the data in the worksheet, create a chart, and change it to a line graph.
PS> $ws.UsedRange.Select()
PS> $chart = $e.Charts.Add()
PS> $chart.Type = 4
Conclusion
Now you should have everything you need to get access to the data behind the performance graphs. There are many ways you could take these examples and I'll leave it to the chart gurus out there to figure out interesting ways to represent the data. If anyone does find some interesting ways to manipulate this data, please let me know!
