5 Years Later: OpenAJAX Who?
Five years ago the OpenAjax Alliance was founded with the intention of providing interoperability between what was quickly becoming a morass of AJAX-based libraries and APIs. Where is it today, and why has it failed to achieve more prominence? I stumbled recently over a nearly five year old article I wrote in 2006 for Network Computing on the OpenAjax initiative. Remember, AJAX and Web 2.0 were just coming of age then, and mentions of Web 2.0 or AJAX were much like that of “cloud” today. You couldn’t turn around without hearing someone promoting their solution by associating with Web 2.0 or AJAX. After reading the opening paragraph I remembered clearly writing the article and being skeptical, even then, of what impact such an alliance would have on the industry. Being a developer by trade I’m well aware of how impactful “standards” and “specifications” really are in the real world, but the problem – interoperability across a growing field of JavaScript libraries – seemed at the time real and imminent, so there was a need for someone to address it before it completely got out of hand. With the OpenAjax Alliance comes the possibility for a unified language, as well as a set of APIs, on which developers could easily implement dynamic Web applications. A unified toolkit would offer consistency in a market that has myriad Ajax-based technologies in play, providing the enterprise with a broader pool of developers able to offer long term support for applications and a stable base on which to build applications. As is the case with many fledgling technologies, one toolkit will become the standard—whether through a standards body or by de facto adoption—and Dojo is one of the favored entrants in the race to become that standard. -- AJAX-based Dojo Toolkit , Network Computing, Oct 2006 The goal was simple: interoperability. The way in which the alliance went about achieving that goal, however, may have something to do with its lackluster performance lo these past five years and its descent into obscurity. 5 YEAR ACCOMPLISHMENTS of the OPENAJAX ALLIANCE The OpenAjax Alliance members have not been idle. They have published several very complete and well-defined specifications including one “industry standard”: OpenAjax Metadata. OpenAjax Hub The OpenAjax Hub is a set of standard JavaScript functionality defined by the OpenAjax Alliance that addresses key interoperability and security issues that arise when multiple Ajax libraries and/or components are used within the same web page. (OpenAjax Hub 2.0 Specification) OpenAjax Metadata OpenAjax Metadata represents a set of industry-standard metadata defined by the OpenAjax Alliance that enhances interoperability across Ajax toolkits and Ajax products (OpenAjax Metadata 1.0 Specification) OpenAjax Metadata defines Ajax industry standards for an XML format that describes the JavaScript APIs and widgets found within Ajax toolkits. (OpenAjax Alliance Recent News) It is interesting to see the calling out of XML as the format of choice on the OpenAjax Metadata (OAM) specification given the recent rise to ascendancy of JSON as the preferred format for developers for APIs. Granted, when the alliance was formed XML was all the rage and it was believed it would be the dominant format for quite some time given the popularity of similar technological models such as SOA, but still – the reliance on XML while the plurality of developers race to JSON may provide some insight on why OpenAjax has received very little notice since its inception. Ignoring the XML factor (which undoubtedly is a fairly impactful one) there is still the matter of how the alliance chose to address run-time interoperability with OpenAjax Hub (OAH) – a hub. A publish-subscribe hub, to be more precise, in which OAH mediates for various toolkits on the same page. Don summed it up nicely during a discussion on the topic: it’s page-level integration. This is a very different approach to the problem than it first appeared the alliance would take. The article on the alliance and its intended purpose five years ago clearly indicate where I thought this was going – and where it should go: an industry standard model and/or set of APIs to which other toolkit developers would design and write such that the interface (the method calls) would be unified across all toolkits while the implementation would remain whatever the toolkit designers desired. I was clearly under the influence of SOA and its decouple everything premise. Come to think of it, I still am, because interoperability assumes such a model – always has, likely always will. Even in the network, at the IP layer, we have standardized interfaces with vendor implementation being decoupled and completely different at the code base. An Ethernet header is always in a specified format, and it is that standardized interface that makes the Net go over, under, around and through the various routers and switches and components that make up the Internets with alacrity. Routing problems today are caused by human error in configuration or failure – never incompatibility in form or function. Neither specification has really taken that direction. OAM – as previously noted – standardizes on XML and is primarily used to describe APIs and components - it isn’t an API or model itself. The Alliance wiki describes the specification: “The primary target consumers of OpenAjax Metadata 1.0 are software products, particularly Web page developer tools targeting Ajax developers.” Very few software products have implemented support for OAM. IBM, a key player in the Alliance, leverages the OpenAjax Hub for secure mashup development and also implements OAM in several of its products, including Rational Application Developer (RAD) and IBM Mashup Center. Eclipse also includes support for OAM, as does Adobe Dreamweaver CS4. The IDE working group has developed an open source set of tools based on OAM, but what appears to be missing is adoption of OAM by producers of favored toolkits such as jQuery, Prototype and MooTools. Doing so would certainly make development of AJAX-based applications within development environments much simpler and more consistent, but it does not appear to gaining widespread support or mindshare despite IBM’s efforts. The focus of the OpenAjax interoperability efforts appears to be on a hub / integration method of interoperability, one that is certainly not in line with reality. While certainly developers may at times combine JavaScript libraries to build the rich, interactive interfaces demanded by consumers of a Web 2.0 application, this is the exception and not the rule and the pub/sub basis of OpenAjax which implements a secondary event-driven framework seems overkill. Conflicts between libraries, performance issues with load-times dragged down by the inclusion of multiple files and simplicity tend to drive developers to a single library when possible (which is most of the time). It appears, simply, that the OpenAJAX Alliance – driven perhaps by active members for whom solutions providing integration and hub-based interoperability is typical (IBM, BEA (now Oracle), Microsoft and other enterprise heavyweights – has chosen a target in another field; one on which developers today are just not playing. It appears OpenAjax tried to bring an enterprise application integration (EAI) solution to a problem that didn’t – and likely won’t ever – exist. So it’s no surprise to discover that references to and activity from OpenAjax are nearly zero since 2009. Given the statistics showing the rise of JQuery – both as a percentage of site usage and developer usage – to the top of the JavaScript library heap, it appears that at least the prediction that “one toolkit will become the standard—whether through a standards body or by de facto adoption” was accurate. Of course, since that’s always the way it works in technology, it was kind of a sure bet, wasn’t it? WHY INFRASTRUCTURE SERVICE PROVIDERS and VENDORS CARE ABOUT DEVELOPER STANDARDS You might notice in the list of members of the OpenAJAX alliance several infrastructure vendors. Folks who produce application delivery controllers, switches and routers and security-focused solutions. This is not uncommon nor should it seem odd to the casual observer. All data flows, ultimately, through the network and thus, every component that might need to act in some way upon that data needs to be aware of and knowledgeable regarding the methods used by developers to perform such data exchanges. In the age of hyper-scalability and über security, it behooves infrastructure vendors – and increasingly cloud computing providers that offer infrastructure services – to be very aware of the methods and toolkits being used by developers to build applications. Applying security policies to JSON-encoded data, for example, requires very different techniques and skills than would be the case for XML-formatted data. AJAX-based applications, a.k.a. Web 2.0, requires different scalability patterns to achieve maximum performance and utilization of resources than is the case for traditional form-based, HTML applications. The type of content as well as the usage patterns for applications can dramatically impact the application delivery policies necessary to achieve operational and business objectives for that application. As developers standardize through selection and implementation of toolkits, vendors and providers can then begin to focus solutions specifically for those choices. Templates and policies geared toward optimizing and accelerating JQuery, for example, is possible and probable. Being able to provide pre-developed and tested security profiles specifically for JQuery, for example, reduces the time to deploy such applications in a production environment by eliminating the test and tweak cycle that occurs when applications are tossed over the wall to operations by developers. For example, the jQuery.ajax() documentation states: By default, Ajax requests are sent using the GET HTTP method. If the POST method is required, the method can be specified by setting a value for the type option. This option affects how the contents of the data option are sent to the server. POST data will always be transmitted to the server using UTF-8 charset, per the W3C XMLHTTPRequest standard. The data option can contain either a query string of the form key1=value1&key2=value2 , or a map of the form {key1: 'value1', key2: 'value2'} . If the latter form is used, the data is converted into a query string using jQuery.param() before it is sent. This processing can be circumvented by setting processData to false . The processing might be undesirable if you wish to send an XML object to the server; in this case, change the contentType option from application/x-www-form-urlencoded to a more appropriate MIME type. Web application firewalls that may be configured to detect exploitation of such data – attempts at SQL injection, for example – must be able to parse this data in order to make a determination regarding the legitimacy of the input. Similarly, application delivery controllers and load balancing services configured to perform application layer switching based on data values or submission URI will also need to be able to parse and act upon that data. That requires an understanding of how jQuery formats its data and what to expect, such that it can be parsed, interpreted and processed. By understanding jQuery – and other developer toolkits and standards used to exchange data – infrastructure service providers and vendors can more readily provide security and delivery policies tailored to those formats natively, which greatly reduces the impact of intermediate processing on performance while ensuring the secure, healthy delivery of applications. API Jabberwocky: You Say Tomay-to and I Say Potah-to OpenAjax Metadata 1.0 and the Adobe Dreamweaver Widget Browser OpenAjax Alliance AJAX-based Dojo Toolkit The Stealthy Ascendancy of JSON JSON Continues its Winning Streak Over XML JSON versus XML: Your Choice Matters More Than You Think I am in your HTTP headers, attacking your application The Web 2.0 API: From collaborating to compromised IT as a Service: A Stateless Infrastructure Architecture Model JSON Activity Streams and the Other Consumerization of ITXML Threat Prevention
Where should security live? The divide between operations and application developers is pretty wide, especially when it comes to defining who should be ultimately responsible for application security. Mike Fratto and I have often had lively discussions (read: arguments) about whether security is the responsibility of the developer or the network and security administrators. It's wholly inappropriate to recreate any of these discussions here, as they often devolve to including the words your mother said not to use in public. 'Nuff said. The truth is that when XML enters the picture then the responsibility for securing that traffic has to be borne by both the network/security administrators and the developers. While there is certainly good reason to expect that developers are doing simply security checks for buffer overflows, length restrictions on incoming data, and strong typing, the fact is that there are some attacks in XML that make it completely impractical to check for in the code. Let's take a couple of attack types as examples. XML Entity Expansion This attack is a million laughs, or at least a million or more bytes of memory. Applications need to parse XML in order to manipulate it, so the first thing that happens when XML hits an application is that it is parsed - before the developer even has a chance to check it. In an application server this is generally done before the arguments to the specific operation being invoked are marshaled - meaning it is the application server, not the developer that is responsible for handling this type of attack. These messages can be used to force recursive entity expansion or other repeated processing that exhausts server resources. The most common example of this type of attack is the "billion laughs" attack, which is widely available. The CPU is monopolized while the entities are being expanded, and each entity takes up X amount of memory - eventually consuming all available resources and effectively preventing legitimate traffic from being processed. It's essentially a DoS (Denial of Service) attack. ... ]> &ha128; It is accepted that almost all traditional DoS attacks (ping of death, teardrop, etc...) should be handled by a perimeter security device - a firewall oran application delivery controller - so why should a DoS attack that is perpetrated through XML be any different? It shouldn't. This isn't a developer problem, it's a parser problem and for the most part developers have little or no control over the behavior of the parser used by the application server. The application admin, however, can configure most modern parsers to prevent this type of attack, but that's assuming that their parser is modern and can be configured to handle it. Of course then you have to wonder what happens if that arbitrary limit inhibits processing of valid traffic? Yeah, it's a serious problem. SQL Injection SQL Injection is one of the most commonly perpetrated attacks via web-based applications. It consists basically of inserting SQL code into string-based fields which the attacks thinks (or knows) will be passed to a database as part of an SQL query. This type of attack can easily be accomplished via XML as well simply by inserting the appropriate SQL code into a string element. Aha! The developer can stop this one, you're thinking. After all, the developer has the string and builds the SQL that will be executed, so he can just check for it before he builds the string and sends it off for execution. While this is certainly true, there are myriad combinations of SQL commands that might induce the database to return more data than it should, or to return sensitive data not authorized to the user. This extensive list of commands and combinations of commands would need to be searched for in each and every parameter used to create an SQL query and on every call to the database. That's a lot of extra code and a lot of extra processing - which is going to slow down the application and impede performance. And when a new attack is discovered, each and every function and application needs to be updated, tested, and re-deployed. I'm fairly certain developers have better ways to spend their time than updating parameter checking in every function in every application they have in production. And we won't even talk about third-party applications and the dangers inherent in that scenario. One of the goals of SOA is engendering reuse, and this is one of the best examples of taking advantage of reuse in order to ensure consistent behavior between applications and to reduce the lengthy development cycle required to update, test, and redeploy whenever a new attack is discovered. By placing the onus for keeping this kind of attack from reaching the server on an edge device such as an application firewall like F5's application firewall, updates to address new attacks are immediately applied to all applications and there is no need to recode and redeploy applications. Although there are some aspects of security that are certainly best left to the developer, there are other aspects of security that are better deployed in the network. It's the most effective plan in terms of effort, cost, and consistent behavior where applications are concerned. Imbibing: Mountain Dew Technorati tags: security, application security, application firewall, XML, developers, networking, application deliveryLightboard Lessons: OWASP Top 10 - XML External Entities
The OWASP Top 10 is a list of the most common security risks on the Internet today. XML External Entities comes in at the #4spot in the latest edition of the OWASP Top 10. In this video, John discusses this security riskand outlines some mitigation steps to make sure your web application doesn't process malicious XML data and expose sensitive information. Related Resources: Securing against the OWASP Top 10: XML External Entity attacksLayer 7 Switching + Load Balancing = Layer 7 Load Balancing
Modern load balancers (application delivery controllers) blend traditional load-balancing capabilities with advanced, application aware layer 7 switching to support the design of a highly scalable, optimized application delivery network. Here's the difference between the two technologies, and the benefits of combining the two into a single application delivery controller. LOAD BALANCING Load balancing is the process of balancing load (application requests) across a number of servers. The load balancer presents to the outside world a "virtual server" that accepts requests on behalf of a pool (also called a cluster or farm) of servers and distributes those requests across all servers based on a load-balancing algorithm. All servers in the pool must contain the same content. Load balancers generally use one of several industry standard algorithms to distribute request. Some of the most common standard load balancing algorithms are: round-robin weighted round-robin least connections weighted least connections Load balancers are used to increase the capacity of a web site or application, ensure availability through failover capabilities, and to improve application performance. LAYER 7 SWITCHING Layer 7 switching takes its name from the OSI model, indicating that the device switches requests based on layer 7 (application) data. Layer 7 switching is also known as "request switching", "application switching", and "content based routing". A layer 7 switch presents to the outside world a "virtual server" that accepts requests on behalf of a number of servers and distributes those requests based on policies that use application data to determine which server should service which request. This allows for the application infrastructure to be specifically tuned/optimized to serve specific types of content. For example, one server can be tuned to serve only images, another for execution of server-side scripting languages like PHP and ASP, and another for static content such as HTML , CSS , and JavaScript. Unlike load balancing, layer 7 switching does not require that all servers in the pool (farm/cluster) have the same content. In fact, layer 7 switching expects that servers will have different content, thus the need to more deeply inspect requests before determining where they should be directed. Layer 7 switches are capable of directing requests based on URI, host, HTTP headers, and anything in the application message. The latter capability is what gives layer 7 switches the ability to perform content based routing for ESBs and XML/SOAP services. LAYER 7 LOAD BALANCING By combining load balancing with layer 7 switching, we arrive at layer 7 load balancing, a core capability of all modern load balancers (a.k.a. application delivery controllers). Layer 7 load balancing combines the standard load balancing features of a load balancing to provide failover and improved capacity for specific types of content. This allows the architect to design an application delivery network that is highly optimized to serve specific types of content but is also highly available. Layer 7 load balancing allows for additional features offered by application delivery controllers to be applied based on content type, which further improves performance by executing only those policies that are applicable to the content. For example, data security in the form of data scrubbing is likely not necessary on JPG or GIF images, so it need only be applied to HTML and PHP. Layer 7 load balancing also allows for increased efficiency of the application infrastructure. For example, only two highly tuned image servers may be required to meet application performance and user concurrency needs, while three or four optimized servers may be necessary to meet the same requirements for PHP or ASP scripting services. Being able to separate out content based on type, URI, or data allows for better allocation of physical resources in the application infrastructure.The BIG-IP Application Security Manager Part 5: XML Security
This is the fifth article in a 10-part series on the BIG-IP Application Security Manager (ASM). The first four articles in this series are: What is the BIG-IP ASM? Policy Building The Importance of File Types, Parameters, and URLs Attack Signatures This fifth article in the series will discuss the basic concepts of XML and how the BIG-IP ASM provides security for XML. XML Concepts The Extensible Markup Language (XML) provides a common syntax for data transfer between similar systems. XML doesn't specify how to display data (HTML is used for that), but rather it is concerned with describing data that can be manipulated and presented using other languages. XML documents are built on a core set of basic nested structures, and developers can decide how tags are named and organized. XML is used extensively in web applications today, so it's important to have a basic understanding as well as a strong defense for this critical technology. The XML specification (described in this W3C publication) defines an XML document to be well-formed when it satisfies a list of syntax rules provided in the specification. If an XML processor encounters a violation of these rules, it is required to stop processing the file and report the error. A valid XML document is defined as a well-formed document that also conforms to the rules of a schema like the Document Type Definition (DTD) or the newer and more powerful XML Schema Definition (XSD). It's important to have valid XML documents when implementing and using web services. Web Service A web service is any service that is available over a network and that uses standardized XML syntaxes. You've heard of the "... as a Service" right? Well, this is the stuff we're talking about, and XML plays a big role. On a somewhat tangential note, it seems like there are too many "as a Service" acronyms flying around right now...I really need to make up a hilarious one just for the heck of it. I'll let you know how that goes... Anyway, back to reality...a web service architecture consists of a service provider, a service requestor, and a service registry. The service provider implements the service and publishes the service to the service registry using Universal Description, Discovery, and Integration (UDDI) which is an XML-based registry that allows users to register and locate web service applications. The service registry centralizes the services published by the service provider. The service requestor finds the service using UDDI and retrieves the Web Services Definition Language (WSDL) file, which consists of an XML-based interface used for describing the functionality offered by the web service. The service requestor is able to consume the service based on all the goodness found in the WSDL using the UDDI. Then, the service requestor can send messages to the service provider using a service transport like the Simple Object Access Protocol (SOAP). SOAP is a protocol specification for exchanging structured information when implementing web services...it relies on XML for its message format. Now you can see why XML is so closely tied to Web Services. All this craziness is shown in the diagram below. I know what you're thinking...it's difficult to find anything more exciting than this topic! (Picture copied from Wikipedia) Because XML is used for data transfer in the web services architecture, it's important to inspect, validate, and protect XML transactions. Fortunately, the BIG-IP ASM can protect several applications including: Web services that use HTTP as a transport layer for XML data Web services that use encryption and decryption in HTTP requests Web services that require verification and signing using digital signatures Web applications that use XML for client-server data communications (i.e. Microsoft Outlook Web Access) ASM Configuration Before you can begin protecting your XML content, you have to create a security policy using the "XML and Web Services" option. After you create the security policy, you create an XML profile and associate it with the XML security policy. You can read more about creating policies in the Policy Building article in this series. To create an XML profile, you navigate to Application Security >> Content Profiles >> XML Profiles. When all this is done, the XML profile will protect XML applications in the following ways: Validate XML formatting Mask sensitive data Enforce compliance with XML schema files or WSDL documents Provide information leakage protection Offer XML encryption and XML signatures Offer XML content based routing and XML switching Offer XML parser protection against DoS attacks Encrypt and decrypt parts of SOAP web services Validation resources provide the ASM with critical information about the XML data or web services application that the XML profile is protecting. As discussed earlier, many XML applications have a schema file for validation (i.e. DTD or XSD) or WSDL file that describes the language used to communicate with remote users. The XML profile is used to validate whether the incoming traffic complies with the predefined schemas or WSDL files. The following screenshot shows the configuration of the XML profile in the ASM. Notice all the different features it provides. You can download the all-important configuration files (WSDL), you can associate attack signatures to the profile (protects against things like XML parser attacks -- XML Bombs or External Entity Attacks), you can allow/disallow meta characters, and you can configure sensitive data protection for a specific namespace and a specific element or attribute. Another really cool thing is that most of these features are turned on/off using simple checkboxes. This is really cool and powerful stuff! I won't bore you with all the details of each setting, but suffice it to say, this thing let's you do tons of great things in order to protect your XML data. Well, that does it for this ASM article. I hope this sheds some light on how to protect your XML data. And, if you're one of the users who implements anything "as a Service" make sure you protect all that data by turning on the BIG-IP ASM. The next time someone throws an XML bomb your way, you'll be glad you did! Update: Now that the article series is complete, I wanted to share the links to each article. If I add any more in the future, I'll update this list. What is the BIG-IP ASM? Policy Building The Importance of File Types, Parameters, and URLs Attack Signatures XML Security IP Address Intelligence and Whitelisting Geolocation Data Guard Username and Session Awareness Tracking Event Logging
XML Scripts to deploy 3-Tier Application with Cisco APIC and F5 BIG-IP LTM [End of Life]
The F5 and Cisco APIC integration based on the device package and iWorkflow is End Of Life. The latest integration is based on the Cisco AppCenter named ‘F5 ACI ServiceCenter’. Visit https://f5.com/cisco for updated information on the integration. As described in a previous article Under the hood of F5 BIG-IP LTM and Cisco ACI integration – Role of the device package , Cisco APIC provides the user with the ability to define a service graph to automate L4-L7 service insertion using F5 BIG-IP device package. In this article, learn how to deploy an application with Cisco APIC policy model and F5 BIG-IP LTM device package using Northbound APIs (XML) scripts. Let's look at the different APIC logical constructs before diving into the cookbooks of scripting. Application Policy Infrastructure Controller (APIC) Policy Model The Application Centric Infrastructure policy model provides a convenient way to specify application requirements, which the APIC then renders in the network infrastructure. The policy model consists of a number of constructs such as tenants, contexts, bridge domains, end point groups and service graphs. When a user or process initiates an administrative change to an object within the fabric, that change is first applied to the ACI policy model and then applied to the actual managed end point .All physical and logical components of the ACI fabric are represented as a hierarchical Management Information Tree (MIT). Some of the key components contained within the MIT are shown in the flow diagram Tenant A tenant is essentially a ‘container’, used to house other constructs and objects in the policy model (such as contexts, bridge domains, contracts, filters and application profiles). Tenants can be completely isolated from each other, or can share resources. A tenant can be used to define administrative boundaries – administrators can be given access to specific tenants only, resulting in other tenants being completely inaccessible to them Learn how to Create Tenant SJC Learn how to Create Tenant LAX Contexts A context is used to define a unique layer 3 forwarding domain within the fabric. One or more contexts can be created inside a tenant. A context is also known as a ‘private network’ and can be viewed as the equivalent of a VRF in the traditional networking world. As each context defines a separate layer 3 domain, IP addresses residing within a context can overlap with addresses in other contexts. Bridge Domains and Subnets A bridge domain is a construct used to define a layer 2 boundary within the fabric. A BD can be viewed as somewhat similar to regular VLANs in a traditional switching environment. BDs however are not subject to the same scale limitations as VLANs, and have a number of enhancements such as improved handling of ARP requests and no flooding behavior by default. A subnet defines the gateway(s) that will be used within a given bridge domain. This gateway will typically be used by hosts associated with a bridge domain as their first hop gateway. Gateways defined within a bridge domain are pervasive across all leaf switches where that bridge domain is active. End Point Groups (EPG) The End Point Group (EPG) is one of the most important objects in the policy model and is used to define a collection of end points. An end point is a device connected to the fabric (either directly or indirectly) and has an address, a location and other attributes. End points are grouped together into an EPG, where policy can be more easily applied consistently across the ACI fabric. An end point may be classified into an EPG based on a number of criteria, including: • Virtual NIC • Physical leaf port • VLAN Contracts A contract is a policy construct used to define the communication between End Point Groups (EPGs). Without a contract between EPGs, no communication is possible between those EPGs. Within an EPG, a contract is not required to allow communication as this is always allowed. An EPG will provide or consume a contract (or provide and consume different contracts). For example, EPG “Web” in the XML scripts will provide a contract which EPG “App” will consume. Similarly, EPG “App” provides separate contracts which are consumable by the “Web” and “DB” EPGs. Learn how to create contracts for Tenant SJC Learn how to create contracts for Tenant LAX Filters A filter is a rule specifying fields such as TCP port, protocol type, etc. and is referenced within a contract to define the communication allowed between EPGs in the fabric. A filter contains one or more “filter entries” that actually specify the rule. Subjects A subject is a construct contained within a contract and which typically references a filter. For example, contract “Web” contains a subject named “Web-Subj”, which references a filter named “Web-filter”. Application Profile The Application Profile is the policy construct that ties multiple EPGs together with contracts that each EPG provides or consumes. An application profile contains as many EPGs as necessary that logically relate to the capabilities provided by an application. Learn how to create Application Profile for Tenant SJC Learn how to create Application Profile for Tenant LAX Service Graph A service graph is a chain of service functions such as Web application Firewall (WAF), Load balancer or network firewall including the sequence with which the service functions need to be applied. The graph defines these functions based on a user-defined policy for a particular application. One or more service appliances might be needed to render the services required by the service graph. Learn how to create Service Graph "WebGraph" and how to attach the graph to contract in Tenant SJC Learn how to create Service Graph "WebGraph" andhow to attach the graph to contract in Tenant LAX Creating a Device Cluster Learn how to create Logical Device with device type Physicalunder Tenant mgmt Learn how to create F5 BIG-IP LTM concrete devices under the device clusterand confuring high availability Learn how to bind the logical interfaces with physical interfaces of BIG-IP LTM Exporting a Device Cluster to Tenant SJC and LAX from Tenant mgmt Learn how to export the device cluster created in Tenant mgmt to Tenant SJC Learn how to export the device cluster created in Tenant mgmt to Tenant LAX Setting up the Fabric for service Insertion Learn how to setup the VMM domain to integrate APIC with VMware VCenter environment to run BIG-IP LTM VE or Server VMs Learn how to setup the physical domain and assigning the vlan namespace to enable datapath forwarding on leaf switches Learn how to setup vlan namespace to dynamically assign the vlans to end points Wondering how to run these scripts ? Here is the recipe, run the two scripts below within python environment and verify the configurations on Cisco APIC and F5 BIG-IP LTM. Make sure you have a device package downloaded from download.f5.com and saved in the same directory with the scripts 1. python request.py infra.cfg 2. python request.py tenant.cfg Complete XML scripts directort can be downloaded from here . Video (showing the configuration through APIC Graphical User Interface) The recorded video here shows how to configure the ACI policy model to deploy an application in Cisco APIC and BIG-IP LTM through graphical user interface.
Heatmaps, iRules Style: Part2
Last week I talked about generating a heat map 100% via iRules, thanks to the geolocation magic in LTM systems, and the good people over at Google letting us use their charting API. This was an outstanding way to visualize the the traffic coming to your application. For those interested in metrics it provides a great way to see this data in a visually pleasing manner. That said, it was pretty basic. All it showed was the United States which, for anyone that has used the internet much, is obviously not representative of the entire web. To be truly useful we’ll need to show the entire world. That’s simple enough. We’ll update the region the map we’re drawing zooms to, so it will look more like this: Let’s take a look at how this is going to work. Since we were collecting data based on state abbreviations before, we’ll need to first switch that up to use country codes instead. We’ll then change up our Google call so that we’re setting the range covered by the map to the entire world, rather than just the US. While we’re at it, let’s change the name of the subtable we’re using from states to countries, just to keep things more clear. What we end up with is some code that looks very familiar, if you’ve already seen last week’s solution, with a few minor changes: set chld "" set chd "" foreach country [table keys -subtable countries] { append chld $country append chd "[table lookup -subtable countries $country]," } set chd [string trimright $chd ","] HTTP::respond 200 content "<HTML><center><font size=5>Here is your site's usage by Country:</font><br><br><br><img src='http://chart.apis.google.com/chart?cht=t&chd=&chs=440x220&chtm=world&chd=t:$chd&chld=$chld&chco=f5f5f5,edf0d4,6c9642,365e24,13390a' border='0'><br>br><br><br><a href='/resetmap'>Reset All Counters</a></center></HTML>" So using that in place of the similar logic in last week’s solution you can get a simple world view of the traffic passing through your site. That’s great and all, but what if you can’t see the detail you’re looking for? What if you want to see the details of Asia’s traffic and be able to decipher the patterns in Japan and the Middle East? What we really need is to build a simple interface to make this more of an application, and less of a single image displayed on a web page. Well, first of all, we already have all the data collected that we’ll need, if you think about it. We’re already tracking the requests per country, so all we need to do is build out options to allow for users to click a link and zoom to a different region of the map. To do this we’ll set up some simple HTML navigation links at the bottom of the page being generated via the iRule, and set up a switch structure to handle each URI the links pass back into the iRule, and use those to format the HTML appropriately so that we get the right Google charts call. That sounds more complicated than it is. Here’s what it looks like: "/heatmap" { set chld "" set chd "" foreach country [table keys -subtable countries] { append chld $country append chd "[table lookup -subtable countries $country]," } set chd [string trimright $chd ","] HTTP::respond 200 content "<HTML><center><font size=5>Here is your site's usage by Country:</font><br><br><br><img src='http://chart.apis.google.com/chart?cht=t&chd=&chs=440x220&chtm=world&chd=t:$chd&chld=$chld&chco=f5f5f5,edf0d4,6c9642,365e24,13390a' border='0'><br><br>Zoom to region: <a href='/asia'>Asia</a> | <a href='/africa'>Africa</a> | <a href='/europe'>Europe</a> | <a href='/middle_east'>Middle East</a> | <a href='/south_america'>South America</a> | <a href='/usa'>United States</a> | <a href='/heatmap'>World</a><br><br><br><a href='/resetmap'>Reset All Counters</a></center></HTML>" "/asia" { set chld "" set chd "" foreach country [table keys -subtable countries] { append chld $country append chd "[table lookup -subtable countries $country]," } set chd [string trimright $chd ","] HTTP::respond 200 content "<HTML><center><font size=5>Here is your site's usage by Country:</font><br><br><br><img src='http://chart.apis.google.com/chart?cht=t&chd=&chs=440x220&chtm=asia&chd=t:$chd&chld=$chld&chco=f5f5f5,edf0d4,6c9642,365e24,13390a' border='0'><br><br>Zoom to region: <a href='/asia'>Asia</a> | <a href='/africa'>Africa</a> | <a href='/europe'>Europe</a> | <a href='/middle_east'>Middle East</a> | <a href='/south_america'>South America</a> | <a href='/usa'>United States</a> | <a href='/heatmap'>World</a><br><br><br><a href='/resetmap'>Reset All Counters</a></center></HTML>" … That section can be repeated once for each available region that Google will let us view (Asia | Africa | Europe | Middle East | South America | United States | World). That then gives us something that looks like this: As you can see, we now have a world view map that shows the heat of each country, and we have individual links that we can click on along the bottom to take us to a zoom of each country/region to get a more specific look at the info there. As an example, let’s take a look at the data from Asia: So we now have a nice little heatmapping application. It pulls up a world view of app traffic going to your site or app, it allows you to click around to the different regions of the world to get a more detailed view, and it even lets you re-set the data at will. I can hear some among you asking “What about the states, though?”. If I take away the state view of the US and give a world view, then I’m really trading one limitation for another. Ideally we’d be able to see both, right? If I want to be able to give a detailed view on both the countries around the world and the states within the US, then I need to expand my data collection a bit. I need to collect both country codes for incoming requests and state abbreviations, where applicable. This means creating a second sub-table within the iRule, and issuing a second whereis per request coming in. Something like this should do: set cloc [whereis [IP::client_addr] country] set sloc [whereis [IP::client_addr] abbrev] if {[table incr -subtable countries -mustexist $cloc] eq ""} { table set -subtable countries $cloc 1 indefinite indefinite } if {[table incr -subtable states -mustexist $sloc] eq ""} { table set -subtable states $sloc 1 indefinite indefinite } Above we’re using the cloc (country location) and sloc (state location) variables to simultaneously track both country codes and state abbreviations in separate sub tables within the iRule. This way we don’t mix up CA (Canada) and CA (California) or similar crossovers and throw our counts off. When doing this, don’t forget to update the resetmap case as well to empty both sub tables, not just one. This also means that we’ll need to slightly change the logic in the “usa” case as opposed to all of the other cases when doing a lookup. If the user wants to view the USA details, we need to do a subtable lookup on the states sub table, everything else uses the countries sub table. Not too horrible. Okay, we now have heatmaps for all countries, all available zoom regions and a zoom to state level in the US, complete with some rudimentary HTML to make this feel like an application, not just a static image on a web page. Unfortunately, we also have around 140 lines of code, much of which is being repeated. There’s no sense in repeating that HTML over and over, or those logic statements doing the lookups and whatnot. So it’s time to take out the scalpel and start slicing and dicing, looking for unnecessary code. I started with the HTML. There’s just no reason to repeat that HTML in every single switch case. So I set that in some static variables in the RULE_INIT section and did away with that all together in each switch case. Next, the actual iRules logic is identical if I want to view asia or africa or europe or anything other than the US. The only difference is the HTML changing one word to tell the API where to zoom in. Using a little extra “zoom” logic, I was able to cut down most of that repetitive code as well, by having all of the switch cases other than the USA fall through to the world view case, giving us just two chunks of iRules logic to deal with. Not including the extra variables and tidbits, the core of those two chunks of logic are: foreach country [table keys -subtable countries] { append chld $country append chd "[table lookup -subtable countries $country]," } foreach state [table keys -subtable states] { append chld $state append chd "[table lookup -subtable states $state]," } Don’t stop there, though, there’s more to trim! With some more advanced trickery we can combine these two table lookups into a single piece of logic. When all is said and done, here is the final iRule trimmed down to fighting form with a single switch case handling the presentation of all the possible heatmaps generated by Google…pretty cool stuff: when RULE_INIT { set static::resp1 "<HTML><center><font size=5>Here is your site's usage by Country:</font><br><br><br><img src='http://chart.apis.google.com/chart?cht=t&chd=&chs=440x220&chtm=" set static::resp2 "&chco=f5f5f5,edf0d4,6c9642,365e24,13390a' border='0'><br><br>Zoom to region: <a href='/asia'>Asia</a> | <a href='/africa'>Africa</a> | <a href='/europe'>Europe</a> | <a href='/middle_east'>Middle East</a> | <a href='/south_america'>South America</a> | <a href='/usa'>United States</a> | <a href='/heatmap'>World</a><br><br><br><a href='/resetmap'>Reset All Counters</a></center></HTML>" } when HTTP_REQUEST { switch -glob [string tolower [HTTP::uri]] { "/asia" - "/africa" - "/europe" - "/middle_east" - "/south_america" - "/usa" - "/world" - "/heatmap*" { set chld "" set chd "" set zoom [string map {"/" "" "heatmap" "world"} [HTTP::uri]] ## Configure the table query to be based on the countries subtable or the states subtable ## if {$zoom eq "usa"} { set region "states" } else { set region "countries" } ## Get a list of all states or countries and the associated count of requests from that area ## foreach rg [table keys -subtable $region] { append chld $rg append chd "[table lookup -subtable $region $rg]," } set chd [string trimright $chd ","] ## Send back the pre-formatted response, set in RULE_INIT, combined with the map zoom, list of areas, and request count ## HTTP::respond 200 content "${static::resp1}${zoom}&chd=t:${chd}&chld=${chld}${static::resp2}" } "/resetmap" { foreach country [table keys -subtable countries] { table delete -subtable countries $country } foreach state [table keys -subtable states] { table delete -subtable states $state } HTTP::respond 200 Content "<HTML><center><br><br><br><br><br><br>Table Cleared.<br><br><br> <a href='/heatmap'>Return to Map</a></HTML>" } default { ## Look up country & state locations ## set cloc [whereis [IP::client_addr] country] set sloc [whereis [IP::client_addr] abbrev] ## If the IP doesn't resolve to anything, pick a random IP (useful for testing on private networks) ## if {($cloc eq "") and ($sloc eq "")} { set ip [expr { int(rand()*255) }].[expr { int(rand()*255) }].[expr { int(rand()*255) }].[expr { int(rand()*255) }] set cloc [whereis $ip country] set sloc [whereis $ip abbrev] } ## Set Country ## if {[table incr -subtable countries -mustexist $cloc] eq ""} { table set -subtable countries $cloc 1 indefinite indefinite } ## Set State ## if {[table incr -subtable states -mustexist $sloc] eq ""} { table set -subtable states $sloc 1 indefinite indefinite } HTTP::respond 200 Content "Added" } } } There we have it, an appropriately trimmed down and sleek application to provide worldwide or regional views of heatmaps showing traffic to your application, all generated 100% via iRules. Again, this couldn’t be done without the awesome geolocation abilities of LTM or the Google charting API or, of course, iRules. In the next installment we’ll dig even deeper to see how to turn this application into something even more valuable to those interested in what the users of your site or app are up to.
You Say Tomato, I Say Network Service Bus
It’s interesting to watch the evolution of IT over time. I have repeatedly been told “you people, we were doing that with X, back before you had a name for it!” And likely, the speaker is telling the truth, as far as it goes. Seriously, while the mechanisms may be different, putting a ton of commodity servers behind a load balancer and tweaking for performance looks an awful lot like having LPARs that can shrink and grow. You put “dynamic cloud” into the conversation and the similarities become more pronounced. The biggest difference is how much you’re paying for hardware and licensing. Back in the day, Enterprise Service Busses (ESB) were all the rage, able to handle communications between a variety of application sources and route things to the correct destination in the correct format, even providing guaranteed delivery if you needed it for transactional services. I trained in several of these tools, most notably IBM MQSeries (now called IBM WebSphere MQ, surprised?) and MS MQ. I was briefed on a ton more during my time at Network Computing. In the end, they’re simply message delivery and routing mechanisms that can translate along the way. Oh sure, with MQSeries Integrator you could include all sorts of other things like security callouts and such, but core functionality was restricted to message flow and delivery. While ESBs are still used today in highly mixed environments or highly complex application infrastructures, they’re not deployed broadly in IT, largely because XML significantly reduced the need for the translation aspect, which was a primary use of them in the enterprise. Today, technology is leading us to a parallel development that will likely turn out much more generically useful than ESBs. Since others have referred to it as several things, but the Network Service Bus is the closest I’ve seen in terms of accuracy, I’ll run with that term. This is routing, translation, and delivery across the network from consumer to the correct service. The service is running on a server somewhere, but that’s increasingly less relevant to the consumer application, merely that their request gets serviced is sufficient. Serviced in a timely and efficient manner is big too. Translated while servicing is seeing a temporary (though not short, in my estimation) bump while IPv4 is slowly supplanted by IPv6, but has other uses – like encrypted to unencrypted, for example. The network of the future will use a few key Strategic Points of Control – like the one between consumers and web servers – to handle routing to a service that is (a) active, (b) responsive, and (c) appropriate to the request. In the interim, while passing the request along, the Strategic point of control will translate the incoming request into a format that the service expects, and if necessary will validate the user in the context of the service being requested and the username/platform/location the request is coming from. This offloads a lot from your apps and your servers. Encryption can be offloaded to the strategic point of control, freeing up a lot of CPU time, and running unencrypted within your LAN, while maintaining encryption on the public Internet. IPv6 packets can be translated to IPv4 on the way in and back to IPv6 on the way out, so you don’t have to switch everything in your datacenter over to IPv6 at once, security checks can occur before the connection is allowed inside your LAN, and scalability gets a major upgrade because you now have a device in place that will route traffic according to the current back-end configuration. Adding and removing servers, upgrading apps, all benefit from the strategic point of control that allows you to maintain a given public IP while changing the machines that service requests as-needed. And then we factor in cloud computing. If all of this functionality – or at least a significant chunk of it – was available in the cloud, regardless of cloud vendor, then you could ship overflow traffic to the cloud. There are a lot of issues to deal with, like security, but they’re manageable if you can handle all of the other service requests as if the cloud servers were part of your everyday infrastructure. That’s a datacenter of the future. Let’s call it a tomato. And in the end it makes your infrastructure more adaptable while giving you a point of control that can harness to implement whatever monitoring or functionality you need. And if you have several of those points of control – one to globally load balance, one for storage, one in front of servers… Then you are offering services that are highly adaptable to fluctuations in usage. Like having a tomato, right in the palm of your hands. Completely irrelevant observation: The US Bureau of Labor Statistics (BLS) mentioned today that IT unemployment is at 3.3%. Now you have a bright spot in our economic doldrums.JSON Activity Streams and the Other Consumerization of IT
The JSON Activity Stream specification could allow the (other and oh so soon forgotten side of) consumerization of IT to make its way into the data center. Remember when I posited that the Next-Generation Management of Data Centers Should be Modeled on Social Networking and introduced the concept of “Infrabook” – a somewhat silly-but-serious-at-the-time idea that infrastructure should get “social”? The recent publication of JSON Activity Streams – in addition to being very exciting from an infrastructure architecture perspective – may be exactly what is needed to bring this concept to life. Seriously. Infrastructure already knows how to “speak” a variety of management languages such as SNMP and even XML, so why not adopt a simple HTTP + JSON approach to share real-time updates and notifications in the data center regarding the operational status of the infrastructure as well as the applications its designed to deliver? JSON ACTIVITY STREAMS at a GLANCE For those not familiar with Activity Streams (or JSON, for that matter) let’s take a quick look at it through a fresh lens. JSON – Javascript Object Notation – is an unstructured data format that is (more and more) commonly used to exchange data between applications using REST APIs as well as between the client (typically a browser) and an application. It’s actually a lot like XML, minus all the really hairy nesting and schematic constraints imposed on XML. While at first used primarily to enable real-time updating of clients a la AJAX, it is more and more frequently being used on the server side of architectures and thus as a means of integration, as well. It’s fairly simple to parse and manipulate and unlike its XML predecessor is far more human-readable. JSON primarily uses a name-value mechanism for serializing data and any old-skool object-oriented programmer will see similarities in its serialization with other, past and present object-oriented serialization techniques. A simple example of a JSON message might be:
