Search
Joe Pruitt - A Software Architect's take on Network Security
You are here: DevCentral > Weblogs

Friday, October 21, 2011 #


iRuleEditorIconFixes have been piling and I finally got my act together and released the latest version of the iRule Editor with support for BIG-IP version 11.0.  Here are a few things that are mentioned in the

Updated to the v11.0.0.1 iControl.dll

While this won’t be something you really “see” as a UI enhancement, it was needed as a basis for some of the upcoming features in the next release.  Be on the lookout for folders and session management in the near future.

Support for v10+ format name/value pair data groups.

iRuleEditorGroupNameValue Prior to BIG-IP version 10, we limited data groups to a single value (either string, address, or value).  In version 10, we expanded data groups to be in the form of a name/value pair.  This simplifies being able to do pattern matching with names to the desired match value with the values. 

Now, for systems that support the values, there is a second edit box for the values.  I used the short form of “name := value” in the listbox to match the way it’s represented in the product GUI.

I should have got this in earlier, but better late than never huh?

Support for Manager Role administration.

We’ve had several reports of unhandled exceptions in the Editor when trying to connect as a user with a Manager role. These exceptions should now be handled. The main piece I see missing for the Manager role is the ability to Load the configs (bigip.conf and bigip_base.conf) and the ability to issue the ConfigSync.save_configuration() method. If you are using the Editor with a Manager level role, make sure you know that any changes you make are only made in-memory and not persisted to disk. So, in other words, they won’t persist across reboots or restarts of the services on the BIG-IP.

Support for iRule description metadata.

iRuleEditorDescriptions In v11, there is now support for adding metadata to virtually any system object.   Bet you didn’t know that!  Well, while it’s not exposed in the GUI yet, it is in iControl! 

For systems that support it, you will now see a default hover tooltip for your iRules that says the following: 

No Description.  Why don’t you set one in the properties?

Well, when you do that, what you type in the description field will now show up in the hover tooltip!

Export/Import support for v11 folder formats.

For version 11 devices, objects now come back with their folder paths prefixed to the object names. This causes some havoc with the Export/Import features as the separators equate to directory paths and the default import from the archive directory no longer worked. I’ve fixed this now. If you want to know how, either take a look at the source distribution for the iRule Editor or export your objects from a v11 system and take a look at the Archive directory.

Fixed applying iRules with like priorities.

This one has peeved me for a long time as well but I never got around to fixing it either. If you try to apply two iRules containing the same event to the same virtual, you’ll likely get a “can’t apply second iRule with the same priority” error (or something like that). I honestly don’t remember what it is because I’ve fixed the issue!

Here is the logic now on how iRules are applied and default priorities assigned.

  • Scan the iRule for a “priority xxx” command and infer the priority from that.
  • If no “priority xxx” command is included default to 500.
  • If an error occurs during the assignment of the iRule, then increment the priority by 1 and try again. This repeats until a valid priority is found, or the max of 1000 is reached.

Let me know how this works for you all…

Data Groups dialog now supports resizing.

iRuleEditorGroupsDialog  For a long time, folks have said that for long data group names, the listboxes were hard to use as you couldn’t see the whole group name.  I added horizontal scroll bars a while back but with the advent of folders in version 11, all objects are now prefixed with their folder paths (unless it’s disabled - which I may add in the future). 

Now data group names are way way long, so I did some tweaking and made the whole dialog box resizable.

You can now resize the window as wide as you wish and view those really really long data group names.

Support for "[All]" partition for admins.

In preparation for support of folders in the next release, I did some working fixing up the partition support.  The GUI has a special folder called “[All]” which isn’t exposed in the Management.Partition.get_partition_list() method but is supported through the APIs to set the partition.  So, if you are an administrator on the device, you will be able to enter into forbidden territory in the magical “[All]” partition.

Partition auto-selection based on permissions.

Not sure why I didn’t do this in the past, but it made sense.  Now when you login, the iRule Editor will auto-set the active partition to the one you have access to.  Kind of a “Duh” feature, but it’s in there now.

Fixed authoriziation on partition selection.

Lastly, in the previous version, the Editor allowed you to select any partition that exists on the system and shows you the iRules for them.  The problem is that when you try to save an iRule, you would get an auth error.

This is now changed so that, while you can still see all the partition names in the File menu, you can only select ones you have authority to manage.

 

That’s about it for now.  I plan to take a more active effort on new features in the coming year so please let me know what you do and don’t like about the iRule Editor and I’ll see what I can do to get rid of those “don’t likes”.

Cheers!

-Joe

Related Links on DevCentral

Monday, October 18, 2010 #


joes-totwSometimes it takes a disaster to realize the importance of backing up your digital information.  I’ve had the same computer running at home for nearly 6 years.  Like many others out there, I store everything on it: family pictures, videos, music, financial records, tax returns, …  The list goes on and on.  I went for years without any worry at all about the implications of losing all that data.

Fortunately for me, I didn’t.  Well, not that data anyway…  I showed up for work one Monday morning a few years back and found my development computer was turned off.  This was odd as I always kept it on over the weekend so I connect remotely to it if I needed to get some work done.  I booted it up and hit the BIOS setting stating that it was missing a hard drive.  After some digging around, it turns out the hard drive went south hard.  I went out and bought a new drive and within a few hours I was back up and running.  The only problem was that all the development work I’d been doing was on that old hard drive.  That along with the nearly 100’s of presentations I’ve done over the years.  At one point, I had copied some things over to a portable hard drive but I had never done a complete system backup.  I tried several ways to restore some of the data off the drive and it turns out the controller on the drive went bad so it couldn’t be accessed at all.  It wasn’t the end of the world but I did lose a week or so of work that I couldn’t get back.

Now back to home.  If my high end development system could go belly up with no warning, then my home computer sure could do the same thing.  I had been toying with buying a backup system for a while but that hard drive crash at work made the decision for me.  I ended up buying a Windows Home Server which has been working wonderfully since the day I bought it.  I haven’t had the need to use a restore from it yet.  Actually, I could care less whether I had to ever use it.  Much like the generator I bought for my house that has been gathering dust in the garage for the last 3 years, the fact that it’s there and ready if I ever need it is good enough for me. 

Oh, and drive failure isn’t the only thing you need to worry about.  What about theft?  What about a fire?  I know a friend who lost 10 years of his family pictures after a small fire took out the computer in his den.  That’s 10 years of his kids lives that he can’t get back.  How do you put a price on that?

databackup B is for Backing Up Your Data

There are many ways you can go about protecting yourself from a data disaster in many different pricing levels.  It can be as simple as creating Data CD/DVDs with the contents of your drives, using a commodity external hard drive, investing with a dedicated backup system, or even going with an online service that will back things up remotely.

The main point of caution here is that no matter what solution you go with, include in your plan a way to store your data in a different physical location than your computer.  Having dozens of backups sitting right next to your computer won’t help much when there’s a fire…

External Hard Drives

External hard drives can be purchased at nearly every store that sells any sort of electronics.  They fall in the $50-$200 price range depending on the brand and storage capacity.  At the time of this post, you can get a decent 500GB drive for well under $100.

Most of these types of drives have a USB interface.  That means that using it is as simple as plugging it into your computer and allowing your computer time to recognize it as a hard drive.  You then use file system tools (or built-in backup software) to copy the contents of your files to the backup disk.

If you go this route, I would recommend purchasing two drives, one to keep up to date at home and one to store away from home.   By making it a plan to swap the two every couple of weeks means you will likely be covered for any disaster you may have.

Dedicated Backup System

The downside with using external hard drives is complacency.  It’s easy to put off backing things up if you have to spend any amount of dedicated time to do so.  Next thing you know it, it’ll be 6-months since your last backup.  That’s where a dedicated backup system will help folks like me.  There are many of these on the market.  For Windows users, the various Windows Home Server appliances are great choices.  Mac users can use Apple’s Time Capsule product.  These devices sit in the background and rely on software running on your computer to backup the data at regular intervals.  The price here is for devices can fall in the $300-$700 range.

Internet-Based Backup Service

If you are too lazy to manually back things up and don’t want to pay all the money up front on a higher-price device, then an internet based online backup system, may be just the ticket for you.  This solution has all the benefits of the two previous options.  The main difference is that you will likely get tied into a monthly service contract.  Do a search for online backup and you’ll see the likes of Mozy, Carbonite, Dropbox, and many others.  Here’s a link to an review of various online data backup services to give you an idea of the features they provide and what it will cost you.  The cost really depends on the amount of data you will be storing and in some cases, how often you access it. 

My main word of caution on using an online based service so to make sure you go with a reputable vendor.  This is your sensitive information so make sure that their storage is encrypted and redundantly located.

Backup Now!

No matter what option you go with, get started on a backup plan as soon as you can.  The minimal cost up front will save you many times over if anything unexpected happens.

Related Articles on DevCentral
Technorati Tags: Data, backup, drive, Joe Pruitt, ToTW ABCs, ABCs

Monday, October 11, 2010 #


joes-totwFor the last few months I’ve been negligent on my regular blogging.  I can come up with a bunch of excuses but it all boils down to feeling that I’ve got something meaningful to say.  I’m not the type to just blog for the sake of blogging but I’ve begun to miss the great feedback I receive when I put something out there that helps folks. 

What has worked for me in the past is to come up with the concept of an ABC’s series where I put sequential posts out with each post having a topic based on the letters in the alphabet.  I started with The Networking ABC’s, followed that up with Social Media ABC’s and most recently finished up the PowerShell ABC’s.  I’ve been trying to figure out a new series that could be interesting to folks and I think I’ve found my next one.

I tend to tinker around with lots of tools and utilities with regards to development and website and server maintenance.  Most recently I’ve been working on some performance testing and tuning for our website when I came across the automation features of HttpWatch.  I thought to myself: A is for Automating!  That’ll be a start of a fun series.

So here we are, the first post in a set of 26 and since these will be around tools and tips, I’ve decided to name this series the Tips Of The Week ABC’s.  I’m not sure I can push out one per day like I’ve done in the past, so I’m going to shoot for a more manageable one per week.  I hope you all find this useful.

A is for Automating HTTPWatch

HttpWatch

HttpWatch is an integrated HTTP sniffer for Internet Explorer and FireFox hat provides new insights into how your website loads and performs.  HttpWatch can be used for a wide variety of tasks including:

  • Testing a web application to ensure that it is correctly issuing or setting headers that control page expiration
  • Finding out how other sites work and how they implement certain features
  • Checking the information that the browser is supplying when you visit a site
  • Verifying that a secure web site is not issuing sensitive data in cookies or headers
  • Tuning the performance of a web site by measuring download times, caching or the number of network round trips
  • Learning about how HTTP works (useful for programming and web design classes)
  • Alowing webmasters to fine tune the caching of images and other content
  • Performing regression testing on production servers to verify performance and correct behavior

What some don’t know is that HttpWatch includes an automation interface to allow for a user to build automation scripts around HttpWatch’s functionality.  There are a set of Windows COM Objects that you can program to with your tools of choice.  The included documentation for HttpWatch includes examples of using their COM interfaces with C#, JavaScript, Ruby, and VBScript.  In this post, I’ll cover using using PowerShell to do the autmation.

HttpWatch.Controller

The core object is the HttpWatch.Controller.  It contains the following members

  • FireFox –Returns a reference to the FireFox object.  Use this property if you want to be using FireFox in your testing.
  • IE – Returns a reference to the Internet Explorere object.  Use this property if you want to use IE for your testing.
  • IsBasicEdition – Returns true if the product is the basic edition, false for Pro.
  • OpenLog(logFileName) – This allows existing HttpWatch log files to be opened and examined using the automation library.
  • Wait(plugIn, timeOutSecs) – Wait for a page to be fully loaded and is normally used after the GotoURL method.
  • Version – A string containing the current version of HttpWatch.

Creating this object in PowerShell is as easy as calling the New-Object cmdlet.

PS> $HttpWatch = New-Object -ComObject "HttpWatch.Controller"

HttpWatch.Plugin

The next step in your testing will be picking the browser you will be using.  In this example, I’m going to use FireFox because Internet Explorer is setup to run in protected mode where it won’t let me save my logs to the disk.  I’m sure a work-around can be made but in the mean time I’ll go with FireFox for this post.  The IE and FireFox members have a method called “New” that you can use to create a new instance of the browser and HttpWatch running with it’s context.

There are too many methods in the HttpWatch.Plugin class to document here, but I’ll show you the ones I found most useful.  The following example will take the previously created HttpWatch object and instantiate a new FireFox plugin.  From there it sill start recording with the “Record” method, load the Google website, wait for the page to load with the “HttpWatch.Wait” method, “Stop” the recording, Write some of the log details to the console with the “Log.Entries” property, “Save” the HttpWatch log file and also export the log contents to a Comma-Separated-Value file with the “ExportCSV”, and finally close the browser instance with the “CloseBrowser” method.

PS> $plugin = $HttpWatch.FireFox.New()
PS> $plugin.Record()
PS> $plugin.GotoUrl("http://www.google.com")
PS> $HttpWatch.Wait($plugin, -1)
PS> $plugin.Stop()
PS> foreach($entry in $plugin.Log.Entries) {
Write-Host "$($entry.Time) : $($entry.StatusCode) : $($entry.Url)" }
PS> $plugin.Log.Save("c:\temp\mytrace.hwl")
PS> $plugin.Log.ExportCSV("c:\temp\mytrace.hwl")
PS> $plugin.CloseBrowser()

What used to be a manual process for us for testing certain performance aspects of our site has now been fully automated allowing us to deploy our testing to our helpers out in the field in a much simpler manner.  I’m sure there are many ways this can be used that I haven’t thought of, but I figured that if you are using HttpWatch at all, there is probably a way that automation will help you.

Related Articles on DevCentral

Tuesday, September 07, 2010 #


windows-live-writer-logo So, a few weeks ago, Lori asked me if I’d be up for writing a plugin for Windows Live Writer to insert DevCentral related links into her blog posts.  For those that don’t know about Windows Live Writer and post to a blog regularly, you are doing yourselves a disservice by not checking it out (that is, if you are a Windows user).

I’ve messed around with some Live Writer plugins in the past and figured this wouldn’t be too big a task.  In fact, Microsoft has made it brain-dead simple to build a plugin and hook into their UI.  Here’s the steps I took to build ours in C# with Visual Studio 2010.

Create a Class Library Project

The first step is to create the project.  In Visual Studio, go into the New Project wizard and select a Class Library project. 

You’ll then want to add a reference to the WindowsLive.Writer.Api.dll assembly that can be found in the Windows Live Writer installation directory.

The main class that is created will be the one that Live Writer hooks into.  You can tell Live Writer that this is the one you want to use by adding the following attributes before the class declaration

   1: [WriterPluginAttribute 
   2:     ("XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX",
   3:     "DevCentral Related Links Plugin",
   4:     ImagePath = "DevCentralWLWPlugin.jpg",
   5:     HasEditableOptions = false,
   6:     PublisherUrl = "http://devcentral.f5.com",
   7:     Description = "Insert Related Links from DevCentral.f5.com into your blog post.")]
   8: [InsertableContentSourceAttribute("DevCentral Links")]
   9: public class DevCentralWLWPlugin : ContentSource

You’ll want to add your own GUID in for the first attribute and replace the above descriptions with the ones specific to your project.  You’ll also want to make sure your class is derived from the ContentSource defined in the WindowsLive.Writer.Api namespace.

Create A Popup Form

DCWLWPluginForm You’ll likely want your plugin to do something and the easiest way to get input from the users is by building a Windows Form.  In my code, I created new Windows Form object and named it DCSearchLookupForm and added the appropriate search and filter fields to allow the user to query for search results and select the ones they want included in their post.  In my Form, I had the resulting links stored in a member variable named “outputHTML”.

The form consists of the following sections

  • Search Terms – Here you can enter different combinations of search terms.  Instead of making everyone figure out the special ways you can enter logical expressions with Google Search, the user just enters the fields and the search phrase will be build for you.  In case you want to see what the resulting phrase is, double click on the “All These Words” label and a dialog will popup with the generated phrase.
  • Search Engine – Right now this is only DevCentral but you can change this in a custom build by changing the Search Id (CSE_ID) value.
  • Refinement – Google Custom Search allows you to build refinements (or subsets) of the index of your search engine.  This drop down will be propulated by the refinements sent back from a simple search query when the dialog is loaded.
  • Number of Links – Google limits requests to 20 items but I’ve added the ability to go up to 100 by iterating through numerous requests.  So, if you can’t find the results you want in the first 10 items that are the default, you can bump this up to 100.
  • Related Links – This is the output of the most recent search.  All of the items are selected by default.  You can deselect the ones you do not wish included in your list.
  • Formatting Template – Instead of hard coding a output format, I opted to allow the user to customize it with a head, item, and footer template.  The head and footer are written at the beginning and end and the item template is printed once per search result item.  The substitution parameters are listed in the dialog box.  There are a few other hidden ones that you can use, but you will have to dig into the source to find them B-).
  • Reload Default Template – If you mucked up the template and want to start over, the “Reload Default Template” link will repopulate the fields with the default values.

Integrating With Live Writer

Once you have the form created and tested out, you’ll integrate it into Live Writer by implementing the CreateContent method.  This is the method that is called by Windows Live Writer when the user clicks on the link created by your plugins InsertableContentSourceAttribute value above.  In this method, I just create an instance of the form, let the user do his thing and if the OK button is pressed, insert the formatted search results into the newContent variable.

   1: public override DialogResult CreateContent(IWin32Window dialogOwner, ref string newContent)
   2: {
   3:     try
   4:     {
   5:         DCSearchLookupForm f = new DCSearchLookupForm();
   6:         DialogResult dr = f.ShowDialog();
   7:         if (dr == DialogResult.OK)
   8:         {
   9:             newContent = f.outputHTML;
  10:         }
  11:     }
  12:     catch (Exception ex)
  13:     {
  14:         MessageBox.Show(ex.Message.ToString(), "Error", 
  15:             MessageBoxButtons.OK, MessageBoxIcon.Exclamation);
  16:     }
  17:     return DialogResult.OK;
  18: }
  19:  

Building A Test Application

DCWLWTestApp I found it helpful when debugging to build a test application that had a button and WebBrowser control to create the form and display the returned values.  This took two minutes and saved me from having to debug Windows Live Writer each time through.  I included this in my project as an executable called WLWPluginTestApp and set that as the Startup Project in my solution so that I could debug right into it when making changes in the plugin.

Creating A Setup Program

Installing a Windows Live Writer plugin is as simple as copying it to the plugins directory under the Windows Live Writer installation directory but that may not be the easiest way to deliver your plugin to your users.  It’s very simple to create a setup project that copies the output from the plugin project into a specified directory.  Just create a simple setup program and add the “Primary Output”, “Content Files”, "and “Localized Resources” from the plugin project and change the “Application Folder” property to the Windows Live Writer plugin directory.  I used “[ProgramFilesFolder]Windows Live\Writer\Plugins” in my setup.

Project Downloads

So, if you are yearning to include some DevCentral related content in your blog posts, then go ahead and download the installation package and give it a go.  I’ve also made the source code available if you want to see how the plugin was made or if you want to build one of your own.  The source should give you everything you need to custom make a related links module for your own Google Custom Search Engine.

Enjoy!

Related Articles on DevCentral

-Joe

Wednesday, July 21, 2010 #


I recently received an internal iRule email and one of our folks created a search provider for FireFox to search DevCentral.  Lori quickly responded and asked if we could get this posted to DevCentral.  Why not if it will help the community so I took a look.  Then it occurred to me that a while back I created a search provider definition based on the OpenSearch specification.  For some reason, on our last site refresh, the links in our website were removed so the browser didn’t natively pick them up.   I fixed that so now you can add DevCentral as a native search target in your browser of choice.

Here’s a little background on OpenSearch, how I implemented it on DevCentral, and how to set it up in your browser.

OpenSearch

OpenSearch is a format that can be used to describe a search engine so that it can be accessed and used by search client applications such as web browsers.  It’s basically just an XML file that you put on your webserver and by adding a hidden tag in your application pages, a browser is able to automatically access the search pages on your site.

Creating the Search Engine

The first step is to create the search engine definition file.  In this case, I called that file OpenSearch.xml.  The format for that file is defined here.  For DevCentral, you can view the definition directly at OpenSearch.xml.

   1: <OpenSearchDescription>
   2:   <ShortName>DevCentral Search</ShortName>
   3:   <Description>Search F5's DevCentral Developer Community</Description>
   4:   <Url type="text/html" method="get" 
   5:     template="http://devcentral.f5.com/Default.aspx?..q={searchTerms}"/>
   6:   <Image width="16" height="16">http://devcentral.f5.com/favicon.ico</Image>
   7:   <InputEncoding>UTF-8</InputEncoding>
   8:   <SearchForm>http://devcentral.f5.com/</SearchForm>
   9: </OpenSearchDescription>

It’s pretty self explanatory.  The only thing you need to make sure you include in is the {searchTerm} template value into your search url.

Exposing it to Browsers

Once you have the definition setup, you need to broadcast it to the browsers accessing your site.  This can be done with a hidden link tag in the header of your HTML pages.  By adding a “link” element of type “application/opensearchdescription+xml”, the browser will look at the “href” attribute for the location of the search provider.

   1: <head>
   2:   ..
   3:   <link rel="search" type="application/opensearchdescription+xml" 
   4:     title="DevCentral Search" href="http://devcentral.f5.com/opensearch.xml" />
   5: </head>

That’s it.  Now site search is ready to be configured in the browsers.  For each of the following steps, click on the DevCentral homepage before starting.

Installing in FireFox

OpenSearchFireFox1

Click on the drop down on the left side of the search control in the browsers address bar.  You will see a Add “DevCentral Search” menu item. 

OpenSearchFireFox2 Select this item and you will now have “DevCentral Search” as your default search provider.  This can be changed back by selecting another drop down from the menu.

Now you should see the F5 logo in your search control and by typing a search term in it, it will take you to the DevCentral search results for that query. 

At this point the search provider is available to you on any website you visit, not just when you are visiting DevCentral.

Installing in Internet Explorer

OpenSearchIE1a

Installation for Internet Explorer is almost as easy as FireFox.  The search control dropdown on the top right of the browser will have a “Add Search Providers” menu item. 

OpenSearchIE2OpenSearchIE3 Select that and you will then see “DevCentral Search” as an option.  By selecting that menu item, you will be presented with a “Add Search Provider” dialog box.  Click the “Add” button.

At this point, you now have Internet Explorer configured to browse DevCentral anytime you please.  If you didn’t make it your default search provider in the Add dialog box, you’ll have to select it from the menu.

Now by typing a search term, you will be taking to the DevCentral search just like you were with the previous Firefox installation.

Installing in Google Chrome

OpenSearchChrome1

Google Chrome behaves a bit different as it doesn’t have a dedicated search control, but relies on the address box to do both navigation and search.  The search provider settings can be found by right clicking in the address control and you will find an “Edit search engines…” menu item. 

OpenSearchChrome2OpenSearchChrome3 By selecting that menu item, you will be presented with a dialog box containing the currently configured default engines, and others that Google thinks you might like. 

Look for “DevCentral Search” at the bottom of the list.  Highlight it and click the “Make Default” button on the right side of the dialog box.

Now, just enter your search term in the address control and Chrome will take you to the DevCentral search results page.

Enjoy!

-Joe

Wednesday, December 23, 2009 #


iis7-highlight For those who of you that are having problems with logging client addresses in their server logs because you are running your web servers behind a proxy of some sort, never fear, your solution is here.  For those that don't, I already discussed in my previous posts about what the X-Forwarded-For header is so feel free to click back into those to read about it.

History

Back in September, 2005 I wrote and posted a 32-bit ISAPI filter that extracted the X-Forwarded-For header value and replaced the c-ip value (client ip) that is stored in the server logs.  Lots of folks have found this useful over time and I was eventually asked for a 64-bit version which I posted about in August, 2009

The Question

question_mark_3d Well, it looks like it's time for the next generation for this filter…  I received an email from a colleague here at F5 telling me that his customer didn't want to deploy any more ISAPI filters in their IIS7 infrastructure.  IIS7 introduced the concept of IIS Modules that are more integrated into the whole pipeline and was told that Microsoft is recommending folks move in that direction.  I was asked if I had plans to port my ISAPI filter into a HTTP Module. 

The Answer

Well, the answer was "probably not", but now it's changed to a "yes"!

The Solution

In reading about IIS Module, I found that you can develop in managed (C#/VB) or Native (C++) code.  I loaded up the test C# project to see if I could get it working.  In a matter of minutes I had a working module that intercepted the event when logging occurs.  The only problem was that from managed code, I could find no way to actually modify the values that were passed to the logging processor.  This was a problem so I scrapped that and moved to a native C++ module.  After a little while of jumping through the documentation, I found the things I needed and pretty soon I had a working HTTP module that implemented the same functionality as the ISAPI filter.

Download

The new Http Module hasn't had much testing done so please test it out before you roll it out into production.  I've made the source available as well if you find an issue and care to fix it.  Just make sure you pass back the fixes to me B-).

The filter will require installation into IIS in order for you to be able to add it to your applications.  Both distributions include a readme.txt file with an example installation procedure.  Make sure you use the Release builds for the appropriate platform (x86 or x64) unless you are in need of some troubleshooting as the Debug build will dump a lot of data to a log file.  The module supports customizable headers if you are using something other than X-Forwarded-For.  Instructions for using that are in the readme.txt file as well.

If you have any issues with using this, please let me know on this blog.  Keep in mind that this is an unsupported product, but I'll do my best to fix any issues that come up.  I'm even open to enhancements if you can think of any.

Enjoy!

-Joe

Monday, October 05, 2009 #


PoshFourSquare_thumb[2]Those who know me, can attest that I'm a early adopter when it comes to social networking systems.  The latest one I've been getting into lately is FourSquare.com

FourSquare.com describes itself as 50% friend finder, 30% social city guide, and 20% nightlife game.  They include ways to keep up with what your friends are doing as well as offering up some fun challenges to have you explore cities in different ways.

Once I've played around with a social networking site for a bit, my next step is to see what kind of API's they have exposed and see if I can build my own app to integrate with the site.  It turns out that the FourSquare.com has exposed a public API in their FourSquare-API Google Group.

So, as I did with my Posh-Tweet, Posh-Bing, Posh-TwitPic, and Posh-Delicious libraries, I went ahead and wrote a PowerShell library of functions to access the FourSquare APIs.

The API is broken down into these categories:

Credentials

  • Get-FourSquare.Credentials - Get the credentials for your FourSquare.com account.
  • Set-FourSquare.Credentials - Set the credentials for your FourSquare.com account.

GEO Methods

  • Get-FourSquare.Citites - Returns a list of currently active cities.
  • Check-FourSquare.City - When given a lat/long, returns the closest foursquare city.
  • Switch-FourSquare.City - When given a valid foursquare cityid, changes the user's default city.
  • Get-FourSquare.Checkins - Returns a list of recent checkins from friends.
  • New-FourSquare.Checkin - Allows you to check-in to a place.
  • Get-FourSquare.History - Returns a history of checkins for the authenticated user (across all cities).

User Methods

  • Get-FourSquare.User - Returns profile information (badges, etc) for a given user.
  • Get-FourSquare.Friends - Returns a list of the authenticated user's friends.
  • Get-FourSquare.Venues - Returns a list of venues near the area specified or that match the search term.
  • Get-FourSquare.Venue - Returns venue data, including mayorship, tips/to-dos and tags
  • Add-FourSquare.Venue - Allows you to add a new venue.

Tip Methods

  • Get-FourSquare.Tips - Returns a list of tips near the area specified.
  • Add-FourSquare.Tip - Allows you to add a new tip or to-do at a venue.

Other Methods

  • Get-FourSquare.Status - Returns the string "ok".

Usage:

To use the API's you must first register on FourSquare.com.  The username and password you use to create the account will be used in this API with the Set-FourSquare.Credentials method.  After setting your credentials, you can call any of the other methods and start updating your status from the comfort of your PowerShell prompt.  The following example will register the functions in the current runspace, set the credentials, query the available citites, and then checkin at the Cafe next door for lunch.

PS C:\> . .\Posh-FourSquare.ps1
PS C:\> Set-FourSquare.Credentials -user myuser -pass mypass
PS C:\> Get-FourSquare.Cities | Format-Table id timezone name geolat geolong -- -------- ---- ------ ------- 56 Europe/Amsterdam Amsterdam 52.3789 4.90067 46 America/New_York Atlanta 33.7525 -84.3888 42 America/Chicago Austin 30.2669 -97.7428 58 America/Denver Black Rock City 40.7693 -119.22 24 America/New_York Boston 42.3583 -71.0603 32 America/Chicago Chicago 41.8858 -87.6181 43 America/Chicago Dallas / Fort Worth 32.7887 -96.7676 25 America/Denver Denver 39.734 -105.026 47 America/New_York Detroit 42.3333 -83.0484 48 America/Chicago Houston 29.7594 -95.3594 49 America/Los_Angeles Las Vegas 36.1721 -115.122 34 America/Los_Angeles Los Angeles 34.0443 -118.251 39 America/New_York Miami 25.7323 -80.2436 51 America/Chicago Minneapolis / St. Paul 44.9609 -93.2642 22 America/New_York New York City 40.7255 -73.9983 33 America/New_York Philadelphia 39.8694 -75.2731 53 America/Phoenix Phoenix 33.4483 -112.073 37 America/Los_Angeles Portland 45.527 -122.685 38 America/Los_Angeles San Diego 32.7153 -117.156 23 America/Los_Angeles San Francisco 37.7587 -122.433 41 America/Los_Angeles Seattle 47.6036 -122.326 59 America/Vancouver Vancouver 49.259515 -123.103867 31 America/New_York Washington, DC 38.8964 -77.0447 PS C:\> New-FourSquare.Checkin -venue "Urban Cafe" -shout "It's lunch time!" message : OK! We've got you @ Urban Cafe. This is your 4th checkin here! id : 1398385 created : Mon, 05 Oct 09 12:50:00 +0000 venue : venue mayor : mayor

Get The Source

The full source for the script is available from my blog at Posh-FourSquare.ps1

Enjoy!

-Joe

Friday, September 25, 2009 #


image The folks here on the DevCentral team have been producing a weekly podcast for a while now.  Trying to keep our budget low, we opted to copy what some other podcasters were doing by making use of Skype for our audio communication and found a great little Skype add-on called Pamela which creates high-quality WAV files from Skype conversations. 

We created a dedicated Skype account on an old machine here in the office that will auto-record whenever that account is added to a conversation. We would occasionally have differences in audio levels between the callers so we incorporated the awesome The Levelator (from the Conversations Network) into the post production and we were set.

A few weeks ago we broke through our one hundredth podcast and along with that milestone we decided to expand things a bit by making the podcast more interactive.  I spent a few weeks investigating how to best accomplish this and I ultimately decided on a setup.  I figured that I'd go ahead and share it with you all so you can see how we do it on our end.  Here's the high-level diagram:

podcast-architecture

 

I experimented with a mixing board but ended up with a software only solution that anyone can implement.  Everything besides the top left three items in the image were added to enable us to record our Skype podcast as well as stream it to UStream and allow callers with TalkShoe. 

The Components

Here's a list of the following components and how we used them.

Skype - http://www.skype.com - $0

Skype, for those that don't know, is a software program that allows you to make video and voice calls across the internet.  Calls are free to other skype users and you can purchase the ability to call out to land line numbers or to get your own number so you can accept phone calls.

We have a dedicated server with Skype and Pamela running to do the recording and each person on the podcast is connected through the main Skype instance on our publishing system.

Pamela Skype Recorder - http://www.pamela.biz/en/ - $24

Pamela is an add-on program for Skype that will allow you to record your Skype conversations.  It features Skype call, video, and chat recording, answering machine, video mail, and skype based publishing.  We use it primarily for the ability to automatically record skype calls to either our skype account or our Skype dial-in number.  .WAV files are created and we are emailed when a recording is completed.

Windows Media Player - $0

Windows Media Player is the default music/video player on the Windows platforms.  We use it to play background music into our skype and UStream sessions.  You could just as easily use another audio player as long as it supports the ability to customize which output audio device it uses.

Virtual Audio Cable - http://software.muzychenko.net/eng/vac.html - $30

This little gem really saved the day with our setup.  There are various Mac programs that allow you to map audio streams from one program to another one but this is the best one I could find for Windows.  Virtual Audio Cable allows you to transfer audio (wave) streams between applications and/or devices.  In our setup, it allows us to map windows media player back into Skype and our multiple audio streams back into our live video processing through VidBlaster.

VidBlaster - http://vidblaster.com/ - $0

VidBlaster is a powerful, economic way to record, stream, and produce high quality videos.  VidBlaster comes in three versions: Home, Pro, and Studio with the only difference being the number of "modules" you can use at one time.  What's best about this product is that you can use it for free if you can live with a "VidBlaster" add in the top right corner of your video stream.  We'll likely put the couple hundred bucks down for the Pro version but as of now it hasn't costed us a cent.

Adobe Flash Media Encoder - http://www.adobe.com/products/flashmediaserver/flashmediaencoder/ - $0

The Adobe Flash Media Encoder allows you to capture live audio and video while streaming it in real time to a flash media server.  While we could stream directly from VidBlaster to UStream, the quality is not as good as moving some of the processing down to the client.

UStream - http://www.ustream.tv - $0

UStream is the live interactive video broadcast platform that enables anyone with a camera (or VidBlaster!) and an internet connection to quickly and easily broadcast to a global audience of unlimited size.  Best of all, it's free!

TalkShoe - http://www.talkshoe.com - $0

We struggled for a while trying to figure out the best way to include a "live" audience in our recording.  TalkShoe is a service that enables anyone to easily create, join, or listen to live interactive discussions, conversations, podcasts, and audioblogs.  It fit the bill of just what we needed and since we already had our Skype conversation going, bolting on TalkShoe was as simple as adding the conference number to our group Skype conversation.  TalkShoe also has some pretty nifty user controls allowing the moderator to control muting of each of the participants.  Another great free service!

Production Setup Walkthrough

Here's the steps I go through in preparation for our weekly podcast.

  1. Create 3 Virtual Audio Cables with the Virtual Audio Cable Control Panel (VAC #1, VAC #2, VAC #3)
  2. Start up 2 instances of the Virtual Audio Cable Audio Repeater.  Set the first one (R1) from VAC #3 to VAC #1 and the second from VAC #3 to VAC #2.
  3. Start and login to Skype.  Go into the Audio settings and change the audio input to VAC #1 and the output to VAC #2.
  4. Start Windows Media Player and change the speaker device settings to output to VAC #3.
  5. Start VidBlaster and setup various screen captures, set the video resolution to 640x480 and the frame rate to 15fps (much higher than that drives the CPU way up).  Finally click "Start" on the Streamer module.
  6. Start the Adobe Flash Media Encoder.  Load the stream configuration that you can download from your UStream show's advanced settings and then Select VidBlaster for the Video Device and "Line 2 (Virtual Audio Cable)" (ie. VAC #2) for the Audio device.  Click "Start" to begin streaming.
  7. Login to our UStream.tv account and click "Broadcast Now" for your show.  Another browser window will come up detecting the media stream.  Click "Start Broadcast" to start the stream and click "Start Record" to being recording your stream on the server.
  8. Bring up the Skype window and create a conference with the podcast members including our account that auto-records the conversations.  Then finally, call the TalkShoe conference bridge and initiate the meeting on their side.
  9. Login to the TalkShoe account on their website and click the option to join the meeting.  When the admin console is loaded, you can optionally select "Record" from the top left to have TalkShoe make an alternate recording.

At this point we are recording our Skype session in audio, UStream is recording the video stream and TalkShoe is recording a secondary audio recording.  At this point, I cue up the intro music in Windows Media Player and start into the podcast.

When we are finished, I reverse the process above by stopping and saving the various recordings.  I take our Pamela based recording, run it through The Levelator, convert it to a mp3 with Audacity, edit the ID3 tags, and publish it to our media server.  At this point, the blog post is created and we are done for the week.

Reflections

There are a few things that are still causing some issues.  The main issue is that of horsepower on my media system.  The desktop I'm using for all of this was not meant to run these types of CPU intensive applications.  While it works, my dual-core CPU system is hovering at 99-100% CPU usage during the podcast which is a bit worrisome.  Depending on the success of the podcast, we may invest in a new desktop to run this on.  But, seeing how we've spent well under $100 for the entire software suite, I don't think we'll have much trouble justifying it!

Hopefully this helps some of you out there with some ideas and I'd love to hear feedback on how I could do things better!

-Joe

Monday, September 21, 2009 #


poshdelicious

For those of you who don't know what Delicious.com is (yes it's the same as del.icio.us), I'll just borrow their overview from their getting started page:

Delicious is a social bookmarking service that allows you to tag, save, manage, and share Web pages all in one place.  With emphasis on the power of the community, Delicious greatly improves how people discover, remember and share on the Internet.

Here on DevCentral, when new content is added, we regularly add it to various social networking sites, including Delicious.  There are some great browser plugins that help you with sharing content and I regularly use the Shareaholic Firefox plug-in.  But, it's still too much manual work for me so I figured I'd try to automate it a bit.

I previously published a PowerShell library for Twitter (PoshTweet), for TwitPic (PoshTwitPic) as well as one for the Bing Search API (PoshBing) so I figured I'd build on those and write one for Delicious.  The Delicious API documentation is defined here

The API is broken down into the following functions:

Credentials

  • Set-Delicious.Credentials - set the client credentials for the delicious.com account.
  • Get-Delicious.Credentials - get the client credentials for the delicious.com account.

Posts

Tags

Tag Bundles

Usage:

The usage is fairly straight forward.  First you must load the functions into your runspace.  This is most easily done by "dot sourcing" the script.  Then you will call the Set-Delicious.Credentials function with your delicious credentials.  At this point you should be all set to submit bookmarks with PowerShell.  The following code creates a new bookmark for my blog with the Add-Delicious.Post function and then retrieves the entries from the last day you posted with Get-Delicious.Posts.

PS C:\> . .\Posh-Delicious.ps1
PS C:\> Set-Delicious.Credentials -user "joeuser" -pass "joepass"
PS C:\> Add-Delicious.Post -url "http://devcentral.f5.com/weblogs/joe" -extended "An awesome blog!" -tags "weblogs"
ResultCode
----------
done
PS C:\> Get-Delicious.Posts
href : http://devcentral.f5.com/weblogs/joe
hash : ed2760db92010aad2c6a1008afa6ce2b
description : http://devcentral.f5.com/weblogs/joe
tag : weblogs
time : 2009-09-21T15:34:06Z
extended : An awesome blog!

Get The Source

If you want to automate the posting of links to Delicious, feel free to check out the source that can be downloaded from here: Posh-Delicious.ps1

Enjoy!

-Joe

Thursday, August 27, 2009 #


This week marks a major milestone for us here at DevCentral: we have reached our 100th weekly Podcast!  To celebrate, we are going to try some new formats and the first one of these is user interaction!  Starting today, we will be streaming our podcasts live on UStream.tv so you can listen in while we are recording.  Don't worry, we will still be archiving the audio versions on DevCentral, iTunes, etc.

UStream.tv gives us a few benefits.  First, we can broadcast video along with the audio.  At this point we are unable to syndicate everyone's webcam from our team but this gives us some options for the future.  Another benefit here is that you are welcome to comment in our chat session as well if you feel the need to chime in or ask a question. 

We really want to move our podcast into more of a "virtual round table" with the DevCentral community and this is our first step.  We are always looking for guests as well so if you feel like you have something to share with the community, we'd love to have you on.  Just leave a comment on this blog and we'll get back to you to schedule you in.

We've setup a permanent link that will get you to our live show, including previous recordings).

Live_Logo
http://devcentral.f5.com/live

We record our podcast on Thursday's at 2PM PST.  I'm sure as the popularity grows on this, we'll expand our times so make sure you "follow" us on our UStream.tv channel http://www.ustream.tv/channel/dcpodcast for any upcoming recordings.

-Joe

Thursday, August 20, 2009 #


 safety-danceOk, so Pete did his best Weird Al impersonation the other day by throwing up his take-off on Men Without Hat’s “Safety Dance”.  His version “Encryption Dance” is brilliant and we decided to highlight it in this weeks podcast with a challenge out to Pete to give us a live recording of his parody.

Well, it didn’t take Pete long to answer to our call and he sent along his recording of “Encryption Dance”.   For those who aren’t familiar with Men Without Hats, here’s their Safety Dance Video for you to check out first.

And here’s Pete’s “Encryption Dance” performed A Cappella".

Get the Flash Player to see this player.

I know which one I prefer!  Thanks Pete!!!

-Joe

Wednesday, August 19, 2009 #


For those that don't know what X-Forwarded-For is, then you might as well close your browser because this post likely will mean nothing to you…

A Little Background

Now, if you are still reading this, then you likely are having issues with determining the origin client connections to your web servers.  When web requests are passed through proxies, load balancers, application delivery controllers, etc, the client no longer has a direct connection with the destination server and all traffic looks like it's coming from the last server in the chain.  In the following diagram, Proxy2 is the last hop in the chain before the request hits the destination server.  Relying on connection information alone, the server thinks that all connections come from Proxy2, not from the Client that initiated the connection.

TrafficFlow The only one in the chain here who knows who the client really is (as determined by it's client IP Address, is Proxy1.  The problem is that application owners rely on source client information for many reasons ranging from analyzing client demographics to targeting Denial of Service attacks.

That's where the X-Forwarded-For header comes in.  It is non-RFC standard HTTP request header that is used for identifying the originating IP address of a client connecting to a web server through a proxy.  The format of the header is:

X-Forwarded-For: client, proxy1, proxy, …

X-Forwarded-For header logging is supported in Apache (with mod_proxy) but Microsoft IIS does not have a direct way to support the translation of the X-Forwarded-For value into the client ip (c-ip) header value used in its webserver logging.

Back in September, 2005 I wrote an ISAPI filter that can be installed within IIS to perform this transition.  This was primarily for F5 customers but I figured that I might as well release it into the wild as others would find value out of it.

Recently folks have asked for 64 bit versions (especially with the release of Windows 2008 Server).  This gave me the opportunity to brush up on my C skills.  In addition to building targets for 64 bit windows, I went ahead and added a few new features that have been asked for.

Proxy Chain Support

The original implementation did not correctly parse the "client, proxy1, proxy2,…" format and assumed that there was a single IP address following the X-Forwarded-For header.  I've added code to tokenize the values and strip out all but the first token in the comma delimited chain for inclusion in the logs.

Header Name Override

Others have asked to be able to change the header name that the filter looked for from "X-Forwarded-For" to some customized value.  In some cases they were using the X-Forwarded-For header for another reason and wanted to use iRules to create a new header that was to be used in the logs.  I implemented this by adding a configuration file option for the filter.  The filter will look for a file named F5XForwardedFor.ini in the same directory as the filter with the following format:

[SETTINGS]
HEADER=Alternate-Header-Name

The value of "Alternate-Header-Name" can be changed to whatever header you would like to use. 

Download

I've updated the original distribution file so that folks hitting my previous blog post would get the updates.  The following zip file includes 32 and 64 bit release versions of the F5XForwardedFor.dll that you can install under IIS6 or IIS7.download_buttonInstallation

Follow these steps to install the filter.

  1. Download and unzip the F5XForwardedFor.zip distribution.
  2. Copy the F5XForwardedFor.dll file from the x86\Release or x64\Release directory (depending on your platform) into a target directory on your system.  Let's say C:\ISAPIFilters.
  3. Ensure that the containing directory and the F5XForwardedFor.dll file have read permissions by the IIS process.  It's easiest to just give full read access to everyone.
  4. Open the IIS Admin utility and navigate to the web server you would like to apply it to.
  5. For IIS6, Right click on your web server and select Properties.  Then select the "ISAPI Filters" tab.  From there click the "Add" button and enter "F5XForwardedFor" for the Name and the path to the file "c:\ISAPIFilters\F5XForwardedFor.dll" to the Executable field and click OK enough times to exit the property dialogs.  At this point the filter should be working for you.  You can go back into the property dialog to determine whether the filter is active or an error occurred.
  6. For II7, you'll want to select your website and then double click on the "ISAPI Filters" icon that shows up in the Features View.  In the Actions Pane on the right select the "Add" link and enter "F5XForwardedFor" for the name and "C:\ISAPIFilters\F5XForwardedFor.dll" for the Executable.  Click OK and you are set to go.

I'd love to hear feedback on this and if there are any other feature request, I'm wide open to suggestions. The source code is included in the download distribution so if you make any changes yourself, let me know!

Good luck and happy filtering!

-Joe

Wednesday, August 12, 2009 #


iRuleDoYouv1Come one, come all to our third edition of the iRule. Do You? Contest.  For those that don't know, an iRule is F5's embedded packet scripting language allowing you to gain full control over how your applications flow across the network.

There are hundreds of iRules in our iRule CodeShare and thousands of them running out in the wild.  It's time for you iRule geeks out there to show us, and our community, your stuff.  Whether you are an avid developer or just someone who wants to dabble in new technologies, this contest is for you.

The rules are pretty simple:

01  Read the contest rules.
02  Write a working
iRule.
03  Tell us about it by submitting it on our
Contest Entry form.
04  Goto 02 as many times as you want.
05  Sit back and wait for the judges to pick you as a winner and receive some awesome prizes.

We are judging on several criteria so don't worry if your iRule isn't the most complex.  Sometimes it's the simplest ones that provide the most value that will ultimately make it to the winners circle.  The entries will be judged on Business Merit (innovation, business value, and breadth of applicability) as well as Technical Merit (Elegance, Potential for extensibility, and Cool factor) so there are lots of ways to win.

As if the accolades of winning are not enough, we've sweetened the pot with some pretty awesome prizes, if I do say for myself.  How's a 17" MacBook Pro sound?  No, well then, how about a Canon 5D Mark II?  If you win the gold, you'll be taking one of these home with you.  There are also plenty of consolation prizes including a MacBook Air, Apple 30" Cinema display, HP TouchSmart, Amazon Kindle DX, ASUS EeePC, XBox 360, and even a bunch of iRule branded Flip Mino HD video cameras (I've got to get me one of these!). 

So, hurry up and get started submitting your iRules!

-Joe

Wednesday, August 05, 2009 #


AboutTheiRuleEditorIt’s been a while since I posted an update for the iRule Editor so I figured I’d spend some time this week polishing off some of the bugs and enhancements reported on the iRule Editor forum over the past year or so.

So, I can go on and on about the history, but I figure you all just want to bypass that and go straight to what’s been fixed/added since the last version.  We’ll here’s what I’ve got for you in the latest v0.10.0.0 distribution that I released today.

FIX:  Hyperlink click crashes iRule Editor

This was reported in this thread titled iRule Crash.  In the text editor window, the iRule Editor has some special tokens that allow you to click on them and launch a browser to take you to the appropriate documentation webpage for that token.  On some windows system, the registry settings for the default browser were not where I expected them to be.  I removed this registry lookup and relied on the system to determine for me the default browser application.  This works much nicer now!

FIX: Disable auto-configuration saving if the user is in a roll that doesn’t allow it.

Manager roles do now have access to the ConfigSync.save_configuration() iControl method to flush the running configuration to disk.  In the past, an error dialog would display on every save command.  The iRule Editor will now assume you have privileges to do so and if it encounters an error saving the configuration, it will use a new “Auto-Save” flag configurable in the “Tools.BIG-IP Config” menu item.

FIX: iRule Property Virtual Server assignment not persisted to disk.

For some reason, I omitted the call to ConfigSync.save_configuration() on the iRule Property dialog when assigning virtual servers.  Now, on an OK in the dialog, a save operation occurs.

ENH: New “Tools.BIG-IP Config” menu to control BIG-IP configuration saving.

On this menu, you can trigger a manual config save and control the auto-save option.

ENH: HTTP Proxy Server support for Connection Dialog

As implemented in the iControl Assembly, the connection dialog now supports a HTTP proxy for iControl connections to your BIG-IP’s management port.  I had a hard time testing this so please let me know if it works B-).

ENH: Application Self Updates

Since removing the ClickOnce deployment from the iRule Editor setup, there hasn’t been an easy way to check for new versions of the Editor.  With this release, there is an new program installed with the Editor called iRulerUpdater.exe that is called by the iRule Editor’s “Check For Updates” command.  You should never see this unless a new update is available on the site.  BTW, if you want to check for the latest version yourself, you can do so by querying http://devcentral.f5.com/labs/iRuleEditor/iRulerCurrentVersion.txt.

ENH: New GTM syntax tokens in the editing window.

Thanks to Jason, we’ve got an updated list of GTM iRule tokens in the lexer configuration for the Editor.

There’s also a few hidden Easter eggs in there that I didn’t want to enable for the general population.  Let me know if you find out how to enable them. B-).

So, enough said…  Head over to the iRule Editor Labs page and download the latest version and let me know what you think.

Oh, and don’t forget, the source is available as well if you want to dig in and add some features yourself.  I’m always happy to receive changes and incorporate them into the main distribution.

-Joe

Friday, June 19, 2009 #


poshShrinkShrinking your Url’s is all the rage nowadays.  If you are on Twitter, then odds are you have used one.  Despite CodingHorror’s distaste for them in his recent blog post on Url Shorteners: Destroying the Web since 2002, they are a fact of life when we live in a world of 140 character status updates.

So what’s a URL shrinking service anyway?  Well, to put it simply, you supply them with a URL, they then supply you with a shorter URL containing a lookup “key”.  When future requests are made to this shorter URL, connections are routed to that services website where they convert the short URL to the original URL and issue a HTTP Redirect back to your browser to send you off to the original long url website.

So, what’s a guy, or gal, to do if they want to set their status programmatically on Twitter, Facebook, FriendFeed, or the other gazillion social networking sites out there today and need to post a very long URL?  Most of these shrinking services offer API’s to access their “shrinking” services so it’s just a matter of digging into the various APIs to get them implemented.  In my original PowerShell Twitter library PoshTweet, I included support for a couple for shrinking URL’s through a couple of shrinking services but I figured it would be good to separate that functionality out into it’s own library. 

So, here’s my first stab at a generic URL shortening library for PowerShell.  I’ve included the following services:

Some of these services have advanced features such as statistical reporting on all your submitted links.  Those services require users to create accounts and register for API keys.  Bit.ly is one of them, so if you want to use that service, you will have to supply this script with your bit.ly username and API key with the Set-BitlyServiceInfo function.

So, here’s the script, hope you all enjoy!

 

   1: param(
   2:   [string]$longurl = $null,
   3:   [string]$provider = "tinyurl"
   4: );
   5:  
   6: $script:DIGG_APPKEY = "http://devcentral.f5.com/PoshShrink";
   7: $script:BITLY_LOGIN = "";
   8: $script:BITLY_KEY = "";
   9:  
  10: #----------------------------------------------------------------------------
  11: # function Set-BitlyServiceInfo
  12: #----------------------------------------------------------------------------
  13: function Set-BitlyServiceInfo()
  14: {
  15:   param(
  16:     [string]$login = $null,
  17:     [string]$key = $null
  18:   );
  19:   if ( $login -and $key )
  20:   {
  21:     $script:BITLY_LOGIN = $login;
  22:     $script:BITLY_KEY = $key;
  23:   }
  24:   else
  25:   {
  26:     Write-Error "Usage: Set-BitlyServiceInfo -login login -key key";
  27:   }
  28: }
  29:  
  30: #----------------------------------------------------------------------------
  31: # function Shrink-Url
  32: #----------------------------------------------------------------------------
  33: function Shrink-Url()
  34: {
  35:   param(
  36:     [string]$longurl = $null,
  37:     [string]$provider = "tinyurl"
  38:   );
  39:   
  40:   $shorturl = $null;
  41:   if ( $longurl )
  42:   {
  43:     switch ($provider.ToLower())
  44:     {
  45:       "is.gd" {
  46:         $shorturl = Execute-HTTPGetCommand "http://is.gd/api.php?longurl=$longurl";
  47:       }
  48:       "tinyurl" {
  49:         $shorturl = Execute-HTTPGetCommand "http://tinyurl.com/api-create.php?url=$longurl";
  50:       }
  51:       "snurl" {
  52:         $shorturl = Execute-HTTPGetCommand "http://snipr.com/site/snip?r=simple&link=$longurl";
  53:       }
  54:       "digg" {
  55:         [xml]$results = Execute-HTTPGetCommand `
  56:           "http://services.digg.com/url/short/create?url=${longurl}&appkey=${script:DIGG_APPKEY}";
  57:         $shorturl = $results.shorturls.shorturl.short_url;
  58:       }
  59:       "tr.im" {
  60:         [xml]$results = Execute-HTTPGetCommand "http://api.tr.im/api/trim_url.xml?url=$longurl";
  61:         $shorturl = $results.trim.url;
  62:       }
  63:       "bit.ly" {
  64:         if ( !$BITLY_LOGIN -or !$BITLY_KEY )
  65:         {
  66:           Write-Error "ERROR: You must configure your bit.ly LOGIN and APIKEY!"
  67:           exit;
  68:         }
  69:         else
  70:         {
  71:           $results = Execute-HTTPGetCommand `
  72:             "http://api.bit.ly/shorten?version=2.0.1&longUrl=$longurl&login=$script:BITLY_LOGIN&apiKey=$script:BITLY_KEY";
  73:           $shorturl = $results.Split("`n")[7].Split("""")[3];
  74:         }
  75:       }
  76:       default {
  77:         $shorturl = Execute-HTTPGetCommand "http://tinyurl.com/api-create.php?url=$longurl";
  78:       }
  79:     }
  80:   }
  81:   else
  82:   {
  83:     Write-Host "ERROR: Usage: Shrink-Url -longurl http://www.foo.com";
  84:   }
  85:   $shorturl;
  86: }
  87:  
  88: #----------------------------------------------------------------------------
  89: # function Execute-HTTPGetCommand
  90: #----------------------------------------------------------------------------
  91: function Execute-HTTPGetCommand()
  92: {
  93:   param([string] $url = $null);
  94:   
  95:   $user_agent = "PoshShrink";
  96:   
  97:   if ( $url )
  98:   {
  99:     $request = [System.Net.HttpWebRequest]::Create($url);
 100:     $request.UserAgent = $user_agent;
 101:     $request.Credentials = [System.Net.CredentialCache]::DefaultCredentials;
 102:     if ( $request.Proxy )
 103:     {
 104:       $request.Proxy.Credentials = [System.Net.CredentialCache]::DefaultNetworkCredentials;
 105:     }
 106:     $response = $request.GetResponse();
 107:     $rs = $response.GetResponseStream();
 108:     [System.IO.StreamReader]$sr = New-Object System.IO.StreamReader -argumentList $rs;
 109:     $sr.ReadToEnd();
 110:   }
 111: }
 112:  
 113: Shrink-Url -longurl $longurl -provider $provider;

 

You can download the full contents for this script here: Shrink-Url.ps1

Blog Stats

Posts:379
Comments:1067
Stories:1
Trackbacks:301
  

Article Categories

  iRules
  

Image Galleries

  

Joe's bookshelf: read

The Lost Gate
4 of 5 stars
This one started slow but I got really got into it about 1/3 of the way through. If you are an Ender's Game fan, you'll probably like this one as well.

goodreads.com


82,243 Members in 102 Countries and Growing!

Join DevCentral Today!

About DevCentral

DevCentral has been a successful, thriving community for many years. We have always strived to bring you the best technical documentation, discussion forums, blogs, media and much more that we can.

So dive in, get familiar with DevCentral. We hope you like it, we hope it makes your job easier, and lets you get that much more power out of the community. To learn more, make sure to check out the Getting Started section. And if you have any problems, or think something could be easier to use, drop us a line to let us know.

Got It !

We've received your comment and transmitted it directly to DevCentral HQ.

Thanks for taking time to let us know what's on your mind. At DevCentral | Community Matters!

Get In Touch With Us

Have questions, suggestions or just want to get something off your chest?

Use our handy form below to Direct Connect with DevCentral Mission Control.

Send Us Feedback       or