DevCentral

Making Chili and Managing Network Resources.

Don MacVittie — Tue, 15 Nov 2011 16:27:29 GMT

#f5 There’s a new brand of Chili in town.

I don’t usually talk a lot about F5 specific solutions, but since we’re the only ones doing this (so far), the contents of this blog are F5 specific. Though this needs to be industry standard.

So, you’re yearning for some chili. That’s understandable, this time of year is when those of us from the US midwest think of chili, because it’s good hunting season food, and it both fills you and warms you up.

So grab a handful of hamburger and stuff it in your mouth, then grab a handful of dried kidney beans and stuff those in there too, no, don’t worry, we’re about to get to the cayenne pepper…

No? Okay, okay, you want it to actually be mixed before it gets to your stomach. I suppose that’s understandable too. So toss a bunch of hamburger into a pot, throw in some dried kidney beans – don’t forget the water – some chili powder, some cayenne pepper, whatever other spices you like, some tomato sauce, that’ll about do it. Got all of that? Okay, so next you cook it. In all that other stuff, it’ll take a good long while for the hamburger to cook, but since we didn’t soak the beans, they’ll need a good long while anyway… What? That’s not it either?

Okay then, last try. Brown up some hamburger, drain off the grease (or Juice as one of my best friends complains at this step), pour in some canned (or pre-soaked) kidney beans, some tomato sauce, some spices, and cook it up. What? Still not detailed enough? But I told you what to put into it, weren’t you reading?

Oh heck, go to your nearest chili joint and just buy some. In Green Bay we go to Chili Johns. In Cincinnati it’s Skyline chili. But where ever, place the order and get well-made chili. I don’t have to tell you all of the steps, you don’t have to get worked up about grey areas in the directions, you get tasty chili, I can go get some too.

Wouldn’t it be nice if that’s how it worked, and you didn’t have to pay for it?

Now consider that you’re deploying your application behind an ADC.

First you configure the Virtual IP, then you create a pool to service the Virtual IP, then you add nodes to the pool… What?

I know. That’s been a problem with ADCs for a good long while. Lots of steps, all necessary, all with room for miscommunication or error. Not anymore.

I’ll borrow a picture from coworker Karen Jester’s blog to illustrate the point:

There’s more at the link to her blog above (click on her name), but the point is relatively simple. It used to be that you had to configure each of the networking/load balancing/security/app delivery/et cetera. elements of an application deployment separately. Notice in this screenshot that the questions are about the application and your deployment of it, not about nodes and pools. We have some excellent deployment guides, but they run to many pages, and since you’re copying information from a book or PDF, missing steps is possible.

With iApps, that is no longer the case. iApps take an application-centric view of network resources. In essence, they’re Skyline Chili, but you don’t have to pay for them. They come free in V.11. And they know your apps. So if you need to deploy Exchange behind a BIG-IP, open the Exchange version X template, and fill in the few questions. Next thing you know, you’re running an ADC configuration with your requirements considered. No more individual items to configure. And you can modify the configuration at a later date to adapt to changes in your environment.

Of course, if you’re an expert, you can still configure the individual elements, but if you want to utilize the power of an ADC, but don’t have time to go through each and every step in a deployment guide, now with knowledge of your application, you can get it running – secure, fast, and available – in short order. For those applications we don’t have a template for yet, you can build one, download one developed by a peer from F5 DevCentral, or configure the objects individually using one of our deployment guides.

If you don’t already, I’d recommend reading Karen’s blog. She’s wicked smart, and in a location that gives her insight into F5 gear.

And yes, I’d love to talk about how other vendors are turning app delivery into an application-focused tool, since in the end it is all about delivery of applications. But until they do, I’ll just keep telling you how cool iApps are.

Oh and did I mention they give you an astounding look into overall application performance across the network? Yes, they do that too. It’s like the cheese on top of a bowl of Skyline Chili.

Technorati Tags: Application Delivery Controllers,ADC,iApps,F5 DevCentral,Karen Jester,F5 Networks,Don MacVittie,Blog

Connect with Don:	Connect with F5:

Some Days, A Reminder is all You Need.

Don MacVittie — Fri, 27 Aug 2010 04:34:04 GMT

My eldest son has been having car troubles. To be more direct, he needed a new car. We agreed to help him out financially, let him go do his shopping and comparing, and when he chose a car I took him to pick it up. He chose a used PT Cruiser to replace his worn-out Olds Achieva, and on the way home, tried to familiarize himself with the features of the new car. Anything that the Achieva didn’t have and the PT Cruiser did, he found to be odd to him. And I pondered that as I drove him to pick up the poor old Achieva and send it to its final resting place. We all do that. We get a shiny new toy, and then play with the features we were excited about when we chose it, or with the features that are somewhat familiar. In any complex system – cars or networking gear – we tend to lose some very cool bits in the conversion.

But some of those bits – like the heads-up display in his new car – can be a boon that offers him information he didn’t have. In high-tech, there is plenty of that type of thing also. We’re all talking a ton about Cloud and the revival of SaaS in the down economy, and for me at least, WAN Optimization. These discussions leave us forgetting that there is a technological basis for these tools. But of course, these important topics are well below the radar when some of the cool stuff that comes out-of-the-box with F5 gear is being looked at (profiles for example, are high-level and very useful, so they draw admins eyes). This post is your friendly reminder that there are features on your F5 gear that could be garnering you a lot.

<F5 Mode On>

I normally like to present a problem or issue, then talk about solutions. Sometimes those solutions include F5 gear, sometimes they do not. This time I’m going to just talk about some of the less obvious features of F5 gear that are there for you – all for free with the product – that you could be taking advantage of to automate your environment or improve your Application Delivery Architecture. If you’re not an F5 customer, well this is stuff that might change your mind, and if you are, you already have them in your building (or you can download them for free), so you should check the ones you are not familiar with out.

I’m a geek. I started as a developer, learned networking, spent my time as IT management, and now am a Technical Marketing Manager. The items listed here have a relatively high “Geek Factor”, they’re not something you want to run out and tell a conference room full of business managers about… But what they can enable for your organization, what becomes easier/faster/better, that you might want to chat up the business leaders about.

Note that some of these links require a free membership on F5 DevCentral. No worries though, that membership is not culled for prospects or anything, it is a community that F5 happens to host, not a sales engine that you’re getting duped into. Seriously. Ask the MVPs. Or any other member. It does give you access (through groups) to some of the brightest individuals involved in the topics below though.

iRules

What if you had a toolset that made advanced application delivery and highly granular load balancing easy? That would allow you to redirect traffic dynamically without having to rework things all of the time? Well, that’s what iRules do. iRules are a TCL variant that allows you to make routing decisions based upon scripted rules. They allow you to do a lot more (one of the DC guys wrote a game in iRules…), but if the idea is new to you, routing decisions are where most people start. There are a few sample iRules installed on your BIG-IP when you receive it, though IIRC none of them are activated. As samples go they do not include your specific domain information or routing info for your network. That’s a big part of the reason that I wrote the iRule redirector generator a year or two ago, to give you a place to generate simple redirection iRules that you can then cut-n-paste into either the web browser or the iRule Editor written by Joe Pruitt. The iRule editor does most of what you want if you’re running Windows – syntax highlighting, code completion, all that you would expect.

The power of iRules is far too much to cover in a simple blog post, so I direct you to the iRules Wiki for samples of how other users like you have made use of iRules to make their job easier (particularly popular is the credit card scrubber). The community support behind iRules and the sharing of sample iRules amongst members of DevCentral is something that all of us here at F5 are pretty proud of. It shows just how useful iRules can be.

iControl

Where iRules leave off, the iControl API picks up. iControl is a collection of APIs that let you manage your BIG-IP gear. From taking a server off-line to activating a profile on a group of servers to improve their response times, iControl gives you the tools to manage your network at the Application Delivery Controller. The API is massive, with thousands of calls and a whole host of self-defined variables, but it is SOAP 1.0 compliant, and has an extensive documentation Wiki.

While iRules come pre-installed on your BIG-IP, the iControl API needs to be downloaded for integration into your development environment. The server is running on your BIG-IP, you just need these libraries to write code to it. The server uses the same security roles as the BIG-IP, and requires a route to an admin port, so there’s no worry that you have some server running open to the world accepting requests that are dangerous.

And no matter what language you are using, we probably have an implementation and samples for you. While most users develop in Java and .NET, there are a significant number of Perl users, and several other languages have gained in popularity over recent years, most notably python. With a wealth of community maintained samples, you can get started quickly. Note that if you are not a developer, the learning curve is a bit steeper – it is a programming API after all – but many of our most prolific iControl users are admins with only a smattering of development background.

If you are highly virtualized, the iControl interface is useful to automate one piece of your environment – putting servers into and out of pools as demand changes is a common task in iControl – and while we have other tools, the completeness of the API makes it highly useful in your virtualization management efforts.

tmsh – The New Kid

When version 10.0.1 of BIG-IP LTM was released, I was still on the DevCentral Team, and we received early briefings. When 10.1 came out, I was just leaving the team, but also received early briefings… And tmsh captivated me. It is built into your BIG-IP if you’re running version 10.0, and if you’re not, consider upgrading for a wealth of reasons, tmsh being just one of them.

Much of what is today done in iControl from a remote application can be moved into tmsh and executed as a command-line script directly on the BIG-IP. The tmsh syntax is easier to grasp than iControl calls are (for most people anyway, there’s always at least one exception…), and the power of tmsh comes from building reusable scripts to do all of your heavy-lifting for you. The Wiki has syntax formatting and a couple of samples, the Samples page has a bakers’ dozen of scripts that you can download, read through, and execute on your BIG-IP (again, assuming you have the correct credentials). As with iControl, this command interpreter can be highly useful when automation comes into play. It is still relatively new to the DevCentral Community, so there aren’t as many samples as I’d like to see, but they will continue to grow, and I expect to see a stable of often-used “essentials” as users begin to take advantage of the simplicity and power of this scripting tool.

OF COURSE THERE’S MORE…

One difference between you and my son is that he’ll be driving his car cross-country, spending dozens of hours in it a short time after he gets it. While I’m thrilled when I hear about a shop with enough ADC gear to invest that kind of time, most of you have ADCs as one of several networking and/or sysadmin responsibilities. So I thought the occasional reminder refresh was in order, since you won’t likely have 12 uninterrupted hours of quality time to spend with your BIG-IP. I expect my son will return home an expert in the PT Cruiser, this list will just give you a handy list of things you may not have considered, and some shortcuts to getting started… But it’s better than nothing, no?

Technorati Tags: Virtualization,Automation,F5 Networks,F5 BIG-IP,iRules,iControl,tmsh,PT Cruiser,Don MacVittie

Not a Blog

Don MacVittie — Fri, 18 Jun 2010 18:31:49 GMT

Looking for my blog today? It’s in hiding. I posted it over there –>> Lori’s blog. She is running a series called “F5 Friday”, and as I promised a few blogs ago, I have a write-up about Data Manager (DM), complete with screenshots in for today’s installment.

A little weird that it’s on her blog? Not really… Nothing to do with our relationship or anything, she’ll do some of these and will host others by employees, partners, customers, who knows? Mine is just the first of many to be hosted on her blog…

So read and enjoy, DM is an under-acknowledged product, and the trial is free, so go check it out.

Don.

Technorati Tags: Virtualization,Storage,F5 Networks,F5 Data Manager,Don MacVittie

Utilities and Security Rear Their Ugly Head Again

Don MacVittie — Thu, 17 Jun 2010 17:50:42 GMT

BACK IN THE DAY…

Years ago I wrote a piece for Network Computing Magazine about the state of Utility network security and the issues it presents. I focused largely on SCADA security, but also looked at Automated Meter Reading (AMR) and the new issues it brought to the table. That article was not without foundation, I built and lead the team that did the IT portion of the first total AMR replacement in the United States. The field was new, and we used about a dozen different systems to get complete coverage, making integration and security a bit of a bear. What we did we called AMR, but what is today called Smart Metering was included – we could control We had systems that read and controlled meters over the power line, over telephone land lines, over cell infrastructure, over wide area wireless networks, and over private Ethernet networks. It was a huge project, and a blast to put together. Later I took over development of one of the AMR systems that we had used in that project, and participated in SCADA systems design. While in both cases the teams I worked with were astounding – fully half of the people I have ever given recommendations for in my career are from that time – the security implications of what we were doing did not escape me in the least. They could not, as I was the IT manager in the Smart Meter Project case and the designer in the AMR system product. Frankly, there were some areas that could be lots better, and everything was new so the security implications were still being figured out by both us as a power company, and our vendors as AMR server and reading device manufacturers.

THE MORE THINGS CHANGE… Well, the more they change.

Lori sent me an article yesterday that’s hosted on CNET – Money Trumps Security In Smart Meter Rollouts – that discusses, yet again, the ongoing issues with security in Smart Meter deployments. While time has changed some things in this space, much of what the “experts” say is overstated. So I thought I’d take a few minutes out of my F5 blogging duties and fill you in on AMR, how it works, and why in many instances security is not an issue, and in others the “center for power consumer advocacy” is a good idea.

At first I thought this was the “in extremis” type of security concern that we saw – pointing out that the reading servers were insecure, and ignoring that they were as secure as the other servers on our internal network, etc. But the more I researched this blog post, the more concerned I became. Historically, power companies have been rather conservative because they are heavily regulated and have a piece of infrastructure that is darned near essential for life, and in some cases is absolutely essential for life. There was talk of gridding smart meters when our project was rolling out, but we didn’t take it seriously… We were even pretty cautious testing device firmware upgrades pushed from the metering servers, testing and retesting to make sure we wouldn’t end up with some serious problems. So it surprised me that the current level of smart-meter technology is being deployed by power companies.

In fact, The Register had a much better article about the problem that even discusses how this is no longer theoretical, a worm has been designed to exploit the grid model of smart meter technologies. None of the things that these meters are doing is new since I was involved, but some of the how is certainly new

Traditional AMR had a two-way communication with centralized servers (sometimes through a collection point, but mostly not), and it was much more likely that an attacker would go after those core servers than the actual devices. Given access to that server, much like in a SCADA environment, an attacker could wreak havoc, but even assuming someone managed to hack into a power meter without getting fried, the potential gain would be modification of just the power usage of the one meter, or perhaps an attack on the server – which was rather thoroughly protected due to the nature of the data it collected and the tools available from it. We had heard of smart grid and rejected it for a variety of reasons, some of which were security.

In the Smart Grid world, every meter is one part of a vast network that communicates with each other. There are many things that this makes easier or more responsive, but very little that is new vis-a-vis AMR. The most obvious thing that Smart Grid gives you is a more rapid response time to outages. In the AMR world, an operator has to be flagged that a reading failed – sometimes taking as much as a day – and only then is an alarm raised. In most cases humans would be calling first. In the smart grid, meters check in with each other on a predetermined schedule. If one fails to check in, other meters “check on it” and if it is truly non-responsive, a flag is raised in operations much sooner than would happen under AMR.

The problem comes in from this very connectedness combined with an ever-increasing list of requirements for what a meter needs to do. At first it was just readings that needed reporting, but once there was a connection, other things came to mind like automating shut-off, outage troubleshooting, usage curtailment (turning off non-essential things like water heaters in shortage situations), etc. These required increasingly complex systems, and eventually the truly embedded nature of the devices was replaced by embedded versions of mainstream operating systems like Windows and Linux. This increases risk significantly, but it speeds delivery of customer-required features.

So the article linked above talks about how IOActive took advantage of some serious bugs in the update software to infect a whole grid with a worm. The only bit that isn’t covered is how they delivered the worm to meter zero. That would interest me, not many hackers will risk life and limb unless the return is huge, and a power meter can be a deadly device… But it doesn’t change the fact that if one does, they could do a lot of harm before anyone even knew the worm was there.

IN THE END, IT’s ALWAYS ABOUT THE DATA

The more salient argument – because of the volume of data that can be collected and the detail of that data – is to protecting consumer rights in regards to data about them. The old metering system required a person visit each meter, and ranged from weekly for the most high-volume customers to monthly or even quarterly for the average consumer. That tells the power company something about usage patterns, but not a ton. With a solid AMR system, data is read real-time for the most high-volume customers to hourly or even every five minutes for the average consumer. This data is put into databases, and represents a full-motion video of usage. In a Smart Grid world, these readings can be programmed to be even more frequent than AMR. Most uses for this data are valid and very cool – like noticing when a vacation property has been broken into because of an increase in usage, or notifying a consumer that their bill will be higher due to increased usage, or detecting broken meters because readings stop changing. But with that volume of data there is always the potential for abuse. Before I left the industry, some utilities were informing police of houses with potential marijuana growing operations due to overly large usage information that spiked during the night-time hours, for example. Which doesn’t sound so bad unless you’re the person that uses more power at night innocently, and suddenly the police are knocking at your door to ask you questions through no action of your own. Predicting when you will be away from home is also relatively simple with six months of data to review, and competitive information about industrial competitors can be had relatively easily if you can watch when they powered up additional machines.

That data must be protected, both from hackers and from industry insiders that think they’re doing a good thing. So the suggestion that an independent regulatory authority be created to determine valid uses of this data is appealing to me. And if you know me, you know that increased regulation in general is not appealing to me, but the job of government is to protect the interests of the people, and the percentage of people without electricity in the US is so miniscule as to be negligible. This is a case where regulation can help protect the majority with no real loss to anyone.

THE REAL QUESTIONS ARE…

So the security question is real, I just rate the risk as lower than the experts quoted do. Most of these systems utilize encryption or some form of proprietary protocol, all of them are tied to enough voltage to kill a person messing with them, and all of them are monitored more regularly than most automated access points. The weakness is real, just not as readily exploitable as say your WAP. Should they ever try to turn on local Wireless Access to create a “home network”, then I will revise my view, because giving any old device wireless access makes these much more appealing avenues of attack for hackers. Until then, the arguments are not the best. I’ve got a 900 Mhz transmitter and receiver in the basement that was being used by a couple of wireless AMR vendors, and it really was not that easy to intercept, decode, and make sense of the data they were sending… Even with full documentation on how they worked and drivers to use them. Yeah, we played with it, yes we have masters degrees in computer science, yes, we’re pretty smart and have hacker-like qualities. Still took a while to make them work. And I never did figure out how to mess them up by sending faked data, though admittedly I was just playing with them, not seriously hacking them.

On a more amusing note, I was working next to a group that developed an industrial meter that could do about anything, and when they went to sell it to customers, the sales guys never got past “embedded windows”. They had to redesign the product on Linux to get anyone to take it in – enterprises were worried about OS updates, who has control, a whole slew of security issues. I felt for the team, but that should have turned up in the market research, so not too much ;-).

I seriously hope they manage to lock these meters down without a huge hardware expense – take a 300,000 meter deployment, and add $10 USD to the cost, and suddenly your budget is destroyed. And it’s not difficult to at $10 USD to each meter if you have to send people out to add hardware to the installation. Man-hours plus hardware will eat that kind of money quickly. Until then, I’m very glad that my power company is using AMR, and I know the devices on my meter very well. I ought to, I set up the IT infrastructure and systems that read them.

Technorati Tags: Smart Meters,Smart Grid,Automated Meter Reading,AMR,Security,Don MacVittie,F5 Networks

Farewell DevCentral Team, not Farewell DevCentral!

Don MacVittie — Fri, 07 May 2010 01:34:34 GMT

Now that Jason has blogged about me leaving the team, I’ll chime in with my “cool things coming” post. As of the first of May, I am the shiny new member of the Technical Marketing Team.

I’m really pleased with the change and its timing. I was able to go out on the high note of the release of the new DevCentral, and felt like I had an impact on its delivery, and now I can move off and start my new role fresh. You’ll note the blog regularity since the first of the month, I hope to continue that trend, but needs of the company/team/role might impact that goal, we’ll have to see where this takes me.

During my three years on the DevCentral team I was party to two major revisions to the site, development of some super-secret stuff, bringing on some great contributors, and some labs that I’m pretty proud of – even though as labs they are proof-of-concept and not completed solutions (always did want to make the Blackberry monitor application into a full blown management app, just never had the time). That and some documents to help the community get more out of their F5 gear – and maybe a few overly snarky blogs. Just a few.

The team has some great members on it, and since I’ll still be blogging and they’ll be doing what they do, you’re unlikely to notice much change on the community member end. Hopefully that’s good news to you. I’ll even miss Scott, who “Never really liked me”. ;-) Still think he rocked the tutorial page video player, and should be commended loudly for it.

So farewell to my DevCentral Teammates, it has been an experience… But don’t count me out yet, the new boss has ideas for me. I don’t think they include a clown suit, but I guess you never know :-).

I’ll leave you with a couple of “blast from the past” DC images that I downloaded for one reason or another over the years…

Everybody’s favorite top-right image!

The wayback machine says this F5 DevCentral logo was once cool.

Yeah, I linked that last one.

And thanks to Jason for the fond farewell. I am but a Skype-click away.

There Are Several I’s in This Team

Don MacVittie — Fri, 12 Mar 2010 17:18:51 GMT

Posted a new article last night on configuring BIG-IP LTM VE In a VMWare Team environment with servers (DevCentral login required) and just wanted to let you all know.

It’s a relatively complex topic, considering that the pieces all work well separately, but if you’ve configured networks in VMWare before it isn’t too bad to get going. I chose CENT-OS as the server OS because then if any of you decide you want to toy with the team, you can email me and I’ll get you a copy somehow. You’ll need VMWare, and a license key for the BIG-IP VM, but otherwise you should be able to mess with it as-is. I prefer though that you use the article, which tells you how to set up the same environment, so if you’ve already got BIG-IP LTM VE and VMWare, just follow the steps.

For my part, I’d like to thank Jason and Lori for their JIT help with two different issues I ran into. They’re smart cookies, read their stuff.

There’s plenty of great LTM VE content on DevCentral, so check it out if you have an interest.

Until Next Time – when I’ll be talking about shadow copies…

Don.

ps: There are several I’s in this team because it’s running a BIG-IP, so there are definitely I’s in it.

ARX Config: Finally.

Don MacVittie — Mon, 08 Mar 2010 16:28:27 GMT

Last week was golden for me. Three of my projects had major blocking issues, all three were resolved in the course of the week. That makes this week writing time, since two of the three projects were to support writing I want or need to do.

This is the start of that process. The first item to get a break last week was my ARX configuration. When I left off, some of the storage could join the domain and some could not. I needed everything to play nice in the Domain so that I could pull them all together under the ARX. On Wednesday evening, RDP just dropped to the ADS server. I walked over to the lab and checked from the console what was going on, and it couldn’t get to the local network, let alone anywhere else. I rebooted, and it was better, could get to some things but not everything. Finding this to be terribly odd behavior with no real obvious symptoms like messed up routes or anything, I traced the ethernet cable. And found that it was plugged into a place it shouldn’t have been. While this is clearly a leftover from a previous bit of testing Lori and I were doing, I’m a little confused how it ever communicated. And yet most of the machines I was using for testing were joined to the domain, so it certainly was communicating. Sometimes.

I moved the cable, and everything started playing much more nicely. In fact, that cleared up most of the remaining issues.

Since I had a lot of non-functional items left in the ARX configuration, I opted to wipe the user level configuration on the ARX and start cleanly. It’s pretty easy to wipe an ARX config, it’s simply deletion of a startup file and reboot, so I did that, removed everything from the domain and rejoined it while the ARX was rebooting just to make certain all was communicating cleanly, and 20 minutes later the ARX was configured with both NAS devices behind it, and exposing shares to the domain.

Ta-Daaaa!

So I went through and snapped you some screenshots. I kept saying I thought my problems were not the ARX, and the speed with which everything was added and working explains what I meant. Here come the screenshots with everything basically configured. It is not set up to do anything fun yet… I understand you may have forgotten this by now, but the point of this blog series was to show you what that cool stuff was and how to do it. So the rest of this blog shows some screenshots and talks about the architecture, and the next blog will hop right in with configuring shadow-copy.

All of my config was done with the UI. While I did trouble-shooting command-line, the UI gave me the opportunity to show you some pretty pictures, so I used it.

First thing you do is configure a namespace – a container for most of the other items you need to create. It holds publicly advertised shares and back-end filer shares, tells how to communicate with the filers and how to expect end users to communicate with you. It also holds the location that all shares are to use for metadata storage. My namespace is ingeniously named “ARXStorage”. The interface for the namespace is CIFS – I turned of NFS completely in this namespace because if it is included as a communications option, every NAS must have NFS access to every share. For simplicity, I disabled it. Some of our shares do support NFS, but we didn’t need NFS access through the ARX. And trust me, after the pain I went through with ADS, this device was going to use it.

The Namespace then contains “Managed Volumes” – exports from NAS devices that are going to be (eventually) presented by the ARX. I have two of them in this configuration – backup (which maps to one NAS device), and Dell (which maps to the other). Lori and I normally immediately back up the primary to a new NAS when we receive one so that a single PDU failure doesn’t drop our storage environment cold. More on why I bothered to tell you that in a moment. First, the Managed Volumes in Namespace ARXStorage.

There you have it, not much to see. Currently neither is listed as a shadow-copy target, both are enabled and online.

These are shares the ARX is going to manage for us, The /backup share is mapped to /backup1 and is actually actively used – hazards of a growing and changing network – but the name is all over so I’m unwilling to change it. The /Dell share is the default share on Powervault servers - /NASShare.

If we take a look at the volume by drilling down in, we can see…

As you can see, there is a lot going on here. The “files” line is way off base (there is over a terabyte of data on the disk), but I took this screenshot as soon as it was up, so import was likely still going on. Notice that Metadata Free Space and Free Space are the same – this is on a NAS that uses thin provisioning, so I would expect them to be very close.

At this point the back-end is set up. We have two shares imported from two filers, the ARX knows how to communicate with them, and it is doing so well enough to tell us how much space is used and free on the disks. Next we need to add the front-end, a way for users to access these shares through the ARX.

So we first create a Virtual. You don’t have to create a virtual first, you can just start defining exports and if there is no Virtual it will ask you to create one (okay, require you to, not ask, but you know). I’m showing you the Virtual first to keep things logically consistent and understandable. No matter how you create one, you must have the Virtual “first” or there is no way to export shares.

You will need an IP address for each Virtual you create. This is the entry point where users will access your shares – it masquerades as a Filer, in essence. Using my expansive wit, I chose to name this Virtual “ARXStore”. Note that it is already joined to the domain and is up and running. This screen shot was taken after all was configured.

Finally, we are ready to set up the exports. These are publicly exposed shares and/or mount points. I’ve made mine all CIFS for the reasons noted above, and here they are…

So they have a “Domain Name” which is the Virtual name in the domain, what Namespace they’re in (which impacts which backend shares they can see), Volume and Virtual Volume Path. The volume is the backend share, the path is the path that it will present to users.

Once those and the Virtual show online, you’re in!

Now if you go to any machine that is authenticated to the domain, and type in \\ARXStore in the Explorer (or equivalent) Address bar, you will see this:

There you have it, the two shares exposed in the Virtual. They are accessible and can be mapped or mounted from any machine that can authenticate to internal (which is purposefully few, we don’t like giving out actual information about our network, so it’s locked down).

Next we can start using these exports to do some interesting stuff. Remember I said that we normally block-copy the backup1 share (and a couple of others) to a new NAS? Well we’re going to try setting up shadow-copy next on the ARX to see if that will just copy it for us in the background. But that’s the next blog, not this one.

Until then,

Don.

ARX Config – Wellllll… Kind of.

Don MacVittie — Fri, 19 Feb 2010 01:32:03 GMT

So I’ll bet you’re wondering where the updates have been?

I took on this little side project knowing it would be an “after-hours” and “as time allows” project.

I knew that we had some big projects coming like the BIG-IP LTM VE and others that I’m not able to discuss yet.

What I didn’t expect is ADS re-installs, hardware failures, new equipment (other than the ARX), etc. If I had, I would have gone with delayed broadcasting so you didn’t see all my wardrobe malfunctions ;-).

So those other projects – five of them that are big by my count – are tromping all over my free time, and thus the ARX install. Not bad planning on my part – except the part where nothing goes as expected and I know to pad my time – but just a fact of life/work.

The new Dell NAS is in place and a member of the ADS domain, the Seagate NAS is configured, and both are also accessible outside the ADS domain, so all is set. There are some inside-ADS issues with the Dell I have to work out, but they’re just ADS config, I’m pretty certain, so that’ll go quickly.

Ohhh… Ahhh…. Lori and my new Dell PowerVault 3000 series, unboxed and ready to rack.

I just need to have some time to sit and get it all working as expected. I’m getting an hour here, an hour there, and that’s slowed me down quite a bit.

So unless something changes, my updates between now and Interop will be sporadic at best, since I’m going to skip from the blow-by-blow to the meat of the problem and only offer updates when I have something relevant to say about the ARX configuration and use.

So until next time, I’ll be busy, you should be too… And when I’m not? I’ll be pestering Lori about getting a Dell EqualLogic to do all that heavy lifting like hosting PDFs ;-)

Don.

ARX Config – Week Three

Don MacVittie — Fri, 05 Feb 2010 03:47:27 GMT

Well, I’ll bet you’re wondering how it’s going?

First, the reasons for my silence that you haven’t heard. Last Thursday my wonderful Dell Latitude D820 died. I loved this machine, thought so much of it that last time I updated my home machine I got a D830. But sadly, it was over three years old, and I spend 8+ hours a day abusing it, so no surprise.

The warranty ran out in December, so that left me (IT actually) no option but to replace it. The real reason to include this is to point out to you that F5 IT rocks, and many IT departments could learn from them. I’m a remote worker, I was limping by working on my home machine which had most of what I needed, but some key software like MS Project wasn’t installed, and webmail is… painful in any situation.

I told IT the machine was definitely dead late on Tuesday, on Wednesday I had a new machine. With my login info and the licensed corporate software installed. You don’t do any better than that.

So they sent me a Latitude E6400, and honestly, I’m pleased as can be. The only little problem I’ve had with it was (so far) not work related. I listen to DVD lectures from The Teaching Company in the evenings while working or painting or writing for non-work, and for some reason my newest set of DVDs plays fine on the machine but doesn’t have sound. Local WMV files play and have sound, the DVDs work on my home machine… So I don’t exactly know what’s going on there, but everything else works perfectly, so I’m happy. I’ll figure out what oddity makes them work on other Dell Machines and not on this one.

And I was complaining that I was out of space for VMs… No more! Much larger hard disk.

Anyway, you can imagine that getting the machine, pulling the hard disk from my old one (don’t tell IT, I’m not certain they’d approve), hooking up the disk via USB and dumping all the important stuff, reconfiguring just about everything – from bookmarks to networking settings – to work well in my environment sucked up just a bit of my time.

On the bright side, the days that I had no machine (it was nearly a week because we’d hoped we could fix the old one, but alas, Dell said “motherboard, fixing is a bad choice”) gave me a chance to get my storage house in order.

What did I do? Well I wiped the box running ADS and started over. It had ADS and DNS installed from who-knows-how-long-ago, but it was shut down… So I tried with the installed copies, but wasn’t real confident and it wasn’t working the best.

So I wiped the server and reinstalled, set ADS up again, joined my home laptop to the ADS domain, then worked at getting the storage into the domain. One required using the WINS name instead of the domain name to get it to work, the other required that I add it by had to ADS and DNS, and THEN tell the storage to join the domain. And as usually happens in that case, all went well.

Finally a chance to join the ARX to the domain. This is something I had not attempted up to that point because I wanted to have the things an ARX requires – storage and users – in ADS so that once it was joined I could get rolling. So I went to join the ARX into the domain… And realized I did not have the faintest idea how to do so.

AD Forest/domain list.

RTFM time, so I went and looked. The help on the system I have is very nice, and coworkers tell me that the help on DMOS 5.X is indeed very nice overall. That helped me get rolling, as did the logs, which are very verbose and I cannot recommend enough. In fact, all the oddities I’ve encountered to date – failure to access disks for metadata, failure to connect to shares, failure to negotiate NFS versions… All were ultimately the fault of my storage, and all we ultimately made clear to me via the ARX logs.

Lots of logs – and this does not count the automatically generated reports for lots of common activities.

The Managed Volume created under ADS.

One really odd thing I ran into that I am working around by ignoring it – because I can – is that I have a Namespace whose drive mountings failed – a leftover from the work I was doing in NFS and CIFS without Active Directory. It is stuck in the “starting” state, and I can’t get it out. Since the ARX won’t let me delete it, I’m ignoring it for now, and need to look up how to point out to the ARX that it will never finish starting since it has no volumes allocated to it. I’m pretty certain that this is a user error, so don’t judge the ARX poorly, even if it is an ARX error, you can ignore a single Namespace easily enough. Or better, don’t use SMB class storage so you’re not jerking the poor ARX around for three weeks ;-).

A Virtual defined on the ADS domain Internal.

Everything before that last picture that I’ve talked about has been the backend. Now that all the backend pieces were working together, it was time for me to set up the user-facing bit… The Virtual Service. This is the presentation “volume” – where the device advertises the Virtual Directory Tree to the network. It went easy enough on creation and making CIFS exports, and it’s up and running now.

The problem I’m stopped at now is another RTFM – I need to join the Virtual to the domain, but haven’t read how – it told me that I needed to and how to do so when I created the exports on the Virtual Service, but it was 3am and I thought “I’ll figure that out later…” And indeed I will, for this blog is long enough, and that’s where I’ll pick up the next installment.

Until then, enjoying my new laptop and seeing this all working together.

Oh yeah, and I have to make my regular everyday user not be SuperADSMan. I toggled him up to Enterprise ne’er-do-well while testing, and don’t want to forget to make him normal, and create the storage background users I need. More on that next time, when I’m sure what storage background users I want/need.

Don.

ARX Config – Week 2

Don MacVittie — Fri, 29 Jan 2010 03:17:39 GMT

I wanted to do at least two updates a week on this series, but circumstances conspired to keep me from an update earlier this week. In case you missed it, we’ve had a release or two going on (that link also has the “F5 joins NetApp Alliance Partner Program” Press Release on it if you missed that one), and I’ve got my bit to play in that. I also inherited a rather large project that I need to drive home, and it took a chunk of time just figuring out where it was and what the next steps were. There all the excuses but the one you came for are done.

Now the one you came for… My network, my devices.

The ARX is up and running beautifully, it behaves as expected except for one niggling bit that I suspect is due to the fact that I’m using SMB class NAS devices, so I’m not going to bring up. If you’ve got a NetApp or EMC NAS, you’re probably not going to see it, so I’ll leave it at that.

My devices on the other hand… Arggghh.

I’ll skip the hoops I jumped through and the number of times I attempted to add shares trying to get my NAS devices to play well with others. One was requiring a login to access a drive marked public, the other was giving me access denied errors. Both of these problems were evident from both servers and the ARX. I’ve changed quite a few settings over the last week, so I went back and started again. It turns out that one NAS device requires the volume in the nfs path, the other does not. Problem one solved. Access wasn’t denied (as the device told me), but the share I was trying to mount didn’t exist. I got the name straight. The other was a setting in the global config that I tracked down – it defaulted to no access for all new nfs shares, and I had created new ones for testing, so I wasn’t messing with production data. A few mouse clicks later, and theoretically both are ready to go. As a bonus, after nearly two weeks of changing things on these boxes to get one of them fully functional – the NetGear was partially functional last week – All of the clients on the network could still get to their shares.

So I go back to the ARX management screen, and attempt to mount a share on my Seagate BlackArmor NAS. This is where owning an SMB NAS really started to hurt. With a fully qualified path, it tried, and it failed because root_squash was turned on. This is a cool protection mechanism of nfs that changes the uid of root to be “nobody” so root has no special privileges and cannot break anything. Fine, I turned it off on the NetGear/Infrant, so I would just turn it off on the Seagate. Remember that the ARX is a file virtualization tool with a lot going on inside. It needs root rights to move things about (particularly files in a tiered environment), manage file access privileges, and to manage the metadata share.

Guess what? After lots of research, I discover that the BlackArmor NAS doesn’t let you turn off root_squash. So I have a solution for this, I have another Namespace (think virtual tree container) on the ARX that I can use that has CIFS enabled. I’ve SMBmounted this box a zillion times, and our XP clients access it fine with CIFS also. So I pop back into the ARX manager, change to that Namespace, and try to add it as CIFS.

“NAS Device does not support Kerberos Authentication” The ARX tells me.

Sigh. So I can’t do NFS because root_squash can’t be disabled, I can’t do SMB without an ADS machine.

The BlackArmor is our primary NAS, so I don’t want to move forward without it, but Lori took down our ADS machine a while back, and it’s physically gone from the building.

That leaves me trying to use SMB PDC functionality (vaguely recall doing that once), or setting up a new ADS server and hoping that the BlackArmor knows how to use that.

So a chunk of the reason I skipped blogging earlier in the week was simple… I had nothing much to report other than the obvious – Seagate BlackArmor isn’t enterprise class NAS. Duh.

And now I have a project for this weekend. Setting up ADS to move this project along, I’m tired of blogging about my network/storage issues and want to move on to actually using the ARX.

I tried to turn this into an excuse to snag a NetApp – something like a FAS2020 would do, but that fell through when a fellow F5er brought reason into the discussion… So that idea is out. For now.

Until next time,

Don.

All Options Were Considered… *

* Photo by Alex Nash and used under the Creative Commons License.

Click the image to view the original picture on Flickr