The Real Cloud

When I first saw the title of this article over at Ars I rolled my eyes: Cloud Computing: a short introduction

Then I read it.

It turns out to be an incredibly thorough, yet brief, technical description of the term Cloud Computing. The article does a great job of defining Grid Computing and then uses that definition as point of comparison to arrive at a very real and very useful definition of The Cloud. I'll be a little less embarrassed to use the term after reading it. Only a little. But still...

Well worth a read for anyone who needs some clarity on the whole Cloud thing.

Why We Tell You To Reboot: Redux

I recently wrote an article, entitled Why We Tell You To Reboot, that described a Final Cut Pro bug which, after going to great troubleshooting lengths, I was ultimately only able to solve by a simple reboot. Shortly after posting I received a single comment from a fellow admin and blogger:

"You really tell people to reboot for no particular reason?

I don’t believe we should accept that standard from OS X, and what kind of an IT person are you if you’re blindly proposing solutions without any reasoning to back them up?"

What kind, indeed.

I posted a response to the comment that basically explained my position in a nutshell, but I thought it was worth writing a follow-up on the question, both for thoroughness and for those who didn't happen by the comments section of the previous article, or who may have had a similar reaction.

When a user comes to me with a problem, my primary goal is to fix the problem and get the user working again. Typically what happens is that we have a little discussion about what's going on. Once I feel I have a good handle on the symptoms, often the next words out of my mouth are, "Did you reboot?" If the answer is no, and the situation permits, I will recommend that they do so. Rebooting is often my first step in troubleshooting.

I believe (though I can't be completely sure) that my commenter took issue with the approach because blindly telling the user to reboot to solve problems gives the sysadmin no information about what those problems are and what caused them. But in fact, as I'll demonstrate in a moment, it's not blind, and it does tell us one important thing: rebooting either solves or doesn't solve the problem. This in and of itself can be crucial troubleshooting information if there is a deeper problem at work.

But the fact of the matter is that, probably 80% of the time, there is no deeper issue. The fact is that rebooting routinely fixes problems with no other practical solution, such as the one I described in my article. Moreover, it provides the end-user with a method of troubleshooting that is likely to achieve the desired results — allowing them to get the system or an application back to a working condition — without the need for admin intervention. This is win-win: it saves both the user and me time and energy and, by determining whether or not a reboot is helpful, still provides valuable troubleshooting information.

I would even argue that rebooting should almost always be the first step in troubleshooting. When a user comes to me with a problem, I have no idea what they've been doing on that system. I have no idea how many nor which applications they have open. I have no idea what sorts of preferences they've set. There's simply no way for me to reliably predict the state the user has put their machine in, and thus whether or not this is a system- or user-level problem. The only way for me to get things back to some semblance of a known, working state is to reboot the system. Rebooting has myriad benefits, not the least of which are: clearing stale caches; recreating network connections; and freeing up RAM and disk space. In fact, it seems almost crazy to proceed with most troubleshooting without first rebooting.

You may have noticed that I keep saying that rebooting should almost always be the first troubleshooting step. That almost is there because, obviously, there are times when rebooting is not a good first step. Primarily, when a user stands to lose work by rebooting. If an application is hung and the user hasn't saved his document, for instance, I don't tell them to reboot. Rebooting would be bad in this instance. Also, I usually try other troubleshooting methods first on my own systems, with which I am better acquainted (though this is often to my own detriment and rebooting would have been quicker and easier, as happened with the Final Cut bug).  Another instance in which I avoid a reboot is when there is a persistent problem that is not solved, or is only temporarily solved, by a reboot. Then I do need to get on that system and attempt to understand a problem. And that's what I do.

But, because Mac OS X is very reliable on the whole, these instances are extremely rare in my experience. The majority of problems are minor and are easily and permanently rectified by a simple reboot. I stand behind that recommendation, and any search of Mac troubleshooting articles will reveal that the advice is almost universal. That's because it works.

So, hopefully it's clear by now that I'm not "blindly proposing solutions without any reasoning to back them up." Hopefully it's clear now that there are a lot of good reasons to try rebooting as a first troubleshooting step.

And hopefully it's clear that the kind of sysadmin I am is the kind that likes to get his users back up and running again with a minimum of friction.

Satellite Home Directories

There are three basic methods in use today for hosting home accounts on networks in such a way that users have a single home account that follows them from computer to computer, giving them the same environment no matter where they log in. None of these three strategies works in a way that reflects how most people in the lab I currently work in — nor many of the labs I've freelanced for — use their computers and access their data. So I'd like to propose a third strategy that does.

Let's start with a rundown of the existing approaches.

Roaming Profiles

The approach Windows computers use is called Roaming Profiles. The way Roaming Profiles work is pretty simple. Users' home account data is stored on a centralizd server. When the user logs in to a client system her data is downloaded from the server to the client machine. She will access her data locally for the duration of the session. When she logs out the data will be synced back up to the server. The advantage of this approach is that the user has local access to her data and isn't beholden to the network while actually working. This makes data access generally faster and more reliable. The big disadvantage here is that if the user makes any big changes or creates any big files, a large data transfer will happen at log out, and then again at login to subsequent machines that aren't yet synced to the server. This both slows down the login/logout process and places an often undue burden on the network.

Because of the sorts of environments I tend to work in — data-intensive, video and image oriented facilities that create a lot of data — my experience with Roaming Profiles has been fairly poor. For my uses they've required a lot of management and have been somewhat unreliable. But, for the purpose of maintaining a user environment across multiple networked systems, they work well enough if you understand and plan for their inherent limitations.

Network Home Accounts

The method used by *NIX systems, Mac OS X included, for time in memorial, is generally referred to these days as Network Home Accounts. In the Network Home Account model, as with Roaming Profiles, the user's home account data is stored on a server. But when the user logs in using Network Home Accounts no data transfer occurs. Instead, the home account data is accessed directly from the server: new files are written directly to the server; settings files are read directly from the server; everything happens over the network and the network share that contains the user's home account data is treated just like a local volume. The speed advantage over Roaming Profiles at login and logout is obvious; there's simply no lag time as data gets transferred between the client and the server, because there simply is no data transfer. On the other hand, accessing your entire home account over the network can be slower than a local account even on the speediest of networks. And on slower networks, or networks with a great deal of traffic, you'll definitely notice the slowdown. There are also potential problems due to the constant reliance on the network and server. If the network becomes congested or the share becomes unavailable even for a second you're liable to feel the pain. If either goes down you're dead in the water until they've returned to service.

As network home account models go, I like this one the best. I've used it a great deal in educational settings in which resources are almost completely shared and it's fairly reliable and usable. But even this model can be frustrating and is less than ideal when compared to working from a local home account.

Portable Home Directories

The final model is called Portable Home Directories. Devised by Apple for laptop computers with occasional — but not constant — access to the network hosting home account data, Portable Home Directories attempts to combine the best of the Roaming Profile and Network Home models by providing finer-grained control over the sync process in what is otherwise a Roaming Profile approach. So, Portable Homes sync to specific data at specified times when they're on the network. Fine-grained control over what is synced and when is intended to mitigate performance issues at login and logout.

My main problem with this approach is that, in my admittedly limited tests, it doesn't seem to work very well. I also don't like the level of management required. The other models, once set, require little if any tweaking whatsoever. But I could see spending a great deal of time and effort getting my Portable Home Directory settings just so.

The Problem

But my overarching beef with all these models is that they don't really jive with the way most people in most of the environments I've encountered actually use their computers. This makes them use system resources less efficiently and yields a poorer user experience than if they did.

So how do most people work? Well, what I've tended to see in the media-based environments in which I've worked is that users are generally assigned a single computer. It's this computer from which they work almost all the time. Indeed, this is how I work in my current job. I'm almost always working from the computer in my cubicle. Almost.

Every now and then, however, I need to work from a different machine, and there are often times when I'm doing this that I realize that it would be extremely handy to have my entire home account — all my environment settings, files and folders — available to me on this other machine. But I don't. They're over there, on my cubicle machine. If only I could use the home account on my main computer directly, as thought it were a Network Home Account.

And this is the basic idea behind Satellite Home Accounts.

Satellite Home Directories

All the current models rely on the user's data being stored on and accessed from a centralized server. But why? Why can't the server be the user's main computer? In the Satellite Home Account model, the user's primary computer becomes the home account server for any user that sets her account as a Satellite Home Directory.

The way I envision it, it would actually be quite simple to set up. In the Accounts preference for the user would a be a tickbox to activate Satellite Home Directories. Once activated, the user's system would begin broadcasting Satellite Home Directory information, just like Mac OS X broadcasts Network Home Account info. The user would then work locally as normal, but when logging into another system on the network — a system that's listening for SHDs — the user would be presented with her home account over the network, shared directly from her primary system rather than from a centralized server. Simple.

Among the great benefits of this system are its simplicity and the fact that it requires no server. But the chief advantage comes from the fact that the Satellite Home Directory system works the way users tend to work. When you're on your main computer, which you are 99% of the time, you get a fast, responsive, local home account. When you move temporarily to another system, your environment follows you. It's a bit slower, sure. But hey, it's only temporary. The network overhead is significantly reduced from the other methods, and the user experience is also enhanced. It's win-win.

There's certainly no technical reason an implementation like this would be impossible or even particularly difficult. Most of the technology already exists, either in Mac OS X client or Server. All we need is for someone to program it. And while I doubt there's likely much interest on Apple's part to build something like this, I really think it'd be damn sweet.

And a boy can dream, can't he?

CalDAV On iPhone

One of the exciting new features in the iPhone 3 OS is the ability to access calendars via plain old, standard, vanilla CalDAV. This allows you to finally keep an updated version of all your calenders without syncing your phone to your computer. Just subscribe to your calendars on your iPhone, just like you do in iCal on your Mac, and you'll always be up to date because you'll always be accessing the centrally-located, server-side calendar. This is right and proper and as it should be. But CalDAV over iPhone does have some quirks and limitations.

Subscribing to a Single Calendar

The first thing that's not readily obvious is how to connect to your CalDAV calendars. It is possible to subscribe to multiple Google calendars, for instance, but doing so is neither straightforward nor apparent. Still, once it's done it's done, and it's mostly better than the alternative.

Subscribing to a single Google calendar — your main Google calendar, if you only have one — is really what the iPhone is interface is designed for. Where to do this is also not necessarily obvious:

1. Open the Settings app.

   

2. Press Mail, Contacts, Calendars.

   

3. Press Add Account...

   

4. Press Other.

   

5. Under the Calendars heading, press Add CalDAV Account

   

6. Enter your CalDAV server URL (for Google it's just "google.com"), username, password and a short description, then hit the Next button.

   

7. The iPhone will verify your information and then, if all goes well, your CalDAV calendar will appear in the Calendar app with the description you provided with the word CalDAV next to it in parentheses.

   

8. Now wait. It can take a few minutes for the iPhone to suck down the CalDAV data. I'm not sure why. But it should appear after a few minutes or so.

Navigating to this calendar will cause the iPhone to read the calendar data right from the server. If you have write access you will also be able to add, delete and modify events right on your iPhone and have the changes propagate to the server.

Multiple Calendar Subscriptions

UPDATE: See the end of the article for a better way to sync multiple Google calendars over CalDAV.

In addition to connecting to your main Google calendar it is also quite possible to connect to other CalDAV calendars to which you might want to be subscribed, though the process is hardly as automatic as the one above. Connecting to a single CalDAV calendar is easy for both you and the iPhone because the URL is simple and easy to predict, so the phone just does it for you. If, however, you want to connect to a shared calendar, the URL that you'll need to supply to the iPhone is long and complicated and, unfortunately, must be manually entered.

To do so, you'll probably want to start by getting that URL into the iPhone's clipboard. There are lots of ways to do this. You can get it right from the Google calendar site (if it's a Google calendar) using Mobile Safari. You could also send it to yourself in an email. The way I handle this is that I actually keep a spreadsheet on Google Docs that contains all my shared CalDAV URLs, that way I always have copy/paste access to them from the iPhone or from a computer. But however you do it, just get that URL into your clipboard, because you don't want to type it by hand. Trust me.

The process for adding additional calendars follows the above steps almost exactly. The one exception is that for the Server entry in step 6 you must pass the long URL you just got into your clipboard. Just paste it right there into the field. After switching fields, it will present the Server text as just "google.com" again, but it should have the full URL stored away in its memory. Continue with the process and you should see the new calendar in the Calendar application as before.

Advanced Options

If a problem occurs during subscription an Advanced Settings button will appear.

This button also becomes available after the calendar has been set up. There are three options to set here:

1. Use SSL

   This allows you to configure your iPhone to connect via SSL, which is required by some servers. Google allows this option and may or may not require it. Generally it's best to have this on if possible. Or at least more secure. If you're having troubles, though, you can try toggling this setting.

2. Port

   This allows you to set the SSL port, in case your server uses something other than the default of 443.

3. Account URL

   This is the actual URL to your calendar. The one you pasted into the Server field. If something went wrong during that step you can check and fix it here.

In Practice

There are some advantages and disadvantages to this system. And while I really believe this is how any serious calendaring system should work — all your computers are clients that simply get the calendar data from the cloud — the implementation on the iPhone is not exactly mature.

For one, CalDAV calendars other than your main Google one are a major pain to add to the phone. Google provides an app for doing this on you desktop system, but for the iPhone it must all be done by hand. If you have many calendars it is a long and tedious process.

For two, receiving CalDAV data on your iPhone is, in my experience, very slow. Any changes made on the server will not be immediately seen by the iPhone. It can be several minutes after opening the Calendar app before you see the changes and there's no indication that the sync is even taking place. This can be frustrating and misleading and requires you to always remember that there might be a change that's coming that your iPhone just hasn't seen yet.

Also, from time to time, though very rarely, calendars fail to load. I can't blame this entirely on the iPhone, though. This happens to me even in the browser on my desktop system. It seems to be a problem with Google's implementation and large numbers of calendars. Apparently Apple's not the only one whose CalDAV implementation is immature.

Still, for me this system has worked pretty well thus far. I have a buttload of calendars to which I must stay subscribed, and which I must keep current. And I just don't sync with my computer that often — plus, calendar changes are made at work — so that system was equally dodgy. My calendars don't change that often either, so waiting for data to sync is less of an issue for me, though when there are major changes, getting everything re-synced really blows.

Overall, now that it's in place, I'm very happy with CalDAV on my iPhone, though there is certainly room for improvement. I suspect it will only get better with time.

UPDATE: It looks like it did just get better. I've just discovered a better way to subscribe to multiple Google Calendars that I'm pretty sure wasn't there when I set all this up a few months ago. It's much easier and doesn't require signing in to each calendar. Just subscribe to your main calendar as described in the above outlined steps, and then, on your iPhone, go to this page:

https://www.google.com/calendar/iphoneselect

Here you can select the calendars that will sync over your main Google account.

This is for Google accounts only. If you just have regular old CalDAV accounts that you want to sync with, manual entry will still be required.

UPDATE 2: This method seems to speed calendar syncing as well. I think the old method required the iPhone to log into each individual calendar, whereas this new method logs in once and syncs all the calendars with the one account. Much less work for the iPhone than before. Nice!

Why We Tell You To Reboot

This is how still images in Final Cut Pro 7 looked to me after installing and updating to the latest version (7.0.1):

This is my dog:

She is not normally purple and green and swirly colored.

After an hour or so mucking about, reinstalling the application, trashing prefs and otherwise performing the usual maneuvers, I decided to take my own medicine and reboot. Why I don't just always do this first — like I tell everyone else to do — is beyond me. But sure enough, it worked.

Ah! That's better! Crazy mutt!