Abandonment Issues

Because of the recent departure of Apple's Senior VP of Enterprise Sales, John C. Welch claims that "the Mac IT crowd" is wondering if Apple is abandoning the Enterprise. He then goes on to say that this depends on whether or not you thought "Apple was, or wanted to be, an 'enterprise' company." Um... No it doesn't...

Mr. Welch raises some good points in his article, the main premise of which is that Apple is, indeed, not an enterprise company, an idea I fully agree with. But the fact is that it's certainly possible to worry that Apple will abandon the enterprise without thinking of them as an enterprise company. And that's because Apple makes enterprise products that some of us have come to rely on.

I don't think many IT folk — Mac or otherwise — think of Apple as an enterprise company. In fact, I'd venture to say that we worry that Apple will abandon us because we know know full well that Apple is not such a company at heart, and that their connections to enterprise are tenuous at best. But some of us do actually appreciate the design and ease-of-use of their server products. In my case, I've come to rely fairly heavily on Mac OS X Server for cross-platform authentication, among other things. I get flack for this sometimes, but the plain fact is that no one has integrated authentication for Mac, Windows and Linux in one spot in such an easy-to-build package as Mac OS X Server. Could I do this with a Windows server? Sure. Could I do it on Linux? Yes, of course I could. But I'll spend twice as long building it, and twice as much time maintaining it, when Mac OS X Server does it out of the box with ease and grace. I suppose I could punch myself in the face over and over again as well. Do I want to? Not particularly.

I often worry that Apple will someday leave the server market altogether. I sometimes even worry that Apple will stop building high-end workstations. Hell, who knows? Maybe Apple will stop selling computers one day. But I don't worry about these things because I think Apple is one kind of company or another. I worry about them because these are products that I enjoy using on a daily basis, and I would like to keep using them for as long as they're the best tool for the job.

Just Open It

One of the new security measures in Leopard is a confirmation dialog that pops up anytime you attempt to open a file or executable bundle that was downloaded from the internet. You may have seen this dialog after downloading and launching a new application, for instance. It looks something like this:

Some have hailed this as a breakthrough in security standards, I'm sure, though I can't find any specific sources to cite. Most folks who notice such things are actually rather annoyed by the alerts, likening them to similar attempts at security in versions of Windows. Indeed, it's difficult to make the argument that such a warning would be very effective at preventing user-initiated security breaches. In reality it's just the sort of warning that most people click through because 99% of the time it's unwarranted. This has most SysAdmins and advanced users asking, "Why bother?" After which they promptly ask themselves, "How do I turn this shit off?"

The bad news is that there doesn't seem to be a GUI way. There is no simple System Preferences checkbox for the alert. The good news is that there appears to be at least a couple possibilities for getting this done.

The first approach involves watching whatever folder you use for downloads, either via a Folder Action or via a Launch Daemon. The basic gist is that anytime that folder gets modified a script runs that removes the flag that causes the warning to appear from all items in the downloads folder. Because, you see, that's all that's going on. Anytime something is downloaded to your computer, Leopard flags it. When you open it in the Finder, the flag triggers the alert. Confirmation of the alert removes the flag. That's the mechanism. Using the watch folder method simply disables the flag, which is akin to hitting that "Open" button in the alert, only it happens automatically in the background.

There are a couple problems with this method. First off, it's difficult to implement, and second, it breaks anytime you decide to start using a different folder for downloads. And then there's the obvious fact that it's just plain ugly: a script workaround to disable an OS feature? Bluglgh! Terrible!

A better solution would be some sort of preference file. Apple often enables — but hides from the GUI — certain advanced preferences, allowing more savvy users to set the preference either by the addition of a preference (.plist) file, or by using the defaults command. There are all sorts of mods for the Dock, for instance, that, while unavailable through the GUI can be set via defaults. It appears that Apple has provided such an invisible preference for the Leopard download alert as well. And if you know how to create and edit a .plist file, activating the preference and disabling the alerts is fairly straightforward.

Some Background The instructions I'm about to provide have been posted in numerous other spots on the Web. But when I went searching for this info there seemed to be no single place that explained not just how to perform the modification, but also what it did and how the mechanism we're defeating works in the first place. So I wanted to provide some background info about just exactly what's going on here.

Metadata The first piece of this puzzle is the basic concept of metadata. Metadata is often best described simply as "data about data." And in Leopard we (or, more particularly, developers) finally have the ability to add arbitrary metadata. That is, as a developer, you can add any sort of descriptive data you want to your files and the files your application creates. Mac OS X 10.5 and up can read that metadata, interpret it and behave accordingly. So, to follow our example, when we download Camino from the Internet, the Mac OS (specifically, the Finder) adds some metadata to our Camino application, and that metada says, "This application was downloaded from the internet and has not been flagged for approval." Now, when we go to open the application for the first time, the Mac OS also knows to behave a certain way based on this metadata, and so we see that alert pop up. Clicking the "Open" button writes new metadata to the file, and that metadata now says, "This application has been opened and flagged for approval." The OS now knows not to present the alert the next time the app is opened. Such is the power of metadata.

By the way, I should mention that you can actually tell which files have metadata associated with them by doing a long file listing:

DrMac:~systemsboy$ ls -l Movies
drwxr-xr-x   9 systemsboy  staff        264 Nov 14 09:35 .
drwxr-xr-x  10 systemsboy  staff        296 Nov  7 17:20 ..
-rw-r--r--@  1 systemsboy  staff  407041761 Oct 31 15:45 DownloadedMovie.mov
-rw-r--r--   1 systemsboy  staff  407041761 Oct 31 15:45 LocalMovie.mov

Notice the "@" symbol at the end of the permissions info for the DownloadedMovie.mov file? That means that this file has metadata associated with it.

You can look at any metadata your files contain by way of the xattr command. The command is simple to use:

DrMac:~systemsboy$ xattr -l Movies/DownloadedMovie.mov
Movies/DownloadedMovie.mov: com.apple.quarantine: 0000;48fa188d;Firefox.app;|org.mozilla.firefox

Here we can see that our DownloadedMovie.mov has an Apple-branded quarantine flag, and that it was downloaded with Firefox. Nice!

UTIs The next piece of the puzzle comes in the form of something called Universal Type Identifiers, or UTIs. UTIs are a specific kind of metadata that are meant to describe and define types of files. So, our Camino application has metadata not only that tells the OS that it came from the Big Bad Internet (BBI), but also what sort of file (or in this case bundle) it is, namely that it's an application. This particular piece of metadata is the UTI, and there are a bunch of them. The complete list can be found on Apple's Developer site. UTIs can be extremely useful, and they'll actually allow us some flexibility in defeating Apple's quarantine mechanism. Because with UTIs we can specify which sorts of files trigger the alert and which don't, which is pretty great if you need it. And even if you don't, it's something that might come in handy down the line.

The Defeat So, with that out of the way, let's step through our method for disabling the Mac OS alert for downloaded files.

1. The first thing we'll need is a file in our Preferences folder called "com.apple.DownloadAssessment.plist." If you have one, great, if not make one, either with Property List Editor, TextEdit or whatever text editor you have handy.
2. Next, populate the file with this data:

    <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
      "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
   <plist version="1.0">
     <dict>
       <key>LSRiskCategoryNeutral</key>
       <dict>
         <key>LSRiskCategoryContentTypes</key>
         <array>
           <string>public.item</string>
         </array>
       </dict>
     </dict>
   </plist>

3. Finally, restart the Finder.

From here on out, the OS will flag any item downloaded from the BBI (the UTI "public.item") as "Neutral." This should completely eliminate any sort of alert regarding downloads. Of course, there are exceptions — and I'll talk about them in a minute — but this should mostly do the trick.

Using UTIs and the Risk Categories outlined here, you can vary and specify the OS behavior for a wide range of file types. Want downloaded applications to raise the alert, but HTML files to be considered safe? Not a problem. Just specify as much in the plist file. It might look something like this:

 <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
   "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>LSRiskCategoryUnsafeExecutable</key>
    <dict>
      <key>LSRiskCategoryContentTypes</key>
      <array>
        <string>public.executable</string>
      </array>
    </dict>
  </dict>
  <dict>
    <key>LSRiskCategory</key>
    <dict>
      <key>LSRiskCategoryContentTypes</key>
      <array>
        <string>public.html</string>
      </array>
    </dict>
  </dict>
</plist>

Now, applications we've downloaded from the Internet (UTI "public.executable") will be treated as "Unsafe Executables" and will prompt the quarantine alert. But downloaded HTML files (UTI "public.html") will be treated as "Safe" and be exempt from the alert. As you can probably gather, you could get really, really fine-grained here if you needed to. Most folks will be covered with the first example, but as a SysAdmin it's always good to know you have options.

Exceptions Now earlier I mentioned exceptions. And one of those exceptions is Camino itself. Remember earlier when I specified that developers could add metadata to files? It's important to understand that it's not always the Finder or even the Mac OS itself that's creating this metadata. Applications themselves can also write metadata to files, and that's exactly what Camino does. Anytime you download a file with Camino it adds it's own metadata to the download. And that metadata will trigger the Mac OS's quarantine alert. So the above hack fails for applications such as these, and there's not much to be done about it. The rationale for Camino's approach is at least partially detailed in bug report 464333 on Camino's developer site.

UPDATE: The folks at Camino are now re-examining the issue.

Further Reading Credit where due, I collected all my info on this topic from the following sources: The Folder Action Hack Arbitrarily Extensible Metadata Uniform Type Identifiers (UTIs) Download Security Assessments Creating and/or Modifying the Preference File

And inspiration and help for this post came from an excellent bunch of comments in another article right here on this very site.

In Conclusion I have to admit, I find what Apple is doing here fascinating. I'm not sure that popup alerts for every downloaded file is a great idea in practice, but I do think it's an interesting example of what's possible with metadata and highlights just how useful this addition to the OS could be if leveraged properly. Imagine if metadata were something in the hands of everyday users. Imagine if you could assign your own set of actions based on your own set of metadata tags. It could fundamentally change the way you interact with your computer, essentially allowing you to program it at will to your whims. And maybe someday it will, if Apple sees the potential here and continues to push forward with this Great Metadata Experiment in fresh and innovative ways.

Leopard umask

This is one of those I-keep-forgetting-how-to-do-this posts, so I'm writing it down. It's certainly been posted elsewhere, but I'm tired of going looking every time I need it. So here it is. In Tiger a simple defaults command could be used to modify a user's umask (a setting that controls the default permissions for newly created files and folders). Leopard, however, changes the way this is done. Leopard instead uses a launchd configuration file. To create a custom umask for all users of a system (i.e. all user-level processes):

1. Create a file called launchd-user.conf.
2. Place the file in /etc/
3. Enter the property, then the umask setting in the file, like so: umask 002
4. Restart the machine.

The restart may not be necessary, but if I recall it was the only way I could get it to work. If you don't want to reboot, you'll at least need to restart launchd and any application or process you want to use the new setting. Rebooting, though, is a nice catch-all.

Some additional info: if you want virtually all applications (i.e. system-level processes) to use a custom umask, you can leave the "user" off the file name. Using /etc/launchd.conf will have said affect, but it is not recommended by Apple (or me for that matter).

Setting a custom umask in general isn't something I recommend either, but it's damn handy in certain file sharing environments in which multiple users need access to the same stuff, but where ACLs — the preferred method for setting up complex file sharing permission sets — aren't an option. Creating a common group for the users and setting up their umask to create files and folders that are group-writable is an acceptable workaround in many scenarios. Which, by the way, is what the above setting will do. A umask of 002 will create files with permissions of 775 (the opposite of 002 — it's a mask, silly).

Okay then. Happy umasking!

Faulty Arguments

I've been following the running commentary on Apple's decision to remove firewire from the MacBook. The latest article comes from Ars Technica, who responds to the "Steve Jobs" quote:

"Actually, all of the new HD camcorders of the past few years use USB 2."

The Ars writer chimes in, chastising people who want firewire by, essentially, telling them there's no demand. But see, what we're trying to tell you is that there is demand. We still want firewire at the consumer level. Making the argument that no one wants it won't really work on folks who do want it.

Ars and others are also equating this to Apple's ahead-of-the-pack decision to lose the floppy drive on the original iMac, when clearly this is different: Apple is now following the pack, not leading it, by Ars' own assessment:

"The truth is, FireWire—and in particular the FW400 variety—has been slowing [sic] disappearing for the last few years. Apple eliminated FireWire from iPods several years ago, since it allowed slimming down of iPods and made the nano and shuffle possible. Even external hard drives rarely have a FW400 port; FW800 and/or eSATA are the ports that are usually offered if there is anything other than USB 2. And, as Jobs says, most if not all of the consumer HD cams on the market now use USB 2."

Oddly, firewire's lack of ubiquity didn't stop Apple from supporting in the first place.

And finally, Ars backs up "Steve Jobs'" idiotic and patently false claim, stating:

"And, as Jobs says, most if not all of the consumer HD cams on the market now use USB 2."

Well, first off, that's not what Jobs said. He said "all," not "most, if not all." He said "HD camcorders" not "consumer HD camcorders." And he said "HD camcorders of the past few years," which is just wrong. In fact, the dominant consumer-level HD format for the past several years has been HDV, which is transferred over firewire. HDV cameras are still plentiful in the market, and will probably remain popular for at least another year or two. It's only been in the last year or two that AVCHD — an MPEG-4 variant with much lower bitrate requirements capable of transferring over USB — has gotten good enough to be taken seriously by anyone who knows anything about video.

Ars even makes the argument that editing video on a MacBook is no fun:

"I can also say from personal experience that trying to edit HD video on a MacBook is pretty much the worst experience ever, and I would never wish it on my worst enemy."

Fine, and I tend to agree. But the fact is, I know plenty of people who do want to continue editing video on a MacBook. They've been doing it for years, and are quite happy with it. This article seems to be addressed to people with no MacBook video experience. But those are the people who are complaining, the people who are currently doing video on their MacBooks. Telling them that they don't really want to be doing that, and if they do, to go buy a MacBook is condescending and insulting.

I'm generally a fan of the reporting that goes on at Ars. But this article has me incensed. It bothers me when big sites like Ars Technica try to tell people that their desires are unresonable, but it's a hundred times worse when their logic is faulty and their facts are just plain wrong.

Syncing Makes Me Cranky

Someone recently noted that every time I attempt to synchronize some portion of my increasingly expanding digital life — be it Address Book contacts, calendars, mail, or any of the other bits of data that seem to be essential everywhere I go and at all times — I get cranky. It's true. Syncing has become the new Holy Grail for just about anyone who has data on more than one device, which is getting to be lots and lots of people. It's no surprise that my most popular post ever has been, essentially, about syncing calendars. (Publishing, actually, but it's basically a sync issue.) There are also any number of sync-related products on the market these days. But I have yet to hear of any that would meet my syncing needs for any length of time — Mobile Me, for instance, looks great on paper, but sounds disastrous in practice.

A new service, Fruxx (currently in beta), looks promising. But so far I've opted for free alternatives, and, most recently, those have revolved around Google (now that it supports IMAP — more on that later), which is increasingly sync-able with the Macintosh platform, and increasingly present in my digital life.

So why the cranky? Two reasons, really. The first is that I always feel that syncing should be easy. The second is that syncing is never easy. Syncing is, at best, a big fat pain in the ass, and at worst a vector for data loss.

Why is syncing so tricky? An illustrative scenario that springs to mind is the moving of data. Let's say I have 5 GBs of data in a folder called Movies. This folder is located in my home directory. That home directory gets synced to a spare hard drive via Time Machine. Now let's say I move those movies to another folder in my home account called Old-Movies. At last sync I had a 5 GB folder called Movies and no folder called Old-Movies. Now I have an empty folder called Movies and a 5 GB folder called Old-Movies. There are a few ways to sync this data move, and none is completely foolproof. What Time Machine will do is what I call an incremental backup. That is, Time Machine will keep a copy of the 5 GB Movies folder and make a backup of the 5 GB Old-Movies folder, even though their contents are the same. This is safe, but not storage-efficient.

Another model is to mirror the data, in which case the source is adhered to strictly, and anything not on the source is deleted from the target. In this case, the files in Movies would be deleted and the ones in Old-Movies would be copied. This is more storage-efficient, but also more destructive; it does not take into account the need to access previous iterations of the data. And all this gets even more complicated when there are two data stores and each is modified separately. For instance, what if I'm storing, accessing and editing copies of my contacts on my phone, my computer and my Google account all at the same time. How do we keep these data stores properly in sync with one another?

Again, there are numerous models. But one of the best so far — and particularly for small bits of data, like contacts and calendars, but not so much for large data like, say, movie files — is what's being referred to these days as "The Cloud" model. "The Cloud" is really just a marketing term for keeping your data in a centrally located, globally accessible place. In my biz we call these servers. Using servers to store data works well for syncing because the "master" data set is accessible from all of its clients. Make a change on your phone, and that change goes directly to the server. Next time you log on to your computer, it too talks to the server and gets the latest changes.

This is the approach I've been trying to use. These days I am syncing my iCal calendar data to Google Calendar, and my Address Book contact info to Gmail. Google is a good service to use for my stuff because it's free, extremely reliable — it's Google for Christ's sake — and it works with the Mac. But setting all this up is inordinately painful. For calendars I am forced to use a Byzanntine setup process involving login info and insane calendar URLs for each calendar on each computer.

With 10 calendars on 3 computers, this has been annoying, time-consuming and error-prone. Not to mention cranky-inducing. For my Address Book contacts the process was much easier: I just entered my Gmail info into a sync panel in Address Book and I was off to the races. There are problems with this too, though. For some reason I only get the Google Sync option if I've attached an iPhone to the machine in question, which is ironic considering the iPhone is mainly a target, rarely a source, and generally relies on a computer to be the data "master." Also, Address Book and Google only seemed to sync when I synced my iPhone to my computer, greatly limiting the usefulness of this feature. My main Mac, my iPhone and Google are now in sync, but what about my other computers?

But what's most annoying is that all this setup is completely redundant. It's all been done before. All the info I need is already there. In Mail.app.

For email I've long used a protocol called IMAP, which is an example of the concept of centrally located data that's been around for some time. Yup, that's right. We've been doing mail in "The Cloud" for over twenty years now. In fact, I'd venture to say that if we'd had the sort of standards for calendars and contacts that we do for mail, we might not even be having the whole "Cloud" conversation. Calendars and contacts would all be on our mail server and served over well-worn, well-understood, free protocols. Fortunately, we're getting there, and again, by running their services using free and open protocols, Google is leading the charge. And Apple is right behind them with support for standard, open formats like CalDAV and VCF in their applications. But the process needs to be simplified: If Mail.app has my Google user info, that should be all the system needs to set up the other services.

Ideally, I think it would be really neat if someone offered up a service that simplified all of this. Imagine a site or an application that consolidated all — and I mean all — your email, calendars and contacts. I know this is what Mobile Me is going for. Google is also, basically headed in this direction. But Mobile Me can't pull your Google data and vice-versa (or at least not easily). No one is taking into account users' current data sets. Thus far, these systems are still too proprietary. This all needs to work like mail does, only better. Ideally, I could enter a username and password, and the URLs of a few servers and The Service would aggregate everything for me. Just like Mail.app does with all my mail accounts now, The Service would do for contacts and calendars as well.

Of course, then what would I get cranky about?

Don't worry. I'll find something.

Addendum: Before I even had a chance to post this, Khoi Vinh, of Subtraction fame, posted some tangential thoughts on cloud computing in a financially unstable world. I found the following passage particularly relevant to at least part of my argument:

"...I pay a company to store my mail on the cloud, ostensibly, but I have no more or less trust in their longevity than I do in any other online business. Through the underestimated miracle and beauty of IMAP though, all of my messages are also mirrored on my hard drive, an invaluable insurance policy against the sudden disappearance of my mail server or host provider. To me, the IMAP approach is really the ideal approach, and I really wish it was a model for more Web services."

See? I told you so.

Khoi's article is less about syncing and more about data storage and archiving, but it's an interesting read if you're interested in where, for instance, all your Google docs will go when the company crashes under the weight of the current financial crisis, and other drawbacks to so-called "Cloud Computing."