Just Open It

One of the new security measures in Leopard is a confirmation dialog that pops up anytime you attempt to open a file or executable bundle that was downloaded from the internet. You may have seen this dialog after downloading and launching a new application, for instance. It looks something like this:

Some have hailed this as a breakthrough in security standards, I'm sure, though I can't find any specific sources to cite. Most folks who notice such things are actually rather annoyed by the alerts, likening them to similar attempts at security in versions of Windows. Indeed, it's difficult to make the argument that such a warning would be very effective at preventing user-initiated security breaches. In reality it's just the sort of warning that most people click through because 99% of the time it's unwarranted. This has most SysAdmins and advanced users asking, "Why bother?" After which they promptly ask themselves, "How do I turn this shit off?"

The bad news is that there doesn't seem to be a GUI way. There is no simple System Preferences checkbox for the alert. The good news is that there appears to be at least a couple possibilities for getting this done.

The first approach involves watching whatever folder you use for downloads, either via a Folder Action or via a Launch Daemon. The basic gist is that anytime that folder gets modified a script runs that removes the flag that causes the warning to appear from all items in the downloads folder. Because, you see, that's all that's going on. Anytime something is downloaded to your computer, Leopard flags it. When you open it in the Finder, the flag triggers the alert. Confirmation of the alert removes the flag. That's the mechanism. Using the watch folder method simply disables the flag, which is akin to hitting that "Open" button in the alert, only it happens automatically in the background.

There are a couple problems with this method. First off, it's difficult to implement, and second, it breaks anytime you decide to start using a different folder for downloads. And then there's the obvious fact that it's just plain ugly: a script workaround to disable an OS feature? Bluglgh! Terrible!

A better solution would be some sort of preference file. Apple often enables — but hides from the GUI — certain advanced preferences, allowing more savvy users to set the preference either by the addition of a preference (.plist) file, or by using the defaults command. There are all sorts of mods for the Dock, for instance, that, while unavailable through the GUI can be set via defaults. It appears that Apple has provided such an invisible preference for the Leopard download alert as well. And if you know how to create and edit a .plist file, activating the preference and disabling the alerts is fairly straightforward.

Some Background The instructions I'm about to provide have been posted in numerous other spots on the Web. But when I went searching for this info there seemed to be no single place that explained not just how to perform the modification, but also what it did and how the mechanism we're defeating works in the first place. So I wanted to provide some background info about just exactly what's going on here.

Metadata The first piece of this puzzle is the basic concept of metadata. Metadata is often best described simply as "data about data." And in Leopard we (or, more particularly, developers) finally have the ability to add arbitrary metadata. That is, as a developer, you can add any sort of descriptive data you want to your files and the files your application creates. Mac OS X 10.5 and up can read that metadata, interpret it and behave accordingly. So, to follow our example, when we download Camino from the Internet, the Mac OS (specifically, the Finder) adds some metadata to our Camino application, and that metada says, "This application was downloaded from the internet and has not been flagged for approval." Now, when we go to open the application for the first time, the Mac OS also knows to behave a certain way based on this metadata, and so we see that alert pop up. Clicking the "Open" button writes new metadata to the file, and that metadata now says, "This application has been opened and flagged for approval." The OS now knows not to present the alert the next time the app is opened. Such is the power of metadata.

By the way, I should mention that you can actually tell which files have metadata associated with them by doing a long file listing:

DrMac:~systemsboy$ ls -l Movies
drwxr-xr-x   9 systemsboy  staff        264 Nov 14 09:35 .
drwxr-xr-x  10 systemsboy  staff        296 Nov  7 17:20 ..
-rw-r--r--@  1 systemsboy  staff  407041761 Oct 31 15:45 DownloadedMovie.mov
-rw-r--r--   1 systemsboy  staff  407041761 Oct 31 15:45 LocalMovie.mov

Notice the "@" symbol at the end of the permissions info for the DownloadedMovie.mov file? That means that this file has metadata associated with it.

You can look at any metadata your files contain by way of the xattr command. The command is simple to use:

DrMac:~systemsboy$ xattr -l Movies/DownloadedMovie.mov
Movies/DownloadedMovie.mov: com.apple.quarantine: 0000;48fa188d;Firefox.app;|org.mozilla.firefox

Here we can see that our DownloadedMovie.mov has an Apple-branded quarantine flag, and that it was downloaded with Firefox. Nice!

UTIs The next piece of the puzzle comes in the form of something called Universal Type Identifiers, or UTIs. UTIs are a specific kind of metadata that are meant to describe and define types of files. So, our Camino application has metadata not only that tells the OS that it came from the Big Bad Internet (BBI), but also what sort of file (or in this case bundle) it is, namely that it's an application. This particular piece of metadata is the UTI, and there are a bunch of them. The complete list can be found on Apple's Developer site. UTIs can be extremely useful, and they'll actually allow us some flexibility in defeating Apple's quarantine mechanism. Because with UTIs we can specify which sorts of files trigger the alert and which don't, which is pretty great if you need it. And even if you don't, it's something that might come in handy down the line.

The Defeat So, with that out of the way, let's step through our method for disabling the Mac OS alert for downloaded files.

1. The first thing we'll need is a file in our Preferences folder called "com.apple.DownloadAssessment.plist." If you have one, great, if not make one, either with Property List Editor, TextEdit or whatever text editor you have handy.
2. Next, populate the file with this data:

    <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
      "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
   <plist version="1.0">
     <dict>
       <key>LSRiskCategoryNeutral</key>
       <dict>
         <key>LSRiskCategoryContentTypes</key>
         <array>
           <string>public.item</string>
         </array>
       </dict>
     </dict>
   </plist>

3. Finally, restart the Finder.

From here on out, the OS will flag any item downloaded from the BBI (the UTI "public.item") as "Neutral." This should completely eliminate any sort of alert regarding downloads. Of course, there are exceptions — and I'll talk about them in a minute — but this should mostly do the trick.

Using UTIs and the Risk Categories outlined here, you can vary and specify the OS behavior for a wide range of file types. Want downloaded applications to raise the alert, but HTML files to be considered safe? Not a problem. Just specify as much in the plist file. It might look something like this:

 <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
   "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>LSRiskCategoryUnsafeExecutable</key>
    <dict>
      <key>LSRiskCategoryContentTypes</key>
      <array>
        <string>public.executable</string>
      </array>
    </dict>
  </dict>
  <dict>
    <key>LSRiskCategory</key>
    <dict>
      <key>LSRiskCategoryContentTypes</key>
      <array>
        <string>public.html</string>
      </array>
    </dict>
  </dict>
</plist>

Now, applications we've downloaded from the Internet (UTI "public.executable") will be treated as "Unsafe Executables" and will prompt the quarantine alert. But downloaded HTML files (UTI "public.html") will be treated as "Safe" and be exempt from the alert. As you can probably gather, you could get really, really fine-grained here if you needed to. Most folks will be covered with the first example, but as a SysAdmin it's always good to know you have options.

Exceptions Now earlier I mentioned exceptions. And one of those exceptions is Camino itself. Remember earlier when I specified that developers could add metadata to files? It's important to understand that it's not always the Finder or even the Mac OS itself that's creating this metadata. Applications themselves can also write metadata to files, and that's exactly what Camino does. Anytime you download a file with Camino it adds it's own metadata to the download. And that metadata will trigger the Mac OS's quarantine alert. So the above hack fails for applications such as these, and there's not much to be done about it. The rationale for Camino's approach is at least partially detailed in bug report 464333 on Camino's developer site.

UPDATE: The folks at Camino are now re-examining the issue.

Further Reading Credit where due, I collected all my info on this topic from the following sources: The Folder Action Hack Arbitrarily Extensible Metadata Uniform Type Identifiers (UTIs) Download Security Assessments Creating and/or Modifying the Preference File

And inspiration and help for this post came from an excellent bunch of comments in another article right here on this very site.

In Conclusion I have to admit, I find what Apple is doing here fascinating. I'm not sure that popup alerts for every downloaded file is a great idea in practice, but I do think it's an interesting example of what's possible with metadata and highlights just how useful this addition to the OS could be if leveraged properly. Imagine if metadata were something in the hands of everyday users. Imagine if you could assign your own set of actions based on your own set of metadata tags. It could fundamentally change the way you interact with your computer, essentially allowing you to program it at will to your whims. And maybe someday it will, if Apple sees the potential here and continues to push forward with this Great Metadata Experiment in fresh and innovative ways.

Secondary DNS in Leopard

I covered secondary DNS configuration in Tiger (10.4) Server a while back. And while the buttons have moved around a bit, most of those instructions apply to Leopard as well. Leopard does have one fairly cool new addition worth mentioning, though: forwarders. Generally I'm setting up secondary DNS for internal networks, and generally those internal DNS servers  serve DNS only for the internal networks. Everything outside the internal network is handled by external DNS servers (or by DNS servers that sit on a network of which we are a subdomain), and our internal DNS servers need to know who those server are. These external servers are called forwarders, in DNS parlance. They are the first stop for all DNS outside your local network. And you can now set them on your secondary DNS server in the Leopard Server Admin application.

To get to the settings, navigate to the "Settings" tab under the DNS service. In the bottom-most pane of the window you will see a box labeled "Forwarder IP Addresses:" Click the plus sign to add a server to the list, then type in an IP address. Typically you will add two addresses, one for the primary external DNS server and one for the secondary. These will often be your ISP's DNS servers, though if you're on a subdomain of a larger network you'll use the DNS servers for that network's domain (i.e. the subdomain systemsboy.com.mail will use the DNS servers for the domain systemsboy.com). Once you've entered and saved the settings, restart your DNS service and you're off to the races.

Requests for internal network resources will still be handled by your internal DNS server, but now external requests for things like "google.com" will be passed to the appropriate external DNS server. Even if your secondary has to take over DNS duties for a long period of time, you'll still be able to properly reach the Big Bad Internet without having to use cached or stale settings.

This is a very handy addition to the DNS configuration GUI in Server Admin.

Cheaper, Better and Only Slightly More Difficult

I don't usually buy albums online. So far my best option here has been the iTunes Music Store, but most songs on iTMS are of the low bitrate, DRMed variety, and I just can't abide by that. Why pay for music that assumes I'm a criminal when I get better, unfettered results by stealing? Generally, however, I try not to steal music either, so my preferred method has been the purchase of the tried and true CD. I'm an adult. I make a decent living. I certainly don't mind paying for music that brings me great enjoyment. Case in point: Elvis Costello's Taking Liberties. My girlfriend owns this excellent album of early, unreleased tracks. On vinyl. And I've been looking for an updated, digital version for my own collection for a while. But the record's been out of print, until now. Taking Liberties is now available, digitally. Via download. Only.

You can get the album on iTunes, or, as my girlfriend pointed out, on Amazon. After checking out the Amazon Store, I decided to give it a whirl. Particularly once I realized that the Amazon tracks are all DRM-free, and all feature high quality 256kbps bitrates, all for less money than iTMS equivalents, which are only available for certain tracks.

Buying music on Amazon for the first time is slightly more painful than via the iTMS, because you need to download and install the Amazon MP3 Downloader software. This is a fast download, and the installer takes no time at all. And once it's installed, downloading whole albums is a simple matter. Just go to Amazon's site, find the album you want and buy it using your Amazon account. The Amazon software will take care of the rest, downloading the files and importing the music into your iTunes library.

While the Amazon process is slightly more convoluted, requiring a separate install and a secondary application, there are advantages even beyond the high quality, DRM-free tracks. For one, I'm saved the trouble of reverse engineering said DRMed tracks when it's time to back up. In fact, never mind backups. I'll just use Time Machine for that from now on. Also, my music isn't tethered to one machine. And I can buy music from any web browser anywhere.

Overall, my first Amazon music purchase experience has been extremely positive. Amazon may just bring me smiling into the age of digital music downloads.

APM Partition Boots Intel Macs

I'd thought that if you wanted to boot Intel Macs you needed to use the recently available GUID partition table, mainly because that's what it says in Disk Utility when you format the drive. In fact, as it turns out (at least as of Mac OS X 10.5.5), using the Apple Partition Map (APM) boots Intel Macs perfectly well. It's exceedingly useful to have a partition format that will boot both architectures, particularly at the museum, where Intel and PPC Macs still very much coexist.

In fact, my Mac is a G5, but all the new hardware is, of course, Intel-based. And I'm trying to create a master build image for setting up new machines. Generally the way I do this is by making a test build on a firewire partition. I can boot into this build and tweak it until it's perfect. And when it is, I image it to an ASR disk image for NetBooting. I was worried that architecture limitations would make this painful — that booting into my test build partition would be impossible on my PPC Mac because of these restrictions. Glad to know I can just use the old reliable APM for everything and it'll do what I need.

Not sure when or how they worked this out, or why the language in Disk Utility has gone unchanged. That fact does give me pause. But so far booting Intel Macs from APM partitions has worked perfectly for me on multiple machines.

UPDATE:

More info at Apple's Secrets of the GPT Tech Note, via Jeff in the comments.

Syncing Makes Me Cranky

Someone recently noted that every time I attempt to synchronize some portion of my increasingly expanding digital life — be it Address Book contacts, calendars, mail, or any of the other bits of data that seem to be essential everywhere I go and at all times — I get cranky. It's true. Syncing has become the new Holy Grail for just about anyone who has data on more than one device, which is getting to be lots and lots of people. It's no surprise that my most popular post ever has been, essentially, about syncing calendars. (Publishing, actually, but it's basically a sync issue.) There are also any number of sync-related products on the market these days. But I have yet to hear of any that would meet my syncing needs for any length of time — Mobile Me, for instance, looks great on paper, but sounds disastrous in practice.

A new service, Fruxx (currently in beta), looks promising. But so far I've opted for free alternatives, and, most recently, those have revolved around Google (now that it supports IMAP — more on that later), which is increasingly sync-able with the Macintosh platform, and increasingly present in my digital life.

So why the cranky? Two reasons, really. The first is that I always feel that syncing should be easy. The second is that syncing is never easy. Syncing is, at best, a big fat pain in the ass, and at worst a vector for data loss.

Why is syncing so tricky? An illustrative scenario that springs to mind is the moving of data. Let's say I have 5 GBs of data in a folder called Movies. This folder is located in my home directory. That home directory gets synced to a spare hard drive via Time Machine. Now let's say I move those movies to another folder in my home account called Old-Movies. At last sync I had a 5 GB folder called Movies and no folder called Old-Movies. Now I have an empty folder called Movies and a 5 GB folder called Old-Movies. There are a few ways to sync this data move, and none is completely foolproof. What Time Machine will do is what I call an incremental backup. That is, Time Machine will keep a copy of the 5 GB Movies folder and make a backup of the 5 GB Old-Movies folder, even though their contents are the same. This is safe, but not storage-efficient.

Another model is to mirror the data, in which case the source is adhered to strictly, and anything not on the source is deleted from the target. In this case, the files in Movies would be deleted and the ones in Old-Movies would be copied. This is more storage-efficient, but also more destructive; it does not take into account the need to access previous iterations of the data. And all this gets even more complicated when there are two data stores and each is modified separately. For instance, what if I'm storing, accessing and editing copies of my contacts on my phone, my computer and my Google account all at the same time. How do we keep these data stores properly in sync with one another?

Again, there are numerous models. But one of the best so far — and particularly for small bits of data, like contacts and calendars, but not so much for large data like, say, movie files — is what's being referred to these days as "The Cloud" model. "The Cloud" is really just a marketing term for keeping your data in a centrally located, globally accessible place. In my biz we call these servers. Using servers to store data works well for syncing because the "master" data set is accessible from all of its clients. Make a change on your phone, and that change goes directly to the server. Next time you log on to your computer, it too talks to the server and gets the latest changes.

This is the approach I've been trying to use. These days I am syncing my iCal calendar data to Google Calendar, and my Address Book contact info to Gmail. Google is a good service to use for my stuff because it's free, extremely reliable — it's Google for Christ's sake — and it works with the Mac. But setting all this up is inordinately painful. For calendars I am forced to use a Byzanntine setup process involving login info and insane calendar URLs for each calendar on each computer.

With 10 calendars on 3 computers, this has been annoying, time-consuming and error-prone. Not to mention cranky-inducing. For my Address Book contacts the process was much easier: I just entered my Gmail info into a sync panel in Address Book and I was off to the races. There are problems with this too, though. For some reason I only get the Google Sync option if I've attached an iPhone to the machine in question, which is ironic considering the iPhone is mainly a target, rarely a source, and generally relies on a computer to be the data "master." Also, Address Book and Google only seemed to sync when I synced my iPhone to my computer, greatly limiting the usefulness of this feature. My main Mac, my iPhone and Google are now in sync, but what about my other computers?

But what's most annoying is that all this setup is completely redundant. It's all been done before. All the info I need is already there. In Mail.app.

For email I've long used a protocol called IMAP, which is an example of the concept of centrally located data that's been around for some time. Yup, that's right. We've been doing mail in "The Cloud" for over twenty years now. In fact, I'd venture to say that if we'd had the sort of standards for calendars and contacts that we do for mail, we might not even be having the whole "Cloud" conversation. Calendars and contacts would all be on our mail server and served over well-worn, well-understood, free protocols. Fortunately, we're getting there, and again, by running their services using free and open protocols, Google is leading the charge. And Apple is right behind them with support for standard, open formats like CalDAV and VCF in their applications. But the process needs to be simplified: If Mail.app has my Google user info, that should be all the system needs to set up the other services.

Ideally, I think it would be really neat if someone offered up a service that simplified all of this. Imagine a site or an application that consolidated all — and I mean all — your email, calendars and contacts. I know this is what Mobile Me is going for. Google is also, basically headed in this direction. But Mobile Me can't pull your Google data and vice-versa (or at least not easily). No one is taking into account users' current data sets. Thus far, these systems are still too proprietary. This all needs to work like mail does, only better. Ideally, I could enter a username and password, and the URLs of a few servers and The Service would aggregate everything for me. Just like Mail.app does with all my mail accounts now, The Service would do for contacts and calendars as well.

Of course, then what would I get cranky about?

Don't worry. I'll find something.

Addendum: Before I even had a chance to post this, Khoi Vinh, of Subtraction fame, posted some tangential thoughts on cloud computing in a financially unstable world. I found the following passage particularly relevant to at least part of my argument:

"...I pay a company to store my mail on the cloud, ostensibly, but I have no more or less trust in their longevity than I do in any other online business. Through the underestimated miracle and beauty of IMAP though, all of my messages are also mirrored on my hard drive, an invaluable insurance policy against the sudden disappearance of my mail server or host provider. To me, the IMAP approach is really the ideal approach, and I really wish it was a model for more Web services."

See? I told you so.

Khoi's article is less about syncing and more about data storage and archiving, but it's an interesting read if you're interested in where, for instance, all your Google docs will go when the company crashes under the weight of the current financial crisis, and other drawbacks to so-called "Cloud Computing."