Three Platforms, One Server Part 7: Testing...

So, our primary authentication server, which will be used to enable network home accounts for our entire internal network — Mac, Windows and Linux — is up and running. I've installed it in our server room, put it on the KVM, and switched over all the workstations. So far, so good. (My fingers are so crossed they hurt.)

For the Windows quota requirements we went with the first solution mentioned in this article, and outlined in detail here: Windows roaming profiles are on a separate, quota-enabled volume on the authentication server. This — and the Windows implementation in general — is my biggest concern, and will require a great deal of testing. Summer Session has begun, and this will give us the opportunity to test on live subjects as a limited pool of students begins using the systems for work again.

There are a couple of immediate benefits of this new system. For one, users can now change their password lab-wide from any Mac in the lab. Now, users don't tend to do this much, but they can if they want to, and it's easy as pie. But as an admin, the most exciting benefit to all this is the ease with which new users can now be created. In the past, creating a new user was a multi-step, multi-computer process that involved coordination between multiple sysadmins: Admin 1 gets the user info, creates the Linux and Windows accounts and hands it off to admin 2, who then creates the Mac account and uploads the Mac skel. This process was lengthy and extremely error-prone. Using the new system, user creation is a breeze. One single, easy-to-use script on the authentication server pretty much does it all. Any sysadmin — or even a trained assistant — can do it. Just log in to the authentication server, run the script, and you're basically done. It's fast, easy and accurate. Which is really the whole reason we're doing all this. I'm pretty happy about it.

There are a few minor details I need to work out. The main one is file sharing. In addition to being an authentication server, our original Mac server is also a file server. On it there are numerous shares for various purposes — one for staff, one for students, etc.. At this point I'm thinking of keeping the file server and the authentication server separate. This will reduce the load on both, and mitigate the need to migrate any data. The catch is that user authentication data on the file server will need to match that of the new authentication server. This should be a simple matter of changing the old server's role to "Connected to a Directory System" and pointing it at the new authentication server. I've never done this though, and it's complicated by the fact that the old server is running Panther whereas the new one is running Tiger. Probably the best thing to do will be to wipe the old server (yes, I have a backup) and put Tiger on it, then redo the shares. But I'll probably test with Panther first, just to see what happens. I'll let you know.

In any case, this is, again, a minor problem that shouldn't affect the overall plan much and should be relatively easy to figure out and implement. It's just one of those little things you forget about when you're drawing up your master plan for world conquest. "Oh yeah... What about the file server?..."

Otherwise, the conversion is on and seems to be going smoothly so far. Once we get into serious testing I'll post the results. Hopefully this will all be done in the next few weeks and the conversion will finally be complete.

Three Platforms, One Server Part 6: More Windows Quota Problems

Of course I knew it was too good to be true. I've found the first fatal flaw in my plan to unify authentication on the internal network. It goes back to the Windows quotas problem I studied some time ago, and to which I'd thought I'd found a solution.

I won't go into great detail about the problems and various solutions to the Windows roaming profile issue. I've already written plenty on it, and the previous posts outline it all fairly well. I will say that the intended solution was to provide Windows roaming profile quotas by setting them locally on each workstation. But, last week, as we moved forward with the plan, one of my fellow sysadmins, who is far more capable with Windows than I happen to be, pointed out the fact that certain applications (i.e. Photoshop, Maya, etc.) need a certain amount of disk space for temp files and what-not in order to operate. Setting small local quotas effectively keeps these applications from running properly.

We are currently testing a few other scenarios in which quotas for Windows roaming profiles can be implemented to our satisfaction:

1. The Authentication Server (Mac OS X Server 10.4.6) has a separate volume for Windows roaming profile storage and that drive has quotas enabled (this was the original plan). The drawback to this is that the user's home account data is stored separately for Mac and Linux — which keep this data on the home account server — than for Windows, which would store this data on a reserved volume. The other drawback is the fact that the client machine is not aware of the quotas until the user logs out and the Windows client attempts to upload the new data, at which point Windows issues an error if the user has exceeded his quota. But the user is not warned about quota violations until logout, and this could cause some minor problems.
2. The Windows Server continues to host roaming profiles and determine quotas for Windows users, but gets user authentication information through a connection to our Mac Authentication Server. The drawback here is that we have to continue using the Windows Server, which we don't really want to do, though it does seem to give us a slightly higher level of control than does the Mac Server. This method is also complicated by the fact that we are currently running Windows Server 2000, which does not include native authentication to LDAP, so third-party solutions would be required. This method could also complicate user creation.
3. Through some combination of Windows and Mac Servers, we convince the Windows roaming profiles to be situated on the Temp volume of the local workstations, rather than the default location on the "C" drive in the "Documents and Settings" folder, and then set quotas for the Temp volume. I'm skeptical that this is even possible. Windows seems to be hard-coded in ways that make specifying the location of roaming profiles anywhere other than "C:\Documents and Settings" impossible. So this last option seems the least likely to succeed, though if it did it would match the way Mac and Linux behave much more closely. And if we could figure out how to do this without the Windows server, it would be almost ideal.

(Actually, I just thought of a problem with this last method: The Windows Temp drives get erased every Friday. If users happen to be working during the deletion period, what happens to their roaming profiles? The same thing happens on the Mac, but the deletion script on Mac does not delete work owned by the currently logged in user. Can such a scenario be implemented on Windows?)

Most likely we will go with the first solution. We already know that it works. It's a little extra effort when creating new users, but that's totally scriptable. We plan to do user creation from a single script from now on anyway, so these extra few steps, once incorporated into our user creation script, won't really be a big deal at all. The only other problem with this is that our replica will now need to sync the Windows roaming profile volume as well if it's to work as a proper fallback system. This, too, should not be terribly difficult to accomplish. Overall, this solution is less elegant than the original one, but it should be workable. Hopefully, Windows Vista will mitigate a lot of these problems. (Yes, that was totally a joke. Please chuckle softly to yourself and move on.)

I guess what amazes me still is how contrary to other operating systems the Windows OS is. Everything we're doing here can be done the same way on Mac as it can on Linux, or virtually any other *NIX system: home accounts can be read directly from a network disk, their locations can be specified, and therefore, all this quota nonsense is unnecessary. On Windows, roaming profiles apparently must be downloaded to the client machine (an unbelievably stupid requirement), and the location of said profiles apparently must always be on the root drive in the "Documents and Settings" folder. I guess these are the ways in which Windows continues to force people to use Microsoft products (I can almost hear Bill Gates whispering in my ear, "Wouldn't it all be easier if you just used Windows Server?") But for software that's become the dominant standard in both the business and personal markets, Windows sure seems non-standard in baffling and infuriating ways. Though this may be how Microsoft has managed to stay on top all these years, I still, perhaps naively, believe that some day, if they don't change this strategy, it will hurt them. Frankly, though I'm sure they're out there, I don't know a single sysadmin that likes Windows. Can you blame them?

Three Platforms, One Server Part 5: Away We Go!

So it's the last week during which students have access to the lab, and that means I can finally implement my plan to unify internal network user authentication. Finally! I'm so jazzed. I've been waiting for months (well, years, really) for the chance to do this, and it's here at last.

The general outline of what I'll be doing over the next few days goes something like this:

• Backup my current Mac server (for safety)
• Build my master authentication server
• Backup a clone of the clean server install
• Configure the new server with:
• Users and Groups
• Home account automounting
• Home account sharing to SMB (for Windows Roaming Profiles)
• A skel account for Windows users (to live on the home account server)
• Other share points
• Create a replica of the new master server

What I did today:
Well, it's amazing how long a base install of Tiger Server can take. I've pretty much been doing that all day. Not that I'm so incompetent that I can't install the software in seconds flat, but software updates take forever and a day. Planning and getting drives to do all this on was a bit of an effort too. Plus I just wanted to make sure I did it right the first time, so I went slow, gave myself the day. I'm also making clones of everything along the way, for building my replica, and in case I goof and need to start over. So that takes a while. I guess I'm just saying that I'm taking my time with this, 'cause I want it to be as perfect as possible from the get-go.

By Monday we should have:

• A base install of Tiger Server 10.4.6 with requisite Software Updates on a firewire drive
• A backup of our old Mac Server
• A new Tiger 10.4.6 authentication server that's configured to host Mac, Windows and Linux users
• A replica of same

We will spend part of next week pointing all our workstations at the new server. The Windows machines will be the biggest pain as 1) they are running Windows, and 2) they need local quotas set (which could really be just a subset of point 1, but whatever). The reason for all this quota nonsense, you ask? Well, for the answer, you'll just have to read the previous posts on the matter. Suffice to say, I'm hoping the quota setting nonsense will be the worst part of this job, which it should if all goes according to plan, which, I'm sure you're aware, it rarely does.

Finally, I wanted to mention this quote that I read on Daring Fireball, by someone called John Gall, author of Systemantics, as it really jives with a lot of the stuff I've been thinking about with regards to the lab:

“A complex system that works is invariably found to have evolved from a simple system that worked….A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over, beginning with a working simple system.”
— John Gall

Next week should be interesting. I'll keep you posted.

External Network Unification Part 2: CMS LDAP Connections

So I've been examining what we have, and thinking about what we want, and thinking about how to get there with regards to external network unification.

Here's what we have:

• A mail server running FreeBSD and getting user info from it's own, local DB
• A web and FTP server running same, getting user info (I believe) from the mail server
• A community site running the Mambo CMS, running on the same BSD machine as the web/FTP server, getting its user data from MySQL
• A custom-built online computer reservations system, also running on the web/FTP server, getting its user data from a second MySQL database
• A Quicktime Streaming Server running Mac OSX Server, getting user info from the local NetInfo database

Here's what we want:

• An LDAP server with all user information
• A mail server running FreeBSD, getting user info from LDAP
• A web and FTP server running same, getting user info from LDAP
• A community site running Mambo (or similar) getting user info from LDAP
• A custom-built (or prefab, if available) online computer reservations system getting user info from LDAP
• A Quicktime Streaming Server running Mac OSX Server, getting user info from LDAP

Are you sensing a pattern? Did you notice how much easier the second list is to read and understand? Boy I sure did. Extrapolate.

So, porting some of these systems — particularly the BSD machines that rely on local databases of users — shouldn't be too bad: build the LDAP server, point the BSD boxes at it, and, bam! we're done. I'm almost not worried about those. They're standard *NIX boxes, and LDAP support is built in and fairly easy to set up, at least in terms of getting user data. Same with the Quicktime Server: Mac OS X has stupid-simple support for authenticating to LDAP, and there's tons of good documentation on the subject. So I've been concentrating on our web apps, which promise to be much tougher, and recently I had what I think will turn out to be a real breakthrough.

When last I wrote about this topic I was experimenting with setting up my first FreeBSD server, and also with some simple PHP/MySQL-driven web apps that purported to authenticate against LDAP. The one I finally got to work was a freebie called MRBS. MRBS is great. We may even modify it and use it for certain staff-centered scheduling tasks. It's great that it works with LDAP, and it's pretty easy to get set up, after some trial and error and help from AFP548. It's given me a way to go with certain other proposed web-apps in the future. And, most importantly, it's allowed me to demonstrate proof-of-concept. But if MRBS is our future, what about our present?

We have a whole lot of time and effort invested in our current Mambo site. Not so much that it would kill us to move to a new system, but enough so that moving would be painful, and we'd better have a plan and damn good reasons to do so before making the attempt. So for the past however many weeks now, I've been building and testing a multitude of CMS systems. In doing so, I've been primarily concerned with two things: 1) Will this system authenticate to LDAP? 2) Does it have all the functionality (or more) that we currently enjoy on our Mambo site?

I figured the easiest thing to do — and a good place to start — would be to get our current Mambo site to work with LDAP. This would save us the trouble of setting up and learning a whole new system and porting over all our content — again, not the end of the world, but not exactly desirable either. Turns out there is an LDAP hack available for Mambo, but the hack is only supported under older versions of Mambo. I tried installing every version of Mambo I could, and every version of the hack, and every combination of these, as well as hacks to the hack I'd found in forums. No luck. I simply could not get the Mambo LDAP hack to work.

It was at this point I began to turn my attention to other CMSes that might support LDAP. After hunting around I stumbled upon Plone, which looked like a worthy contender, and which supposedly supported LDAP authentication. The thing I liked about Plone from the get-go was that it is ported to Mac OS X, which is what I'm testing all this on, so installation was a breeze. Plone even installs in its own folder in /Applications, and it's here that, somehow, the Plone site root lives. The system itself is very nicely structured as well. The interface is clean and easy to understand, and even fairly easy to modify, in minor cosmetic ways. But getting Plone to authenticate to LDAP turned out to be a little scary and labor intensive for my tastes. Plone runs on Python and MySQL (as opposed to Mambo's PHP/MySQL engine), so Python is responsible for making calls to LDAP. According to the LDAP module READ ME, LDAP authentication in Plone requires the python-ldap module to be installed. Installing this looked to be a pain, and no one in my organization (myself included) knows the first thing about Python, so it was at this point I bailed and began to start thinking about another approach. So much for Plone.

The next system I tried was Drupal. Drupal was also supposed to have LDAP support, though I never got around to really looking into it. I really liked Drupal: It's fast and simple, and the interface is sharp and clean. And Drupal has great user management with support for custom roles and permissions. But Drupal doesn't come with much out of the box, and I never really got around to figuring out how to install additional components. In fact, though I guess you could install one, Drupal does not come with a WYSIWYG HTML editor, which is one of the main reasons we're using a CMS in the first place. So I moved on despite some of the really nice things I saw in Drupal.

Some time later I was talking to a fellow sysadmin about all this, and he said, "What about Joomla?" and I said, "I thought that cost money." and he said, "No, it's the new Mambo." And I thought, "Hmmm... The new Mambo, eh?..."

Needless to say, the next day I'd installed my first Joomla install. I liked what I saw. It has a very similar look and feel to Mambo, particularly on the back-end. In fact, it's almost exactly the same because Joomla is developed by the former developers of Mambo. I'm not (yet) sure what went down, but apparently the bulk of the Mambo team jumped ship and began their own, separate CMS project. So Joomla really is the new Mambo.

Joomla also claimed to support LDAP, and according to their documentation, LDAP would be built in to the next release. This is apparently true, as Joomla 1.1 Alpha includes a built-in LDAP plugin. I installed the Beta and gave it a whirl, but no joy. I couldn't get the Joomla beta to authenticate to LDAP. Reading around some more led me to a new variant of the Mambo LDAP hack that's made to work with the latest stable version of Joomla, version 1.0.8. I also read what I believe was a comment in the Joomla forums from the originator of that hack who swears allegiance to the Joomla team, which probably explains why there are no new versions for Mambo.

Last week I installed Joomla 1.0.8 and the ported LDAP hack for Joomla 1.0.8 and guess what? After weeks of scrounging and searching and hoping and praying and cursing and installing CMS after CMS, it worked!

It fucking god damn worked.

This is great. Not only was installing the hack easy as pie, but setting up the LDAP authentication — for the first time since I dug in on this — was a breeze and worked completely as I'd hoped and expected it to. Not only that, but migrating our Mambo site to Joomla should be a fairly easy task since Joomla is built on the Mambo core. The Joomla site even provides instructions on how to do this, and they don't sound terribly difficult at all. The bonus is that the built-in Joomla LDAP authentication looks promising and, down the line, will hopefully eliminate the need for a "hacked" solution. But until then, the hack works great for our purposes.

This is a huge milestone for the External Network Unification project. Getting our CMS — really, our most complex web application — to work with LDAP was one of my biggest concerns. Going with Joomla gives us the LDAP stuff we need, maintains consistent usability on both the front- and back-ends, makes migrating a whole mess easier, and provides good scalability in terms of development and support for the future. Joomla's developer team appears to be solid, the third-party developer community seems very active, and the LDAP support looks to be headed in the right direction and available in the near term. While it's by no means a done deal, this looks very promising.

Next on the list:

• Getting our custom computer reservations system to work with LDAP (or finding/building a replacement)
• Learning and building an LDAP server on FreeBSD (not Mac OSX)

I'm going in. Wish me luck.

Boot Camp and Partitioning: Cooler than You Think?

Today Apple released a beta software package called Boot Camp to facilitate running Windows XP on an Intel-based Mac. I'm practically speechless. Which doesn't happen too often.

Apple's Boot Camp Beta: Brilliant!
(click for larger view)

Now we all know a few weeks ago this nut was already cracked by some enterprising young hackers, so we all knew it was possible, and in fact probable that installing and booting Windows on Mac would become a fairly trivial and commonplace happening. There were even rumors that the next version of the Mac OS would provide "virtualization" software for running Windows in an emulated environment from within Mac OSX. But I have to admit I was quite taken aback by today's news. Official support from Apple for installing and running Windows on the Mac.

Brilliant!

Dual Booting
(click for larger view)

One of the best things about all this is the fact that Apple is providing Windows drivers for Mac-specific hardware. So you no longer need to search the internet to find them. But to me, coolest thing is the portion of the assistant that creates the partitions. Here, for the first time in Mac OS history (at least that I'm aware of), we see an Apple-branded utility that will partition your drive without erasing it. It's uber-smart of Apple to provide this functionality for people who want to test this out without the inconvenience of a full system reinstall. But it also has immensely cool implications if partitions are as important to you as they are to me, or if you happen to use an Xserve RAID. I'll explain.

Space Maker: The Caption Says it All
(click for larger view)

I use partitions on my systems to wall-off user data from system data. So I have one partition with all the system components and applications. Root. The boot drive. I try to keep this drive to as reasonable a size as I think I'll ever need, and then create a second partition with my user data: projects, mail, videos, music — everything else, essentially. This partitioning scheme works great, but it breaks down when someone like Apple decides to release a suite of applications that suddenly takes up eight times more space than the previous version. All of sudden, I no longer have enough space on my root drive for all of my applications. A partitioning system like mine fails when I am unable to accurately predict the amount of space all my apps will require, both now and in the future. And this is becoming increasingly common. A utility that would let me dynamically resize my partitions would be a godsend. Now I don't think the partitioner included in the Boot Camp Assistant does dynamic partition resizing, and it probably never will, and I'll just have to roll with it. But I can dream. And there is another area in which a utility like the one in Boot Camp would have immediate benefits.

I have a client who purchased on of Apple's Xserve RAID systems. This is a pretty cool beast. Fast. Stable. Very reliable. But there's a scalability problem, and it's one reason I won't buy one for my lab. When you read the datasheets and the manuals for the XRAID, you're told that drives can be dynamically added to the RAID in the future, should you ever require more storage. Add the new drive and the RAID will, theoretically, add it to the RAID set and you'll have that much more storage space. This is typical of enterprise-level RAIDs I'm told. It's one of the reasons you get them: scalability. But if you ever try to actually do this on an XRAID you'll run into a little snag. You see, while it is possible to add a drive to the RAID system and have the RAID recognize the new drive and add it to the set, the Mac OS host is responsible for partitioning the RAID. So what happens is, you add the drive, the XRAID sees it and adds it to the set automagically, but the Mac does not recognize the additional capacity. The Mac still sees the original partitioning scheme. I've read through the manuals and discussion forums — hell, I even talked to one of Apple's RAID specialists — and it turns out there is no way to grow the XRAID without repartitioning it from the host Mac. And as you're well aware, I'm sure, there's no way to repartition a Mac drive without wiping it. So, you want to grow your XRAID? Well, you'll have to wipe it. And that just sucks.

Boot Camp on PPC: No Boot Camp for You!
(click for larger view)

Apple's RAID specialist told me that they were working on a solution to this. That was about a year ago, and I've heard nothing since. But the partitioning utility in the Boot Camp Beta gives me great hope that a solution is indeed imminent. It may even be here already, if you have an Intel Mac that is. I've tried it, and no, the Boot Camp Beta will not run on a PPC Mac. No surprise there. But if the information on the Boot Camp page is any indication (Apple calls "Space Maker" the "most elegant hard drive utility ever"), a new and improved partitioning utility is on the horizon for the Mac. And I'm just as excited about this as I am at the prospect of booting Windows on my Mac. Maybe more.

It's been a long time coming.

UPDATE:
I'm told by a colleague that growing a filesystem (like on a RAID) and partitioning a filesystem (what the Boot Camp Assistant does) are two different things, growing being the harder and, according to him, requiring a grow-able filesystem, which HFS+ is not. So maybe this is a pipe dream.

Apple has a FAQ up about the Windows installation process for Intel Macs, and there are some interesting things there. Most notable in the context of this article is the fact that Boot Camp Assistant will only work on "an Intel-based Mac that has one hard disk partition." This does not bode well. Still, I remain hopeful that this utility is indicative of possible future utilities that would allow for more flexible, less destruct ive partitioning.

UPDATE 2:
I've been doing some hunting around on Apple's RAID capabilities and discovered that you can now (as of Tiger, I believe) use Disk Utility to create what called a Concatenated Disk Set (also referred to as JBOD, for Just a Bunch of Old Drives). These sets can be combined with other forms of RAIDs as well. Until yesterday I'd never even heard of Concatenated RAID, but apparently it's the oldest form of RAID. Concatenated RAID doesn't have the redundancy advantages of RAID 0 or the performance advantages of RAID 1. All Concatenated gives you is more contiguous disk space. I've read that Concatenated could be used to append a new drive on an XServe to the RAID set, though all it would do would be to add to the set's capacity and would not be a true member of the set. This is recommended as a stop-gap measure more than anything, and it's recommended that the reformat necessary to truly join the new disk to the set be performed later, at a more convenient time. Interestingly, this seems to suggest that a Concatenated RAID can be performed without wiping the disks in question, though I can't find any documentation that states this one way or the other. Also, the disks in a Concatenated RAIDs do not have to be identical, as with other forms of RAID. Concatenated RAID disks can be all different sizes, shapes and colors. As soon as I can find some drive to offer up as sacrifice, I will be testing out Concatenated RAID. I'll let you know what I find.

UPDATE 3:
It appears that a new verb for the diskutil command has been added in 10.4.6. The verb is resizeVolume and it's what allows the non-destructive partitioning done by the Boot Camp Assistant. Don't look for it in the man pages; it's not mentioned there. But if you just run the diskutil programsan arguments you'll see resizeVolume listed as an option. I first read about this on Daring Fireball, which links to a MacGeekery article that outlines, in-depth, the process of using the new option to non-destructively resize volumes. I don't know yet if this works on PPC Macs — the article suggests it's for Intel machines only. But I'm sure going to give it a whirl. I'll post a follow-up on all my partitioning experiments in the near future. Stay tuned!