Three Platforms, One Server Part 1: Planning, Testing, Building

Background
As I've said probably a thousand-billion times, I work in a very multi-platform environment. Our lab consists of Mac, Windows and Linux workstations, all of which rely, in some form or another, on a centralized home account server. Currently, the way this works is problematic in several ways: Each platform -- Mac, Windows, Linux -- authenticates to a different server, and then mounts the home account server via one of two methods, either NFS or SMB. The main problems arise when we want to create new users, or when a user wants to change his or her password. When creating a user, we must do so on each platform's respective server -- i.e. on the Mac server, the Windows server and the NIS server which hosts Linux accounts. The user's short name, UID, GID and password must match across all three servers. Then, if a user later wants to change his password, he must do so on each platform separately or the passwords will not match. There is also an inconsistency in the way the actual home account location appears and is accessed on the various platforms. On Mac and Linux, the home account directories are mounted at /home, but on Windows, the home account is mounted at a location that is separate from the users' roaming profile, adding another layer of confusion for any user who might use both Windows and any other platform.

These problems make user creation painful and error-prone, and discourage regular password changes for the community. Also, educating the community about this system of servers is next to impossible. It's a bad, old system that's long overdue for an overhaul. And that's just what I intend to do.

The Plan
The plan for this overhaul is to migrate all user accounts to one server which will store all user information -- UID, GID, password, name, everything -- and which will also share out the home account server in as consistent a manner as possible. I've chosen Mac OSX Server for this job because it integrates both LDAP for Mac and Linux and Active Directory for Windows into its Open Directory system, and because it's capable of re-sharing an NFS mount out to each platform in a relatively consistent manner appropriate to said platform. There are three major hurdles to overcome in doing this.

The first problem to overcome is building and setting up the new server to host Windows accounts in an acceptable manner. Windows is the real wildcard here. The problem child. Setting up Linux clients is as simple as flipping a switch. Set the Linux client to get authentication info from the Mac Server and you're done. Linux sees the LDAP data just like Mac does and can mount our NFS server directly in /home. No problem. Standard *NIX stuff. Same as it is on Mac. Windows, on the other hand, understands neither LDAP nor NFS natively. So we need to make special arrangements on our Mac Server to accommodate it.

The second hurdle is getting users migrated from all three platforms over to the Mac Server. Here, two complications arise. The first is that my existing Mac Server has never hosted Windows accounts, and therefore, is not capable of doing so using the existing database. So for all intents and purposes, the user database needs to be rebuilt from scratch, which brings us to the second complication: We need to reset everyone's password. What do we reset the passwords to?

The final major hurdle we face is that of redundancy. Since we're now moving all users to a single database, every system is now dependent upon a single server. A single point of failure. We suddenly find all our eggs in one basket, and this is potentially dangerous. If that server goes down, no user can login in and work, and we're dead in the water. So we need to figure out a method for keeping a live backup of our server that can take over if it goes down.

These are the major hurdles, and there are some minor ones along the way, which we'll talk about as we get to them. So let's start building that server. Our authentication server. The mother of all servers.

Building the Server
I started fresh with a Tiger server. Nice clean install. I set it up to host Mac accounts, as usual, using the standard LDAP stuff that's built-in, as I always do. Master server, and all of that. But this time I've also set the server up as a Primary Domain Controller (PDC) so that it can also host Windows accounts. Getting the server to host Windows was not particularly difficult, but getting it to host them on a re-shared NFS mount, and having the Windows roaming profiles live on that NFS share was a bit more tricky. To do this, I first mounted our NFS export on the Mac Server at /home. I then set /home/username to be the location for our Windows roaming profiles. This should theoretically work. But the Windows client wants to copy everything in the roaming profile directory to the local computer at login, and then copy it all back at logout. This is a problem for any user who might have more than a few megabytes of data in his home account as the time it takes to do the copy is painfully long. Also, Widows ran into permissions problems when it hit the Spotlight directories, and errored out. So I've created a special directory in the home accounts, called Windows, that only holds the roaming profile data, and set the server to have Windows clients look here for said data. This initially failed to work until I realized that roaming profile data must actually exist for Windows clients to understand what's going on. Once I copied some basic roaming profile data (skel data, essentially) to the user's Windows folder, the Windows clients were able to login using this folder for the roaming profile. I'm almost surprised that this worked as we're essentially re-sharing an NFS mount out as SMB from a Mac Server to Windows clients. But work it did, and so far, at least, I've encountered no problems here. Windows users will now use the same home account location as Mac and Linux users. This is extremely cool.

So, on the Mac server we've mounted our home account RAID at /home, and in the Windows section of the user account, we have a roaming profile path that looks something like this:
\\ServerName\home\UserName\Windows

And in that Windows directory we have some basic Windows skel data. And that's pretty much all we need to host our Windows accounts.

Migrating Users
Next we need a plan. How will users access this new system? We'll have to reset their passwords. To what? And how do we inform them of the change? And how do we actually migrate the users from the old system to the new?

To answer the last question first, I decided that I wanted to start fresh. No actual user migration would occur. I want to build the new user database from scratch, since we'll be needing to reset all passwords anyway, and just in case there are any inconsistencies with our old server's database. Which brings us to the how of this, and that will answer the question of password resets as well. What I decided to do was to create a record descriptor -- a text file that can be imported into the Open Directory database on the Mac Server -- using easily obtained and formatted data. All I really need to do this is a list of users, their long names, their short names, their UIDs and GIDs, and their passwords. Since we're resetting their passwords, we can actually use whatever we want for this value. But it makes the most sense to use, for the reset passwords, something that is already in the possession of the users. This way, disseminating passwords is far less of a chore. Each user is given an ID card by the school when they start . This ID card has a number on it which we can use for the new passwords. Building this record descriptor is a fairly simple matter of obtaining the aforementioned data and tweaking it a bit. This can be done using a combination of a basic text editor, like TextEdit, and something like Excel for properly formatting the data. I will leave the details of this process for another exercise. Suffice to say, we have a clean slate upon which to build our user database, and a good method for reassigning passwords. So this step is effectively done.

When it comes time to implement the new system, users will be informed of the password change via email, and signs posted throughout the facility. And if they want to change their passwords after that, it will be as simple as logging onto a Mac and changing the password in the Accounts System Preference pane.

Redundancy
Finally, we have the problem of redundancy, and this is fairly crucial. Having all user data on a single machine is far more convenient, but also has certain risks associated with it. First and foremost among these is the idea of a single point of failure. That is, if our user account server -- our Mac Server, which in our model is soon to be responsible for user authentication for every workstation in the lab -- fails, no user can log in, and therefore, no user can work. Classes cannot happen. We're screwed. Hard. Ideally, we'd have a second server -- a replica, if you will -- of our primary Mac Server. If our primary went down, the replica would automatically step in, and take over the task of authenticating users until the primary is running again. Well, such a thing is indeed quite possible, and is also built in to Mac OSX Server. It will be crucial for us to set this up and make sure it works before we implement our plan.

What's Next
So, this is where we are at this point. I've built a model Mac Server. I've begun compiling the user data into a record descriptor (actually, all students have been added, but I need faculty and staff data to complete the job). I've tested the behavior on test Linux and Windows machines. The next steps will be: more thorough testing of the above, setting up and testing the server replica behavior, and, finally, alerting users to and implementing the new system. These will be covered in upcoming articles. If you're interested in such things, do check back. I hope to have all this completed by the end of the Christmas break. But then, you never know what sort of problems will crop up in a move like this, so we'll see.

Either way, I'll be documenting my experiences here. More soon to come.

Command-Line Smackdown: cp vs. ditto

There are a couple methods for copying files via the command-line in OSX. The standard UNIX cp command exists, as it does on all *NIX systems. Historically -- that is, through Mac OS X 10.3 -- it's had certain limitations with regards to the HFS+ filesystem, however. Specifically, the cp command was never able to handle resource forks common to that filesystem. Any files copied via cp would have their resource forks stripped, which in some cases would render the files useless. The solution was to use a Mac-specific command called ditto. When supplied with the proper arguments, ditto can copy files and retain their resource forks. It's a great tool, with a lot of other features, and I use it all the time.

In Mac OSX 10.4 (Tiger), cp -- and most other UNIX commands -- have been enhanced to remedy the HFS+ resource restrictions. Using cp to copy files in Tiger (or between two systems running Tiger) will handle resource forks quite seamlessly, and the command works just as it would on any *NIX system. So I've been trying to revert to using cp like a regular UNIX fellow, but I still find myself preferring ditto, for a few reasons.

The ditto command has a few idiosyncrasies that anyone who's coming from traditional UNIX and is used to cp might find distressing. Indeed, I initially found them to be confusing. The main difference lies in how each command specifies the copy behavior for files in directories. The cp command uses a trailing slash to indicate that the contents of the directory -- but not the directory itself -- are to be copied to the specified location. So a command like this:

cp -R /Users/systemsboy/ /Users/test

will copy the files inside the systemsboy directory to the test directory. Whereas:

cp -R /Users/systemsboy /Users/test

will copy the directory systemsboy and its contents to the test directory.

This is a nice way to do things. It's how most *NIX commands work, and to my mind makes a lot of sense. But it is not how ditto works. The ditto command doesn't care about trailing slashes at all. The ditto command always copies the contents of a directory. So, following the example above:

ditto -rsrc /Users/systemsboy/ /Users/test

will copy the contents of systemsboy to test. And so will:

ditto -rsrc /Users/systemsboy /Users/test

I repeat, ditto does not give a shit about the trailing slash, and always copies the contents only. The two above commands, therefore, function the same, trailing slash or no. So, if you want to copy the entire systemsboy directory to the test directory, you must say so thusly:

ditto -rsrc /Users/systemsboy /Users/test/systemboy

Here you are specifying a directory that does not exist. If you tried to do that with cp you'd get an error message stating that the directory didn't exist. One nice thing about ditto is that it will create the directory for you. There are other niceties as well.

The ditto command, by default, preserves mode, access time, permissions and all other file attributes. If you want these functionalities with cp, you'll need to specify them with the -p (preserve) flag. The ditto command also has other uses. For instance, ditto can be used to create a .zip file of a directory that is comparable to those created in the Finder.

So, despite the fact that the latest versions of standard UNIX commands included with 10.4 are resource-fork aware and can finally be used as they would on any *NIX system, I've become accustomed to using commands included by Apple to mitigate the previous lack of support of the standard UNIX ones. The ditto command still has a very useful and important place in my admin toolbox, and I continue to use it in favor of cp in many situations. It's tried, it's true, and I trust it. And it will work with previous versions of the Mac OS.

A Tale of Bad Media

I needed to pull a bunch of video projects from my DVD archives this weekend. These are regular old DV-NTSC Quicktime movies, made from Final Cut Pro. But attempting to copy them from the Finder in Mac OSX 10.4.2, I noticed the estimated copy time seemed inordinately long: 45 minutes for 3.5 GB? I can burn a disc faster than that. Then, about halfway through, the copy operation choked, and I got an error message that said that the copy operation couldn't finish because some of the data couldn't be read.


Copy Error
(click for larger view)

Then I started scrolling through the files on the DVD, and as I touched them the files started disappearing before my eyes! Folder names also disappeared, but the folders were still there, but no longer navigable. Doing an ls in the Terminal showed everything intact. Copying the files using cp or ditto yielded complete and usable files.


Disappearing Folders
(click for larger view)

After grabbing my data using the Terminal -- which not only worked, but worked quickly -- I decided to relaunch the Finder, and try copying one of the problematic folders. The folder appeared to copy, but the contents of the folder were incomplete. Only two of the thirteen original movies made it over in the copy. But this time, I received no error message. Then I tried copying individual files from the problem folder on the DVD. Some appeared to copy super-fast, and then just disappeared, while some copied fine, and at apparently normal speed. But the time estimation for some took a while, and for others, barber-poled indefinitely. In fact, the only way to stop the Indefinite-Barber-Pole-of-Death was to relaunch the Finder.


Indefinite Barber-Pole
(click for larger view)

Fucked up.

More tests: I tried another DVD. Copying that data worked perfectly: fast and error-free. I also put the problem DVD in my PowerBook running Panther and tried copying the data. And guess what? Same problem. Better error message though. And no disappearing files and folders.


Panther Copy Error
(click for larger view)

So clearly, something is wrong with my DVD. But it seems really strange to me that, while a simple issue of cp or ditto works fine, if the Finder encounters the slightest disc error, it just gives up. This raises some questions: If the command-line can handle it, and the Finder can't, what is the Finder doing wrong? Moreover, why doesn't the Finder use cp or ditto? Or something that can handle these sorts of errors? Or, at the very least, why doesn't the error message offer some sort of useful information about what went wrong and how, via Terminal, one might go about extracting the data?

Finally, and I took a look at the DVD surface itself, and I have no idea how this happened, but it was scratched to shit. Looked like a Blockbuster rental. Awful. Anyway, the moral of the story? I don't know, actually. I guess one moral would be, "Take good care of your optical media; it is by no means indestructible." Moral two? If the Finder can't copy it, the command-line, perhaps, can.

It pays to learn some UNIX kids. Without it, I'd be hard screwed.

Windows and Linux Dual Boot Part 4: Dell Laptop Fun

So I spent much of this past weekend installing both Windows and Linux on my Dell, who I shall hereby refer to by his rightful, and possibly permanent name, satan. Many mistakes, and much learning in the process. Hooray for learning.

My goal was to have one partition with the original Dell Windows system install, a second partition with Linux, and a third shared data partition. Initially, I screwed up during the partitioning. Setting up my partition scheme (10 GB, 10 GB, 15 GB), I did an NTFS format on the first 10 GB partition, and then a "quick format" on the second partition. Well, I didn't know this, but doing a "quick format" causes the Dell restore disc from which I was booted, to begin the system installation process directly after the "quick format," and onto the quickly formatted partition. So I ended up with Windows on my "D" drive. Oops!

Thinking this might be okay, I tried installing Linux, then, on the first partition, but it wouldn't work. It'd complain during the Fedora partitioning process. I don't really understand why. Perhaps someone here can enlighten me.

So, it was back to the drawing board.

Booted off the Windows CD again, repartitioned -- this time I did 5 GB for Windows, and 5 GB for Linux, leaving about 25 GB for shared data. This seems to have worked, and I now have a functional Windows/Linux dual boot-machine. I realized during the Linux install, though, that I'd really like a bigger partition for my Linux install. After cutting out a lot of cruft (and even some non-cruft I would've liked to have had) with the custom install, I was able to get it down to a size that will fit on the 3 GB of space I have left over after the 2 GB swap partition has been designated. But that only leaves me with 737 MB of free space, which really isn't enough to do much of anything, and it might be nice to, someday, you know, install some software.

So it will be back to the drawing board. Again. Soon.

But before I reinstall everything, I have some NFS tests I need to do -- which is why I started all of this when I did in the first place -- because of a problem I'm having with NFS home accounts and my Tiger workstations. When I do finally reinstall the whole kit and kaboodle, that will have made three times I've created my dual-boot system on satan. And hey, isn't the third time supposed to be a charm?

So what have we learned today kids?
1) Doing a "quick format" immediately installs the Windows OS on the "quick formatted" partition.
2) When setting up partitions, don't forget to account for Linux's swap space.
3) I didn't mention this earlier in the article, but I also learned that all the Dell drivers specific to my machine are included on a second CD, and each of those drivers must be installed individually, from long, idiotic Windows installers ("wizards" -- please!), so it's best to get this whole partitioning scheme thing nailed down before really getting Windows back up to snuff.

That said, can anyone out there recommend a good Windows-Linux-Data partition scheme? I'm thinking, at this point, 6 GB (WIndows XP), 10 GB (Linux FC3) and the remainder (approximately 20 GB) for shared data between Linux and Windows. Any suggestions? Recommendation? Admonitions?

This is no-man's land for me, so any thoughts are sincerely and deeply appreciated.

On a final note, I really like Linux, especially as compared to Windows. Linux is instantly navigable to me. It makes sense. I can find things. I can do things. Nothing (so far) is buried and the UI is clean and unobtrusive. And not one single damn popup. It just goes to show, there are right ways to do things, and there are wrong ways. In my opinion, where Linux (and, of course, Mac) generally get things right, Windows most often fails miserably. Anyway, thanks be to Linux for renewing my interest in my $1400 former doorstop, my good pal satan.

Alright, gonna go Google the phrase "my good pal satan" now. More info as the situation develops.

UPDATE 1:
Tried updating Linux with the up2date software. Yup. Ran out of disk space. Looks like I'll be doing this all again. S'Okay. Practice makes perfect.

UPDATE 2:
Our resident Windows/Linux guru informs me that both OSs need a fair amount of space, and recommends splitting the hard drive into two 17 GB partitions, one for Windows and one for Linux, and foregoing the shared data drive. So that's probably what I'll do. 'Cause she's a smartie.

Windows and Linux Dual Boot Part 3: Yum

I'm discovering the magic of yum in Linux. Such an apt mane, yum. Yes, I did break my system with yum. But I've also just fixed it. And it was a breeze. I can't say the system is as good as new, but using yum, I was able to reinstall KDE, and I have a GUI back. In fact, I now have my choice of desktop environments. It's weird, though: My KDE looks very differnt than the one that was installed from the CD. I'm guessing the one on disc is some custom setup.

Anyway, I'm off to play. Still might reinstall at some point, but i'll wait and see what other coolness I can find.