I'll be honest.
A second hard drive for user data only is a decently ideal practice if you have the technological know-how, as well as are paranoid about your data. However for the common user, this practice really doesn't do them any justice.
from my experience, this practice won't do you much good, even with Windows. I've found that when i've had PC crashes with Windows 98 (god that was ages ago) my secondary drive also crashed with the primary. linux, the same thing can happen if you're not careful. I know there's been advances in how ext3 and the new ext4 file systems recover data, but it's not fool-proof. Backups are your friend, and thats what people should always do.
As far as servers go.. You ALWAYS keep the OS on it's own partition. may it be either a physical partition on a single hard drive, or a software partition on a spanned array. Data goes in it's own drive. (I should follow that with my new dedicated boxes...)