If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below. |
|
|
Thread Tools | Display Modes |
|
#1
|
|||
|
|||
Linux LVM software raid, and SAN question.
In the beefy enterprise-level linux database installs, the ones where
you need massive redundancy supplied by two SAN devices, what is the current ideal for setting them up? Software raid to mirror the luns from both sans, and use static partitions? Software raid, then LVM ontop of it? LVM to mirror and handle the whole damned thing? PowerPath for multipathing, or using the multipathing the kernel provides? The particular instance I'm dealing with is a database server that will have two HBAs, each connected to a different SAN, which is mirrored from the linux side. While there are several ways I know how to set it up, I'm unsure as to which will really be the best way (not necessarily easiest) to do it. Thanks --Kyle |
#2
|
|||
|
|||
Linux LVM software raid, and SAN question.
Kyle Schmitt wrote:
In the beefy enterprise-level linux database installs, the ones where you need massive redundancy supplied by two SAN devices, what is the current ideal for setting them up? I think the linux part conflicts with "beefy", "enterprise-level" and "massive redundancy" |
#3
|
|||
|
|||
Linux LVM software raid, and SAN question.
On Tue, 22 Jan 2008 15:30:05 -0800 (PST), Kyle Schmitt
wrote: In the beefy enterprise-level linux database installs, the ones where you need massive redundancy supplied by two SAN devices, what is the current ideal for setting them up? Software raid to mirror the luns from both sans, and use static partitions? Software raid, then LVM ontop of it? LVM to mirror and handle the whole damned thing? PowerPath for multipathing, or using the multipathing the kernel provides? The particular instance I'm dealing with is a database server that will have two HBAs, each connected to a different SAN, which is mirrored from the linux side. While there are several ways I know how to set it up, I'm unsure as to which will really be the best way (not necessarily easiest) to do it. Thanks --Kyle Someone posted the inherent flaws in your statement about beefy and such so I'll skip that (but they're right you know...). What you're asking about is two fabrics. One hba goes to a switch which goes to a storage port on the array. The other hba goes to a different switch and a storage port on the same array as the other switch. Nothing ties these two switches together. This is fairly common practice when availability is a top priority. Forget LVM based raid, it's almost pointless when the hardware based raid is in the array (I assume you have an array though you do not state that explicitly). However make sure the LVM is in place. Such things make backend migrations much simpler. Do not cross the streams Ray.... Do not create any connection betwee the two fabrics. This allows for independent maintenance to each fabric with zero impact on the application. As for multi-pathing, I thought MPIO was suported on Linux. If so, use it. It works well and is free. Avoid PowerPath like the plague. I'm not even sure they sell it anymore. I think Hitachi has EOL'd their version of pay-for multipathing as well. Hmm, perhaps I'm missing a piece here. Are you asking about having two completely seperate SANs and not just fabrics? So two arrays as well? If so, that's less common but not unheard of. Same practice applies only you would use LVM to mirror. Or at least I would. ~F |
#4
|
|||
|
|||
Linux LVM software raid, and SAN question.
On Jan 22, 10:27*pm, Faeandar wrote:
On Tue, 22 Jan 2008 15:30:05 -0800 (PST), Kyle Schmitt wrote: In the beefy enterprise-level linux database installs, the ones where you need massive redundancy supplied by two SAN devices, what is the current ideal for setting them up? Software raid to mirror the luns from both sans, and use static partitions? Software raid, then LVM ontop of it? LVM to mirror and handle the whole damned thing? PowerPath for multipathing, or using the multipathing the kernel provides? The particular instance I'm dealing with is a database server that will have two HBAs, each connected to a different SAN, which is mirrored from the linux side. *While there are several ways I know how to set it up, I'm unsure as to which will really be the best way (not necessarily easiest) to do it. Thanks --Kyle Someone posted the inherent flaws in your statement about beefy and such so I'll skip that (but they're right you know...). What you're asking about is two fabrics. *One hba goes to a switch which goes to a storage port on the array. *The other hba goes to a different switch and a storage port on the same array as the other switch. Nothing ties these two switches together. This is fairly common practice when availability is a top priority. Forget LVM based raid, it's almost pointless when the hardware based raid is in the array (I assume you have an array though you do not state that explicitly). However make sure the LVM is in place. *Such things make backend migrations much simpler. Do not cross the streams Ray.... Do not create any connection betwee the two fabrics. *This allows for independent maintenance to each fabric with zero impact on the application. As for multi-pathing, I thought MPIO was suported on Linux. *If so, use it. *It works well and is free. *Avoid PowerPath like the plague. I'm not even sure they sell it anymore. *I think Hitachi has EOL'd their version of pay-for multipathing as well. Hmm, perhaps I'm missing a piece here. *Are you asking about having two completely seperate SANs and not just fabrics? *So two arrays as well? *If so, that's less common but not unheard of. Same practice applies only you would use LVM to mirror. *Or at least I would. ~F I probably didn't state it clearly enough then (and lets avoid the flame war that Linux vs Solaris vs AIX for enterprise level databases goes, I can only assume neither of you were silly enough to suggest windows should ever be used The switches won't be attached in any direct way, wasn't suggesting that. The box has two HBAs: HBA1 attaches to Switch1, which attaches n* ways SAN1 (which is floor standing cabinet full of discs, using hardware raid of course). HBA2 attaches to Switch2, which attaches n ways to SAN2. *I think n is 4 in this case We're dealing with two physical sans. The idea is the box will handle the mirroring of Disk1 (mounted on SAN1 via HBA1), onto Disk2 (mounted on SAN2 via HBA2). Does that make more sense pertaining to: software raid 1 vs lvm mirroring. And I'll look into the builtin MPIO (it exists and is considered stable IIRC, though I've never used it). I didn't realize powerpath raised such ire in people. Is it really that bad? Using it for the past half day it hasn't left a good impression so far, that's for sure. Thanks, Kyle |
#5
|
|||
|
|||
Linux LVM software raid, and SAN question.
On Jan 23, 8:05 am, Kyle Schmitt wrote:
The box has two HBAs: HBA1 attaches to Switch1, which attaches n* ways SAN1 (which is floor standing cabinet full of discs, using hardware raid of course). HBA2 attaches to Switch2, which attaches n ways to SAN2. *I think n is 4 in this case We're dealing with two physical sans. The idea is the box will handle the mirroring of Disk1 (mounted on SAN1 via HBA1), onto Disk2 (mounted on SAN2 via HBA2). Does that make more sense pertaining to: software raid 1 vs lvm mirroring. I don't want to suggest a bad architecture here (I'm sure there are political/historical reasons), but I'm curious as to why you would want software mirroring between two raid boxes in different SANs. Aiming for 'massive redundancy' by relying on software on a server to copy every byte to two different arrays isn't exactly common practice, at least not in my world... Just blindly mirroring between the two arrays only gives you some redundancy against hardware failure (while introducing possibility of software failure on your server), but doesn't help at all for logical or user errors, which are much more common. And the most common hardware failures (disk crash, controller crash, cable failure) are presumably already taken care of by the arrays, unless you've got a crappy one. Something more common, as suggested by earlier posts, would be to have one SAN with two separate fabrics, and every end-device (HBA and both storage arrays) connected to both fabrics, switches only in one fabric. Then use one array for primary storage, with dual redundant everything from the HBA down to the array, and use the other array for snapshots or clones. You can have those snapshots generated by your apps or by volume managers on the server if you like. Or some storage arrays support creating 'remote' snapshots. |
#6
|
|||
|
|||
Linux LVM software raid, and SAN question.
On Jan 23, 10:11*am, belpatCA wrote:
I don't want to suggest a bad architecture here (I'm sure there are political/historical reasons), but I'm curious as to why you would want software mirroring between two raid boxes in different SANs. Aiming for 'massive redundancy' by relying on software on a server to copy every byte to two different arrays isn't exactly common practice, at least not in my world... Just blindly mirroring between the two arrays only gives you some redundancy against hardware failure (while introducing possibility of software failure on your server), but doesn't help at all for logical or user errors, which are much more common. And the most common hardware failures (disk crash, controller crash, cable failure) are presumably already taken care of by the arrays, unless you've got a crappy one. Something more common, as suggested by earlier posts, would be to have one SAN with two separate fabrics, and every end-device (HBA and both storage arrays) connected to both fabrics, switches only in one fabric. Then use one array for primary storage, with dual redundant everything from the HBA down to the array, and use the other array for snapshots or clones. You can have those snapshots generated by your apps or by volume managers on the server if you like. Or some storage arrays support creating 'remote' snapshots. Each time I post I realize I forgot some piece of data or another In my description of one hba hooking up to a switch, which has four connections to the SAN, I failed to mention that the 4 connections consist of two fabrics. The consensus here has been that in the final setup, there will be two habs to each switch (total of four on the box), for redundancy. I'm not sure about remote snapshots, but it's possible that our SAN supports it, or that the package/upgrade for that is available. I'll have to look into it. Historical-political issues abound, far too much to bring up in a post. Suffice it to say, there are reasons we need and want two physical sans hooked up here, and the majority are even legitimate. Thanks, Kyle |
#7
|
|||
|
|||
Linux LVM software raid, and SAN question.
Kyle Schmitt wrote:
On Jan 23, 10:11?am, belpatCA wrote: I don't want to suggest a bad architecture here (I'm sure there are political/historical reasons), but I'm curious as to why you would want software mirroring between two raid boxes in different SANs. Aiming for 'massive redundancy' by relying on software on a server to copy every byte to two different arrays isn't exactly common practice, at least not in my world... Just blindly mirroring between the two arrays only gives you some redundancy against hardware failure (while introducing possibility of software failure on your server), but doesn't help at all for logical or user errors, which are much more common. And the most common hardware failures (disk crash, controller crash, cable failure) are presumably already taken care of by the arrays, unless you've got a crappy one. Something more common, as suggested by earlier posts, would be to have one SAN with two separate fabrics, and every end-device (HBA and both storage arrays) connected to both fabrics, switches only in one fabric. Then use one array for primary storage, with dual redundant everything from the HBA down to the array, and use the other array for snapshots or clones. You can have those snapshots generated by your apps or by volume managers on the server if you like. Or some storage arrays support creating 'remote' snapshots. Each time I post I realize I forgot some piece of data or another In my description of one hba hooking up to a switch, which has four connections to the SAN, I failed to mention that the 4 connections consist of two fabrics. The consensus here has been that in the final setup, there will be two habs to each switch (total of four on the box), for redundancy. I'm not sure about remote snapshots, but it's possible that our SAN supports it, or that the package/upgrade for that is available. I'll have to look into it. Historical-political issues abound, far too much to bring up in a post. Suffice it to say, there are reasons we need and want two physical sans hooked up here, and the majority are even legitimate. The main objection you're hearing here is that a database server should be also double as a SAN mirroring device. Say the machine crashes. Which san even has the correct data anymore? There are (costly) hardware devices which will present a a set of LUNs to your host, the database machines that do the mirroring to the separate SANs on the backend. Assuming this is what you really want, you end up with a database server doing only one thing, and that's being a database server, and you have some extra hardware dedicated to mirroring your SANs. Will all this extra stuff increate reliability? I really don't know. |
#8
|
|||
|
|||
Linux LVM software raid, and SAN question.
On Jan 22, 6:30*pm, Kyle Schmitt wrote:
In the beefy enterprise-level linux database installs, the ones where you need massive redundancy supplied by two SAN devices, what is the current ideal for setting them up? Software raid to mirror the luns from both sans, and use static partitions? Software raid, then LVM ontop of it? LVM to mirror and handle the whole damned thing? PowerPath for multipathing, or using the multipathing the kernel provides? The particular instance I'm dealing with is a database server that will have two HBAs, each connected to a different SAN, which is mirrored from the linux side. *While there are several ways I know how to set it up, I'm unsure as to which will really be the best way (not necessarily easiest) to do it. Thanks --Kyle Seems a silly way to pinch some pennies - you've already bought two SAN storage devices. Why not let the SAN storage devices mirror between each other (pref over the SAN), set up a second identical linux server and present both the primary and mirrored LUNS to both hosts over both fabrics? The second linux server and the second san storage device would be passive in this configuration and as long as you don't try to import or mount the san partitions on the second host while the first host has them you should be fine. i'd def use vxvm in this config to manage the luns which will help prevent that situation. it's also easier to recover from errors than ext3 or whatever linux is using now I can't imagine relying on software mirroring for this configuration, especially in linux, especially for databases. If the san devices are in different locations put the second linux box in the second location. throw linux out and replace it with something better first chance you get. You are relying on mirroring that is using device drivers written by random collections of geeks across the world. |
#9
|
|||
|
|||
Linux LVM software raid, and SAN question.
Kyle Schmitt wrote:
Software raid to mirror the luns from both sans, and use static partitions? Software raid, then LVM ontop of it? LVM to mirror and handle the whole damned thing? PowerPath for multipathing, or using the multipathing the kernel provides? The particular instance I'm dealing with is a database server that will have two HBAs, each connected to a different SAN, which is mirrored from the linux side. *While there are several ways I know how to set it up, I'm unsure as to which will really be the best way (not necessarily easiest) to do it. Thanks --Kyle 1 = Well just have read all the reply's you have got so far, let me suggest. http://www.symantec.com/business/the...hemeid=sfbasic And if you can do with max 4 volumes to be mirrored; go for it - Replacing the ext3 (or what ever) with vxfs and LWM with vxvm [remember to enable Dirty Read Logs (fast resync after crash)]. LWN in Linux is OK, but for your setup I would suggest this Software because is has more "guarantees", in fail situations. Simply think (based on 14 years in the IT business as administrator) crash situations is better handled by this commercial product. Faulty PSU's (or even worse, much worse, partly faulty PSU's - PSU's that don't die) I think is cause to a lot of errors in software based RAID solutions. RAID much much better done in hardware!! 2 = Depending on your database, let the database replicate itself to another database instance (on the other disk system). No need at all for software in the OS to complicate things. /Lars H. |
Thread Tools | |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Another question on RAID (software) | Carlos Moreno | Storage (alternative) | 10 | April 9th 05 03:50 AM |
Newbie Question re hardware vs software RAID | Gilgamesh | General | 44 | November 22nd 04 10:52 PM |
A7V600 Raid (linux?) question | tamer kavlak | Asus Motherboards | 0 | August 9th 04 12:41 PM |
Any P4P800-E Deluxe PATA RAID vs. XP software RAID benchmarks? | Shawn Barnhart | Asus Motherboards | 0 | July 21st 04 05:14 PM |
Redhat Linux on A7V333 - Software RAID problem | Steve | Asus Motherboards | 0 | February 27th 04 04:40 PM |