If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below. |
|
|
Thread Tools | Display Modes |
#11
|
|||
|
|||
"Paul Rubin" wrote in message
... Malcolm Weir writes: Still, what you've experienced shows the benefit of writing lots of file marks. Classic Unix (and Unix-like) things (tar, etc.) want to write *one* tape file, and if part of that file is corrupted, then it's all lost. But if you write a filemark every (say) 100MB, then the chances of being able to seek past the bad spot improve dramatically. I've heard that if you write two consecutive EOF marks on a DDS or 8mm tape, that's interpreted as end-of-tape and there's absolutely no way to read past it. It's not like the old days of 9 track tape. One more example of half-truth. Filemarks are logical entities on tape, just like data blocks are. Reading past two filemarks is just as well possible as reading past any other data entity on tape. You can start your tape write with 500 filemarks, then a single block of data. You'll always be able to get that single block back. The story of not being able to read past two filemarks is based on the fact that it normally marks the end of a write session. In many data streams, the end of *logical* data is marked by writing two consecutive filemarks. It serves as an append point for later sessions. Tape, being sequential media, also writes a *physical* end-of-data marker at that point, and usually it's followed by an erase gap. Depending on the implementation, the erase gap is a single track, or can be a number of tracks wide. It invalidates any tracks behind the end of data marker, by partially or completely erasing them. Reading past such an area is normally prevented in firmware, as anything behind the EOD marker is not considered part of the current tape sequence (remember, sequential data). However, most drives can be persuaded to skip such an erase area to the next valid filemark, and can continue to read from there on. Data recovery centers use this feature to get 'overwritten' data back. I've written software to even automate the recovery process on EXB-8200 through 8505XL, it's very easy to do. On most helical scan drives you can do this. On most linear drives you can do this too (except DLT7000, as it destroys multiple tracks with a single write). Rob |
#12
|
|||
|
|||
Rob Turk wrote:
"Marcin Dobrucki" wrote in message ... idunno wrote: I think I'm missing something, but why do people use tape for backups? It seems very costly and slow. August 2003 issue of SysAdmin Magazine (www.sysadminmag.com) has an article just for you. "Tapes: A Modern History, Trends", by Henry Newman, p. 43 This article is so full of semi-technical nonsense that it ain't funny anymore. I agree with the overall conclusion (tape is here to stay) but this guy has definitely not been doing his homework on helical scan recording. This stuff is directly copied out of 10-year old Quantum DLT sales pitches. Yuck... I'll take your word for it since I don't know much about the stuff, but I thought the article pointed out a few good things about the original question, i.e. what's great about tape. And like you, I agree with the conclusion, I can't see it go away anytime soon. Out of curiosity, can you point me at a good reference about the recording technologies? I need to brush up on this subject. /Marcin |
#13
|
|||
|
|||
|
#14
|
|||
|
|||
There are several reason for doing backups of any kind. The main two seem to
be: 1) recovery in case of a drive death 2) archiving, to let you recover deleted information, take snapshots etc The first of these issues can, to a certain extent, be covered by having a disk raid system.... but even that doesn't solve the disater situation (fire, flood etc) The second issue is a way of either allowing you to recover accidentally changed / deleted data, as well as allowing you to "snap-shot" your data (as is often needed for legal reasons), and that sort of thing. Of course, there are 2 aspects of the backup: "protecting" your operating system & software, and "protecting" your data... its all very well having a great data backup, that lets you recover al your data in 2 hours flat, if (because of bad documentation or whatever) it takes you 3 days to correctly re-install the underlying software!!!!! From what I have seen (we have recently intslled a tape system that uses ArcServe) tape backup has NOT really improved in the last 15 years: the software is still very slow, very cumbersome, and still not easy & obvious to use. The next thing to ask is "what alternatives to tape are there": and as others have said, there is only ONE that has enough capacity: hard disks! CDR's are ok for small amounts of data (eg data on a personal computer, rather than a server), but are nowhere near big enough or "good enough" for (longer-term) backups of large amounts of data. As well as needing backup where I work, I am involved with a hospital radio station, where we have a computer playout system: the work needed to recover all the music an associated data is huge, so it HAS to be backed up: in this case, both PCs have a small HD with the OS & software on (which we have ghosted onto CD, and can rebuild the pc in about 10 mins flat!), and a big HD for the music in a caddy. One of the PC's also has a spare caddy, where we plug in a second hd & copy all data to it: that HD is kept off-site. In this case, we have no need for keeping archives. I think there is still very much a place for tapes, mainly for longer-term data archiving of large amounts of data. Paul "idunno" wrote in message ... I think I'm missing something, but why do people use tape for backups? It seems very costly and slow. I can imagine two benefits from tape, though. The backup cartridge is not nearly as fragile as say a hard disk and is therefore better to transport offsite. Some better tape formats and tape libraries can backup vast quantities of data easily and unattended. I've been using hard disks in removable racks & external cases as well as DVD-R. I've had bad experiences with travan a few years ago and I've hear bad things about DAT reliability. I've been thinking about implementing VXA or DLT based backup in a small office, but I'm having a difficult time justifying the cost. These drives and the tapes are quite pricey esp. if these tapes can only be used a limited number of times. I am very worried about various nuisances with the media or drive heads resulting from daily backups. Sorry if my newbiness is showing. What am I missing here? How exactly are the costs justified? For whom is tape backup best suited? |
#15
|
|||
|
|||
"Rob Turk" writes:
I've heard that if you write two consecutive EOF marks on a DDS or 8mm tape, that's interpreted as end-of-tape and there's absolutely no way to read past it. It's not like the old days of 9 track tape. One more example of half-truth. .... Reading past such an area is normally prevented in firmware, as anything behind the EOD marker is not considered part of the current tape sequence (remember, sequential data). However, most drives can be persuaded to skip such an erase area to the next valid filemark, and can continue to read from there on. Data recovery centers use this feature to get 'overwritten' data back. I've written software to even automate the recovery process on EXB-8200 through 8505XL, The subject has come up here on c.a.s. or comp.periphs.scsi a few times in the past and there don't seem to be any such methods available to ordinary users. There was some kludge posted about starting a read and powering down the drive at some critical instant and powering it up again, or something like that, which sometimes confuses the drive enough to read the data. Anyway, yes, I shouldn't have said "absolutely no way" since it has to be possible if you can modify the firmware. However, if it requires firmware mods or the resources of a data recovery center, that's close enough to impossible for most users. |
#16
|
|||
|
|||
On 3 Sep 2003 14:55:07 -0500, Eric Lee Green wrote:
In article , Boll Weevil ruminated: First of all, nobody makes a robotic hard drive changer. We go through about 500 tapes a day using 20 to 40 tape drives concurrently all managed by automated robotic tape libraries. I can't imagine to trying Let me get this straight. You back up 50 terabytes per day? Or are you using older/smaller capacity technology, let's say DLT1, and backing up, say, 10 terabytes per day? 10 terabytes per day = 3650 terabytes per year. You're saying that your installation is pushing 3650 terabytes of data per year through your systems? Or are you saying that, due to the inefficiencies of current tape backup solutions (which operate upon a whole-file basis rather than on a differential block basis), you need 3650 terabytes of tape storage to store, say, 365 terabytes of changed data? See if you can add this up. To start, we have about 200 Sun servers and about 1200 NT servers. About 100 Sun servers and about 100 NT servers are on the SAN and share the following EMC and Hitachi subsystems: 7 EMC 8830 frames with about 13 TB raid 10 useable, each 1 Hitachi 9980V frame with about 45 TB raid 5 useable There are a whole lot of direct attached SCSI disk arrays and internal disks in each of the 1200 NT servers. I can't even start as to how much storage these servers account for. These all get backed up. We use STK 9840A and 9840B drives. These drives can do about 20GB per tape at about 20 to 40 mb per second. 500 tapes??? Well, I think I've mistaken or I kicked out an old number. We probably use twice that. Most of the SAN disk is used for Oracle and SAS databases. Oh yea, we also have about 200 TB of mainframe but IBM manages that. So, do some math and figure out how many tapes we use on a full backup. You can save me some time. |
#17
|
|||
|
|||
"Paul Rubin" wrote in message
... "Rob Turk" writes: I've heard that if you write two consecutive EOF marks on a DDS or 8mm tape, that's interpreted as end-of-tape and there's absolutely no way to read past it. It's not like the old days of 9 track tape. One more example of half-truth. .... Reading past such an area is normally prevented in firmware, as anything behind the EOD marker is not considered part of the current tape sequence (remember, sequential data). However, most drives can be persuaded to skip such an erase area to the next valid filemark, and can continue to read from there on. Data recovery centers use this feature to get 'overwritten' data back. I've written software to even automate the recovery process on EXB-8200 through 8505XL, The subject has come up here on c.a.s. or comp.periphs.scsi a few times in the past and there don't seem to be any such methods available to ordinary users. There was some kludge posted about starting a read and powering down the drive at some critical instant and powering it up again, or something like that, which sometimes confuses the drive enough to read the data. Anyway, yes, I shouldn't have said "absolutely no way" since it has to be possible if you can modify the firmware. However, if it requires firmware mods or the resources of a data recovery center, that's close enough to impossible for most users. What you describe was for 4mm DAT drives, start a write and power down, so no EOD marker would be recorded. Then spacing over the (now corrupted) erase area would work. For Exabyte drives, for 8200's no mods were needed. The firmware (32KB of code on an 8051 CPU) did not require any changes to space to the next filemark after EOD. Not enought smarts to detect EOD while spacing ;-) For all other 85xx drives a standard EE-image implementing 'directory support' allowed you to do the same thing. No tricks, no switching power, just using the features of the drive. Rob |
#18
|
|||
|
|||
"Boll Weevil" wrote in message
... On 3 Sep 2003 14:55:07 -0500, Eric Lee Green wrote: See if you can add this up. To start, we have about 200 Sun servers and about 1200 NT servers. About 100 Sun servers and about 100 NT servers are on the SAN and share the following EMC and Hitachi subsystems: 7 EMC 8830 frames with about 13 TB raid 10 useable, each 1 Hitachi 9980V frame with about 45 TB raid 5 useable There are a whole lot of direct attached SCSI disk arrays and internal disks in each of the 1200 NT servers. I can't even start as to how much storage these servers account for. These all get backed up. We use STK 9840A and 9840B drives. These drives can do about 20GB per tape at about 20 to 40 mb per second. 500 tapes??? Well, I think I've mistaken or I kicked out an old number. We probably use twice that. Most of the SAN disk is used for Oracle and SAS databases. Oh yea, we also have about 200 TB of mainframe but IBM manages that. So, do some math and figure out how many tapes we use on a full backup. You can save me some time. So you work for Boeing, right? One of the largest Powderhorn sites I know of... Rob |
#19
|
|||
|
|||
"Rob Turk" writes:
For Exabyte drives, for 8200's no mods were needed. The firmware (32KB of code on an 8051 CPU) did not require any changes to space to the next filemark after EOD. Not enought smarts to detect EOD while spacing ;-) For all other 85xx drives a standard EE-image implementing 'directory support' allowed you to do the same thing. No tricks, no switching power, just using the features of the drive. Thanks, that helps. How is the situation for current Exabyte/Ecrix drives? Also, suppose you have an M3 or VXA2 cartridge and cut the tape somewhere in the middle with a scissors. Can you get back (most of) the data from both pieces without having to go too crazy with special programming? Is there enough redundancy to not lose data around where the cut is? I figure you can't splice the tape, but maybe you can spool each piece onto another cartridge or something. |
#20
|
|||
|
|||
In article , Boll Weevil wrote:
See if you can add this up. To start, we have about 200 Sun servers and about 1200 NT servers. About 100 Sun servers and about 100 NT servers are on the SAN and share the following EMC and Hitachi subsystems: 7 EMC 8830 frames with about 13 TB raid 10 useable, each 1 Hitachi 9980V frame with about 45 TB raid 5 useable There are a whole lot of direct attached SCSI disk arrays and internal disks in each of the 1200 NT servers. I can't even start as to how much storage these servers account for. These all get backed up. Rough calculations shows that using hardware compression for a LTO-2 setup, one could do all of the above in about 2 (or so) fully decked out IBM 3584 LTO libraries (just as an example), assuming an average size of directly attached storage for each of the NT servers being 1 TB. A decked-out LTO-2 library with 6 frames should yield in the neighborhood of about 720 TB of tape storage capabilities. If 1200 servers * 1 TB = 1200 TB; that'd be one decked out LTO library and a second library with about 240 TB of available tape space. For the other stuff... 13 * 7 = 91 plus 45 TB = 136 TB. So you'd still have 104 TB of free data space, and capable of doing a single full backup for everything with two libraries and about 3500 tapes. This assumes 400 GB (hw compressed) LTO-2 tapes; if you are using 20 GB tapes in uncompressed mode, then your tape requirements goes up by 20 times 3500 for at least 70,000 tapes. Also, if the average per-NT server for storage is other than 1 TB, that would also influence number of tapes required, as well. -Dan |
Thread Tools | |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Cannot eject tape from PV 120T DLT1 Autoloader drive | [email protected] | Dell Computers | 3 | January 25th 05 08:56 PM |
Great storage method, is it available in UK??? | Mark | General | 5 | March 14th 04 10:58 AM |
ati video on demand is great - export sucks | Nicholas Tse | Ati Videocards | 1 | August 28th 03 05:11 PM |
cutting psu wires | Pen | General | 4 | July 27th 03 07:49 PM |
Records great, but what about the audio??? | mxh | Ati Videocards | 7 | July 22nd 03 05:37 AM |