If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below. |
|
|
|
Thread Tools | Display Modes |
#1
|
|||
|
|||
Managing data sprawl
Over last couple of years the certain types of data is exploding in
our enterprise (probably true for others too). These include source code, binary images (and many many variations of those), all kinds of documents (word, excel), wikis etc. These are kept like regular files (available via NAS) because they must be available on demand. These are not being used heavily but it is not something that can be backed up and retrieved. In a sense they are like reference data but some modifications do happen from time to time. This kind of data is growing like mushroom, almost increasing by 20-40TB+ per year and increasing. Is there a way to reduce the amount of these data by doing, - inline compression. I saw some discussion about storewiz, but nobody seems to have used that. Wonder why? - Can it be moved to some kind of CAS box transparently and then pulled in when there is an access. Is this type of growth is happening in other enterprises, businesses? I would suspect it is. How do you manage this? Sorry if I am repeating some earlier discussion. Thanks Sam |
#2
|
|||
|
|||
Managing data sprawl
On Feb 6, 2:15 pm, wrote:
Over last couple of years the certain types of data is exploding in our enterprise (probably true for others too). These include source code, binary images (and many many variations of those), all kinds of documents (word, excel), wikis etc. These are kept like regular files (available via NAS) because they must be available on demand. These are not being used heavily but it is not something that can be backed up and retrieved. In a sense they are like reference data but some modifications do happen from time to time. This kind of data is growing like mushroom, almost increasing by 20-40TB+ per year and increasing. Is there a way to reduce the amount of these data by doing, - inline compression. I saw some discussion about storewiz, but nobody seems to have used that. Wonder why? - Can it be moved to some kind of CAS box transparently and then pulled in when there is an access. Is this type of growth is happening in other enterprises, businesses? I would suspect it is. How do you manage this? Sorry if I am repeating some earlier discussion. Thanks Sam Some new companies are looking into these types of technologies... its very new, but promising... I know we can use something like a StoreWiz in our data center... The issue I had with them so far was their scalability. If they performed faster, my boss would probably consider them I think Dvy |
#3
|
|||
|
|||
Managing data sprawl
On Feb 6, 3:13 pm, wrote:
On Feb 6, 2:15 pm, wrote: Over last couple of years the certain types of data is exploding in our enterprise (probably true for others too). These include source code, binary images (and many many variations of those), all kinds of documents (word, excel), wikis etc. These are kept like regular files (available via NAS) because they must be available on demand. These are not being used heavily but it is not something that can be backed up and retrieved. In a sense they are like reference data but some modifications do happen from time to time. This kind of data is growing like mushroom, almost increasing by 20-40TB+ per year and increasing. Is there a way to reduce the amount of these data by doing, - inline compression. I saw some discussion about storewiz, but nobody seems to have used that. Wonder why? - Can it be moved to some kind of CAS box transparently and then pulled in when there is an access. Is this type of growth is happening in other enterprises, businesses? I would suspect it is. How do you manage this? Sorry if I am repeating some earlier discussion. Thanks Sam Some new companies are looking into these types of technologies... its very new, but promising... I know we can use something like a StoreWiz in our data center... The issue I had with them so far was their scalability. If they performed faster, my boss would probably consider them I think Dvy Thanks. What other companies? Any pointers. Also come to think about, doing this might require a newer type of filesystem. That means let go our big irons, which may not hapen so easily. If it is read only it might be better but read/write (I should say the update) is going to make it harder. Do you see this sprawl in your datacenter |
#4
|
|||
|
|||
Managing data sprawl
On Feb 6, 4:46 pm, wrote:
On Feb 6, 3:13 pm, wrote: On Feb 6, 2:15 pm, wrote: Over last couple of years the certain types of data is exploding in our enterprise (probably true for others too). These include source code, binary images (and many many variations of those), all kinds of documents (word, excel), wikis etc. These are kept like regular files (available via NAS) because they must be available on demand. These are not being used heavily but it is not something that can be backed up and retrieved. In a sense they are like reference data but some modifications do happen from time to time. This kind of data is growing like mushroom, almost increasing by 20-40TB+ per year and increasing. Is there a way to reduce the amount of these data by doing, - inline compression. I saw some discussion about storewiz, but nobody seems to have used that. Wonder why? - Can it be moved to some kind of CAS box transparently and then pulled in when there is an access. Is this type of growth is happening in other enterprises, businesses? I would suspect it is. How do you manage this? Sorry if I am repeating some earlier discussion. Thanks Sam Some new companies are looking into these types of technologies... its very new, but promising... I know we can use something like a StoreWiz in our data center... The issue I had with them so far was their scalability. If they performed faster, my boss would probably consider them I think Dvy Thanks. What other companies? Any pointers. Also come to think about, doing this might require a newer type of filesystem. That means let go our big irons, which may not hapen so easily. If it is read only it might be better but read/write (I should say the update) is going to make it harder. Do you see this sprawl in your datacenter I dont know of the companies by name. I do see this sprawl in my DC... I suspect everyone has similar problems... |
#5
|
|||
|
|||
Managing data sprawl
Where do you store these not so often used files (but often
enuf).....netapp R300 types? On Feb 6, 4:57 pm, wrote: On Feb 6, 4:46 m, wrote: On Feb 6, 3:13 pm, wrote: On Feb 6, 2:15 pm, wrote: Over last couple of years the certain types of data is exploding in our enterprise (probably true for others too). These include source code, binary images (and many many variations of those), all kinds of documents (word, excel), wikis etc. These are kept like regular files (available via NAS) because they must be available on demand. These are not being used heavily but it is not something that can be backed up and retrieved. In a sense they are like reference data but some modifications do happen from time to time. This kind of data is growing like mushroom, almost increasing by 20-40TB+ per year and increasing. Is there a way to reduce the amount of these data by doing, - inline compression. I saw some discussion about storewiz, but nobody seems to have used that. Wonder why? - Can it be moved to some kind of CAS box transparently and then pulled in when there is an access. Is this type of growth is happening in other enterprises, businesses? I would suspect it is. How do you manage this? Sorry if I am repeating some earlier discussion. Thanks Sam Some new companies are looking into these types of technologies... its very new, but promising... I know we can use something like a StoreWiz in our data center... The issue I had with them so far was their scalability. If they performed faster, my boss would probably consider them I think Dvy Thanks. What other companies? Any pointers. Also come to think about, doing this might require a newer type of filesystem. That means let go our big irons, which may not hapen so easily. If it is read only it might be better but read/write (I should say the update) is going to make it harder. Do you see this sprawl in your datacenter I dont know of the companies by name. I do see this sprawl in my DC... I suspect everyone has similar problems... |
#6
|
|||
|
|||
Managing data sprawl
On Feb 7, 9:43 am, wrote:
Where do you store these not so often used files (but often enuf).....netapp R300 types? On Feb 6, 4:57 pm, wrote: On Feb 6, 4:46 m, wrote: On Feb 6, 3:13 pm, wrote: On Feb 6, 2:15 pm, wrote: Over last couple of years the certain types of data is exploding in our enterprise (probably true for others too). These include source code, binary images (and many many variations of those), all kinds of documents (word, excel), wikis etc. These are kept like regular files (available via NAS) because they must be available on demand. These are not being used heavily but it is not something that can be backed up and retrieved. In a sense they are like reference data but some modifications do happen from time to time. This kind of data is growing like mushroom, almost increasing by 20-40TB+ per year and increasing. Is there a way to reduce the amount of these data by doing, - inline compression. I saw some discussion about storewiz, but nobody seems to have used that. Wonder why? - Can it be moved to some kind of CAS box transparently and then pulled in when there is an access. Is this type of growth is happening in other enterprises, businesses? I would suspect it is. How do you manage this? Sorry if I am repeating some earlier discussion. Thanks Sam Some new companies are looking into these types of technologies... its very new, but promising... I know we can use something like a StoreWiz in our data center... The issue I had with them so far was their scalability. If they performed faster, my boss would probably consider them I think Dvy Thanks. What other companies? Any pointers. Also come to think about, doing this might require a newer type of filesystem. That means let go our big irons, which may not hapen so easily. If it is read only it might be better but read/write (I should say the update) is going to make it harder. Do you see this sprawl in your datacenter I dont know of the companies by name. I do see this sprawl in my DC... I suspect everyone has similar problems... We leave them on our 960. We dont know when a file will be accessed, so we leave them there. |
#7
|
|||
|
|||
Managing data sprawl
|
#8
|
|||
|
|||
Managing data sprawl
You could look at products such as the Centera from EMC, or RISS from
HP. Both are designed to store reference data. They both have good indexing tools to enable you to find the information once you off line it! Jc |
#9
|
|||
|
|||
Managing data sprawl
On Feb 8, 9:50 am, "Jc" wrote:
You could look at products such as the Centera from EMC, or RISS from HP. Both are designed to store reference data. They both have good indexing tools to enable you to find the information once you off line it! Jc Yes. But that requires you to move data from one type of storage to another. So the data has to be first classified. I haven't looked at these products, do they offer a NFS front end? Also modification is rare but not exactly uncommon. In that case data has to move from RISS type of platform to another filer and then migrated. |
#10
|
|||
|
|||
Managing data sprawl
On Feb 8, 6:23 pm, wrote:
On Feb 8, 9:50 am, "Jc" wrote: You could look at products such as the Centera from EMC, or RISS from HP. Both are designed to store reference data. They both have good indexing tools to enable you to find the information once you off line it! Jc Yes. But that requires you to move data from one type of storage to another. So the data has to be first classified. I haven't looked at these products, do they offer a NFS front end? Also modification is rare but not exactly uncommon. In that case data has to move from RISS type of platform to another filer and then migrated. Sumandra, there are many types of platforms that try and move data inline, like neopath and acopia.. .from my experience with them, they are like a forklift change of my network and too much hassle... If what you are experiencing is too much file growth, then do look at storewiz... it will actually help reduce duplicates and manage chaotic growth. I'd caution to really test out their performance... unless they have made too many improvements, you are dead in the water. Another area where we could use their help is with backup.... I had a seperate post related to this which Faender eloquently helped me with as usual, but if I can keep my backups on tier2 and have them compressed with a device like storewiz, that would truly aid me. I dont know if they can do that or not. It would involve them speaking to a backup server I suppose. |
|
Thread Tools | |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
The science of Data Recovery according to "Mr. Stack" | [email protected] | Storage & Hardrives | 0 | October 18th 05 11:25 PM |
Modem connection speed | Neil Barnwell | General | 58 | July 14th 04 07:18 PM |
my new mobo o/c's great | rockerrock | Overclocking AMD Processors | 9 | June 30th 04 08:17 PM |
Sata and Data Corruption | Robert Neville | Storage (alternative) | 27 | May 8th 04 06:20 PM |
help with motherboard choice | S.Boardman | Overclocking AMD Processors | 30 | October 20th 03 10:23 PM |