Netbackup very slow

I would like to gauge opinion on the problems that have developed with
our Netbackup system.

Since about August, last year, Our Netbackup system has gone from
very good, to almost a to a crawl.

The setup is as follows:-

1 Master/Media server
2 Media servers
all running Windows 2000

storage is 3x StorageTek 9720 Libriaries, each containing 3 DLT7000
tape drives.

The storage is connected to the Media servers via a Crossroads fibre
channel/SCSI bridge.

All servers backed up are running Gigabit NIC cards.

Approx. total backup size 2.5 to 3 Terabytes
spread over about 27 windows 2000 clients.

The data is multiplexed, and multiple streams run simultaneously
from each client.



Up till about august last year, the full backup, that would start
at about 8PM on a friday evening, would have been almost complete
by sunday afternoon. But a situation has developed where quite
alot of it is now still running Wednesday.

We have looked at all posible problems. NIC Cards, Fibre Channel Bridge,
Network activity during backup, remove/reapply Microsoft patches,
tape library/drives, etc etc etc. The list is endless.

Veritas, seem completly stumped. All solutions they have provided
has been totaly useless, and most of which they tell us appears
to be clutching at straws.

What I do know is, its not the tapes thats causing the problem,
and its not the network, as I can do multiple robocopies from
some of the worst servers at speeds that I consider apropriate.

I would really apreciate any advice from anyone who has seen
this situation before, or knows a reason why this mess
is occuring.

Thank You.

--
Mark Gary
email me at: uk.co.demon.mwgary.nospam@mark - Reverse it, and
remove nospam.

Up till about august last year, the full backup, that would start
at about 8PM on a friday evening, would have been almost complete
by sunday afternoon. But a situation has developed where quite
alot of it is now still running Wednesday.

Mark, what is the transfer rate per drive when you check running backup
jobs after the weekend?

Do you have carefully checked what changes have been made to the whole
system, incredible that such kind of decrease happens without anything
changed. Maybe the data structure of your servers changed, backup will take
really loooong if you backup tons of little files.

We have looked at all posible problems. NIC Cards, Fibre Channel Bridge,
Network activity during backup, remove/reapply Microsoft patches,
tape library/drives, etc etc etc. The list is endless.

What I do know is, its not the tapes thats causing the problem,
and its not the network, as I can do multiple robocopies from
some of the worst servers at speeds that I consider apropriate.

Are you able to segregate the problem to any specific part (server, job ...)?

Rene

If all other things have been checked out fine, one wild guess would be
there is link level retry is going on for I/O. If SRR (Sequence Request
Retransmit) is happening a lot you would see slow performance and no SCSI
errors to high level drivers See if crossroads reports statistics of frame
errs. Try after changin GBICs if there is link level issue.
--sam

"Mark Gary" wrote in message
k...

I would like to gauge opinion on the problems that have developed with
our Netbackup system.

Since about August, last year, Our Netbackup system has gone from
very good, to almost a to a crawl.

The setup is as follows:-

1 Master/Media server
2 Media servers
all running Windows 2000

storage is 3x StorageTek 9720 Libriaries, each containing 3 DLT7000
tape drives.

The storage is connected to the Media servers via a Crossroads fibre
channel/SCSI bridge.

All servers backed up are running Gigabit NIC cards.

Approx. total backup size 2.5 to 3 Terabytes
spread over about 27 windows 2000 clients.

The data is multiplexed, and multiple streams run simultaneously
from each client.



Up till about august last year, the full backup, that would start
at about 8PM on a friday evening, would have been almost complete
by sunday afternoon. But a situation has developed where quite
alot of it is now still running Wednesday.

We have looked at all posible problems. NIC Cards, Fibre Channel Bridge,
Network activity during backup, remove/reapply Microsoft patches,
tape library/drives, etc etc etc. The list is endless.

Veritas, seem completly stumped. All solutions they have provided
has been totaly useless, and most of which they tell us appears
to be clutching at straws.

What I do know is, its not the tapes thats causing the problem,
and its not the network, as I can do multiple robocopies from
some of the worst servers at speeds that I consider apropriate.

I would really apreciate any advice from anyone who has seen
this situation before, or knows a reason why this mess
is occuring.

Thank You.

--
Mark Gary
email me at: uk.co.demon.mwgary.nospam@mark - Reverse it, and
remove nospam.