A computer components & hardware forum. HardwareBanter

If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

Go Back   Home » HardwareBanter forum » General Hardware & Peripherals » Storage & Hardrives
Site Map Home Register Authors List Search Today's Posts Mark Forums Read Web Partners

Disk optimization for multithreaded app.



 
 
Thread Tools Display Modes
Prev Previous Post   Next Post Next
  #1  
Old February 9th 10, 06:03 PM posted to comp.arch.storage
I Understand
external usenet poster
 
Posts: 1
Default Disk optimization for multithreaded app.

Mine is a disk based backup/restore product. It is multithreaded,
which means many backups and restores are happening simultaneously.
The product has basically three types of interactions with the disk.



1. Read. This is synchronous. Most reads happen on mounted
shadow copies (using VSS).

2. Write. FILE_FLAG_WRITE_THROUGH is used since data integrity
is critical to the product.

3. Compute checksums on the files.



Each of the above operations are done in their own dedicated threads
and currently any number of threads can be run at any point in time
doing any of the operations in parallel. Each operation targets
exactly one LUN. The size of the data varies from few KBs to Giga
bytes. Also, we may suggest clients to optimize their disk
architecture (use RAID, faster disks, etc.) but the clients may choose
to ignore suggestions.



My questions are,

1. We have noticed that when we run a Read thread and a compute
checksum thread in parallel against the same disk/spindle the disk
throughput degrades considerably and serializing the two threads gives
us much higher throughput overall. Is this behavior expected?

2. If the behavior in 1. is expected then how should we
serialize/parallelize the three kinds of threads to achieve optimal
performance.

3. Are there Windows APIs to determine the spindles we are
working against and throttle our threads accordingly to achieve
optimal performance?

4. Any pointers to case studies, experiments or white papers/
research papers by folks who have done this before?

 




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Optimization for Windows XP [email protected] Dell Computers 0 August 20th 07 07:04 PM
New version of sys_basher, a multithreaded system exerciser andbenchmarking tool General Schvantzkoph Intel 5 January 2nd 07 05:28 PM
New version of sys_basher, a multithreaded system exerciser andbenchmarking tool General Schvantzkoph AMD x86-64 Processors 5 January 2nd 07 05:28 PM
GA-7N400 Pro 2 Rev. 2.0 Optimization? William C Gigabyte Motherboards 4 June 25th 05 06:43 PM
A question about optimization David C. Intel 8 July 9th 03 09:29 PM


All times are GMT +1. The time now is 05:18 PM.


Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright ©2004-2024 HardwareBanter.
The comments are property of their posters.