If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below. |
|
|
Thread Tools | Display Modes |
#1
|
|||
|
|||
Utility to write huge files instantly???
Windows 2000, FAT32 or NTFS.
For testing, I want a utility to create BIG files ~1GB on storage media quickly. Copying a big file is not an option, too slow. I don't care what's in the file as long as the OS is happy that it's a "valid" file. Needs to work thru normal OS drive letters and drivers. Should be able to just write the FAT without doing anything to the actual allocation units being allocated??? Anything like this exist? Thanks, mike -- Return address is VALID! |
#2
|
|||
|
|||
Utility to write huge files instantly???
Should be able to just write the FAT without doing anything to the
actual allocation units being allocated??? Do you need NTFS support? If you will go thru the usual filesystem, without patching the metadata manually - then lseek() to (1GB - 1) and then write 1 byte of zero. The OS will zero the whole 1GB automatically. This is recommended by MS's filesystem guys to create the minimally fragmented file, and is also portable to UNIXen (UNIXen will create a sparse file in this case though). -- Maxim Shatskih, Windows DDK MVP StorageCraft Corporation http://www.storagecraft.com |
#3
|
|||
|
|||
Utility to write huge files instantly???
Maxim S. Shatskih wrote:
Should be able to just write the FAT without doing anything to the actual allocation units being allocated??? Do you need NTFS support? If you will go thru the usual filesystem, without patching the metadata manually - then lseek() to (1GB - 1) and then write 1 byte of zero. The OS will zero the whole 1GB automatically. I don't think he wants to take the time to zero (or otherwise write) the actual file data. ISTR that an (undocumented?) API exists to preallocate a file without writing it, though (useful for file-copy operations where the final size is known, you want reasonable contiguity, but don't want the useless overhead of zeroing the output file before overwriting it with data: IIRC NT maintains 'high water mark' information similar to VMS's that keeps one from scavenging previous data on the disk in such cases before the real data is written). - bill |
#4
|
|||
|
|||
Utility to write huge files instantly???
Bill Todd wrote:
Maxim S. Shatskih wrote: Should be able to just write the FAT without doing anything to the actual allocation units being allocated??? Do you need NTFS support? If you will go thru the usual filesystem, without patching the metadata manually - then lseek() to (1GB - 1) and then write 1 byte of zero. The OS will zero the whole 1GB automatically. I don't think he wants to take the time to zero (or otherwise write) the actual file data. ISTR that an (undocumented?) API exists to preallocate a file without writing it, though (useful for file-copy operations where the final size is known, you want reasonable contiguity, but don't want the useless overhead of zeroing the output file before overwriting it with data: IIRC NT maintains 'high water mark' information similar to VMS's that keeps one from scavenging previous data on the disk in such cases before the real data is written). - bill Thanks, but when I said "utility", I meant point/click/done. I can bungle my way thru a trivial VB6 program, but anything more than that is a stretch. I dug around in MSDN and found these: IStream::SetSize FileStream.SetLength Method CFile::SetLength But I'm not sure of their behavior relative to end of file and not taking a long time. And I'd be WAY beyond my comfort level trying to program it. Think "UTILITY". :-) mike -- Return address is VALID! |
#5
|
|||
|
|||
Utility to write huge files instantly???
mike wrote:
Windows 2000, FAT32 or NTFS. For testing, I want a utility to create BIG files ~1GB on storage media quickly. Copying a big file is not an option, too slow. I don't care what's in the file as long as the OS is happy that it's a "valid" file. Needs to work thru normal OS drive letters and drivers. Should be able to just write the FAT without doing anything to the actual allocation units being allocated??? Anything like this exist? Thanks, mike There are ports of the "dd" unix program for windows. It can be used to write giant files with minimal effort. |
#6
|
|||
|
|||
Utility to write huge files instantly???
Cydrome Leader wrote:
mike wrote: Windows 2000, FAT32 or NTFS. For testing, I want a utility to create BIG files ~1GB on storage media quickly. Copying a big file is not an option, too slow. I don't care what's in the file as long as the OS is happy that it's a "valid" file. Needs to work thru normal OS drive letters and drivers. Should be able to just write the FAT without doing anything to the actual allocation units being allocated??? Anything like this exist? Thanks, mike There are ports of the "dd" unix program for windows. It can be used to write giant files with minimal effort. Thanks, but minimal effort is not nearly as important as minimal time. The version of dd I tried does work, dd if=infile of=outfile seek=2000000 if is 200bytes. but it's only marginally faster than copying a file. I need something that's a thousand times or so faster than a file copy for a 1GB file. Maybe there's a version that doesn't fill the file with zeros, but just allocates the space?? Think about copying 16 gigabytes to a USB1.1 drive...while we're still young. -- Return address is VALID! |
#7
|
|||
|
|||
Utility to write huge files instantly???
Cydrome Leader wrote:
mike wrote: Windows 2000, FAT32 or NTFS. For testing, I want a utility to create BIG files ~1GB on storage media quickly. Copying a big file is not an option, too slow. I don't care what's in the file as long as the OS is happy that it's a "valid" file. Needs to work thru normal OS drive letters and drivers. Should be able to just write the FAT without doing anything to the actual allocation units being allocated??? Anything like this exist? Thanks, mike There are ports of the "dd" unix program for windows. It can be used to write giant files with minimal effort. May also be able to find the "dt" utility which is a glorified dd with a more tuneable interface. Trick is not to wait for the bytes to be written as opposed to creating a sparse file more instantaneously. fsx could do it, but dunno if a windows port exists. |
#8
|
|||
|
|||
Utility to write huge files instantly???
mike wrote:
Cydrome Leader wrote: mike wrote: Windows 2000, FAT32 or NTFS. For testing, I want a utility to create BIG files ~1GB on storage media quickly. Copying a big file is not an option, too slow. I don't care what's in the file as long as the OS is happy that it's a "valid" file. Needs to work thru normal OS drive letters and drivers. Should be able to just write the FAT without doing anything to the actual allocation units being allocated??? Anything like this exist? Thanks, mike There are ports of the "dd" unix program for windows. It can be used to write giant files with minimal effort. Thanks, but minimal effort is not nearly as important as minimal time. The version of dd I tried does work, dd if=infile of=outfile seek=2000000 if is 200bytes. set a larger block size, bs=65536 etc. The default 512 byte blocks are slow. but it's only marginally faster than copying a file. I need something that's a thousand times or so faster than a file copy for a 1GB file. Maybe there's a version that doesn't fill the file with zeros, but just allocates the space?? Think about copying 16 gigabytes to a USB1.1 drive...while we're still young. Unless you want sparse files, what you want isn't going to happen. 16GB over USB 1.1 is just nonsense to start with. So, if you want to actually write lots of data, you're going ot have to wait. If you want giant empty files where the insides don't matter, you're not copying large amounts of data in the first place. The same applies to sparse files. |
#9
|
|||
|
|||
Utility to write huge files instantly???
I don't think he wants to take the time to zero (or otherwise write) the
actual file data. ISTR that an (undocumented?) API exists to SetEndOfFile, it is documented. -- Maxim Shatskih, Windows DDK MVP StorageCraft Corporation http://www.storagecraft.com |
#10
|
|||
|
|||
Utility to write huge files instantly???
Cydrome Leader wrote:
mike wrote: Cydrome Leader wrote: mike wrote: Windows 2000, FAT32 or NTFS. For testing, I want a utility to create BIG files ~1GB on storage media quickly. Copying a big file is not an option, too slow. I don't care what's in the file as long as the OS is happy that it's a "valid" file. Needs to work thru normal OS drive letters and drivers. Should be able to just write the FAT without doing anything to the actual allocation units being allocated??? Anything like this exist? Thanks, mike There are ports of the "dd" unix program for windows. It can be used to write giant files with minimal effort. Thanks, but minimal effort is not nearly as important as minimal time. The version of dd I tried does work, dd if=infile of=outfile seek=2000000 if is 200bytes. set a larger block size, bs=65536 etc. The default 512 byte blocks are slow. That's clearly not what he wants. but it's only marginally faster than copying a file. I need something that's a thousand times or so faster than a file copy for a 1GB file. Maybe there's a version that doesn't fill the file with zeros, but just allocates the space?? Think about copying 16 gigabytes to a USB1.1 drive...while we're still young. Unless you want sparse files, what you want isn't going to happen. I suspect you're wrong: he probably just needs either to write (may require linking with ntdll.lib from the Platform SDK - see http://www.informit.com/articles/art...22442&seqNum=3) or to find a utility that uses NtCreateFile to create the file, and set the AllocationSize parameter to the size he wants the file to be. - bill |
Thread Tools | |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Undelete Utility for .avi files which works? | news.rcn.com | Storage (alternative) | 6 | October 18th 07 10:27 PM |
Disappointing hard drive value (was: raw files are HUGE) | timeOday | Storage (alternative) | 35 | March 11th 07 11:24 PM |
Looking for program/utility to slow down the write speed of a CD/DVD burner. | Dan G | Cdr | 0 | April 23rd 06 02:13 PM |
Determine write date of files on dvd? | Goblin28 | Cdr | 1 | February 16th 05 12:24 PM |
Utility to find duplicate files? | Ian Roberts | Storage (alternative) | 5 | October 6th 03 05:33 PM |