A computer components & hardware forum. HardwareBanter

If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

Go Back   Home » HardwareBanter forum » General Hardware & Peripherals » Storage & Hardrives
Site Map Home Register Authors List Search Today's Posts Mark Forums Read Web Partners

Utility to write huge files instantly???



 
 
Thread Tools Display Modes
  #1  
Old January 18th 08, 09:18 PM posted to comp.arch.storage
mike
external usenet poster
 
Posts: 121
Default Utility to write huge files instantly???

Windows 2000, FAT32 or NTFS.
For testing,
I want a utility to create BIG files ~1GB on storage media
quickly.

Copying a big file is not an option, too slow.

I don't care what's in the file as long
as the OS is happy that it's a "valid" file.
Needs to work thru normal OS drive letters and drivers.

Should be able to just write the FAT without doing anything to the
actual allocation units being allocated???

Anything like this exist?
Thanks, mike
--
Return address is VALID!
  #2  
Old January 18th 08, 10:02 PM posted to comp.arch.storage
Maxim S. Shatskih
external usenet poster
 
Posts: 87
Default Utility to write huge files instantly???

Should be able to just write the FAT without doing anything to the
actual allocation units being allocated???


Do you need NTFS support?

If you will go thru the usual filesystem, without patching the metadata
manually - then lseek() to (1GB - 1) and then write 1 byte of zero. The OS will
zero the whole 1GB automatically.

This is recommended by MS's filesystem guys to create the minimally fragmented
file, and is also portable to UNIXen (UNIXen will create a sparse file in this
case though).

--
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation

http://www.storagecraft.com

  #3  
Old January 19th 08, 03:17 AM posted to comp.arch.storage
Bill Todd
external usenet poster
 
Posts: 162
Default Utility to write huge files instantly???

Maxim S. Shatskih wrote:
Should be able to just write the FAT without doing anything to the
actual allocation units being allocated???


Do you need NTFS support?

If you will go thru the usual filesystem, without patching the metadata
manually - then lseek() to (1GB - 1) and then write 1 byte of zero. The OS will
zero the whole 1GB automatically.


I don't think he wants to take the time to zero (or otherwise write) the
actual file data. ISTR that an (undocumented?) API exists to
preallocate a file without writing it, though (useful for file-copy
operations where the final size is known, you want reasonable
contiguity, but don't want the useless overhead of zeroing the output
file before overwriting it with data: IIRC NT maintains 'high water
mark' information similar to VMS's that keeps one from scavenging
previous data on the disk in such cases before the real data is written).

- bill
  #4  
Old January 19th 08, 04:10 AM posted to comp.arch.storage
mike
external usenet poster
 
Posts: 121
Default Utility to write huge files instantly???

Bill Todd wrote:
Maxim S. Shatskih wrote:
Should be able to just write the FAT without doing anything to the
actual allocation units being allocated???


Do you need NTFS support?

If you will go thru the usual filesystem, without patching the metadata
manually - then lseek() to (1GB - 1) and then write 1 byte of zero.
The OS will
zero the whole 1GB automatically.


I don't think he wants to take the time to zero (or otherwise write) the
actual file data. ISTR that an (undocumented?) API exists to
preallocate a file without writing it, though (useful for file-copy
operations where the final size is known, you want reasonable
contiguity, but don't want the useless overhead of zeroing the output
file before overwriting it with data: IIRC NT maintains 'high water
mark' information similar to VMS's that keeps one from scavenging
previous data on the disk in such cases before the real data is written).

- bill


Thanks, but when I said "utility", I meant point/click/done.
I can bungle my way thru a trivial VB6 program, but anything more than
that is a stretch.

I dug around in MSDN and found these:

IStream::SetSize

FileStream.SetLength Method

CFile::SetLength

But I'm not sure of their behavior relative to end of file and not
taking a long time. And I'd be WAY beyond my comfort level trying to
program it.

Think "UTILITY". :-)

mike

--
Return address is VALID!
  #5  
Old January 19th 08, 06:36 AM posted to comp.arch.storage
Cydrome Leader
external usenet poster
 
Posts: 113
Default Utility to write huge files instantly???

mike wrote:
Windows 2000, FAT32 or NTFS.
For testing,
I want a utility to create BIG files ~1GB on storage media
quickly.

Copying a big file is not an option, too slow.

I don't care what's in the file as long
as the OS is happy that it's a "valid" file.
Needs to work thru normal OS drive letters and drivers.

Should be able to just write the FAT without doing anything to the
actual allocation units being allocated???

Anything like this exist?
Thanks, mike


There are ports of the "dd" unix program for windows. It can be used to
write giant files with minimal effort.
  #6  
Old January 19th 08, 11:24 AM posted to comp.arch.storage
mike
external usenet poster
 
Posts: 121
Default Utility to write huge files instantly???

Cydrome Leader wrote:
mike wrote:
Windows 2000, FAT32 or NTFS.
For testing,
I want a utility to create BIG files ~1GB on storage media
quickly.

Copying a big file is not an option, too slow.

I don't care what's in the file as long
as the OS is happy that it's a "valid" file.
Needs to work thru normal OS drive letters and drivers.

Should be able to just write the FAT without doing anything to the
actual allocation units being allocated???

Anything like this exist?
Thanks, mike


There are ports of the "dd" unix program for windows. It can be used to
write giant files with minimal effort.


Thanks, but minimal effort is not nearly as important as minimal time.
The version of dd I tried does work,

dd if=infile of=outfile seek=2000000
if is 200bytes.

but it's only marginally faster
than copying a file. I need something that's a thousand times or so
faster than a file copy for a 1GB file.

Maybe there's a version that doesn't fill the file with zeros, but just
allocates the space?? Think about copying 16 gigabytes to a
USB1.1 drive...while we're still young.

--
Return address is VALID!
  #7  
Old January 19th 08, 12:09 PM posted to comp.arch.storage
Who Cares
external usenet poster
 
Posts: 3
Default Utility to write huge files instantly???

Cydrome Leader wrote:
mike wrote:
Windows 2000, FAT32 or NTFS.
For testing,
I want a utility to create BIG files ~1GB on storage media
quickly.

Copying a big file is not an option, too slow.

I don't care what's in the file as long
as the OS is happy that it's a "valid" file.
Needs to work thru normal OS drive letters and drivers.

Should be able to just write the FAT without doing anything to the
actual allocation units being allocated???

Anything like this exist?
Thanks, mike


There are ports of the "dd" unix program for windows. It can be used to
write giant files with minimal effort.


May also be able to find the "dt" utility which is a glorified dd with a
more tuneable interface. Trick is not to wait for the bytes to be
written as opposed to creating a sparse file more instantaneously.
fsx could do it, but dunno if a windows port exists.

  #8  
Old January 19th 08, 04:05 PM posted to comp.arch.storage
Cydrome Leader
external usenet poster
 
Posts: 113
Default Utility to write huge files instantly???

mike wrote:
Cydrome Leader wrote:
mike wrote:
Windows 2000, FAT32 or NTFS.
For testing,
I want a utility to create BIG files ~1GB on storage media
quickly.

Copying a big file is not an option, too slow.

I don't care what's in the file as long
as the OS is happy that it's a "valid" file.
Needs to work thru normal OS drive letters and drivers.

Should be able to just write the FAT without doing anything to the
actual allocation units being allocated???

Anything like this exist?
Thanks, mike


There are ports of the "dd" unix program for windows. It can be used to
write giant files with minimal effort.


Thanks, but minimal effort is not nearly as important as minimal time.
The version of dd I tried does work,

dd if=infile of=outfile seek=2000000
if is 200bytes.


set a larger block size, bs=65536 etc. The default 512 byte blocks are
slow.


but it's only marginally faster
than copying a file. I need something that's a thousand times or so
faster than a file copy for a 1GB file.

Maybe there's a version that doesn't fill the file with zeros, but just
allocates the space?? Think about copying 16 gigabytes to a
USB1.1 drive...while we're still young.


Unless you want sparse files, what you want isn't going to happen.

16GB over USB 1.1 is just nonsense to start with.

So, if you want to actually write lots of data, you're going ot have to
wait. If you want giant empty files where the insides don't matter, you're
not copying large amounts of data in the first place. The same applies to
sparse files.
  #9  
Old January 19th 08, 05:43 PM posted to comp.arch.storage
Maxim S. Shatskih
external usenet poster
 
Posts: 87
Default Utility to write huge files instantly???

I don't think he wants to take the time to zero (or otherwise write) the
actual file data. ISTR that an (undocumented?) API exists to


SetEndOfFile, it is documented.

--
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation

http://www.storagecraft.com

  #10  
Old January 19th 08, 05:52 PM posted to comp.arch.storage
Bill Todd
external usenet poster
 
Posts: 162
Default Utility to write huge files instantly???

Cydrome Leader wrote:
mike wrote:
Cydrome Leader wrote:
mike wrote:
Windows 2000, FAT32 or NTFS.
For testing,
I want a utility to create BIG files ~1GB on storage media
quickly.

Copying a big file is not an option, too slow.

I don't care what's in the file as long
as the OS is happy that it's a "valid" file.
Needs to work thru normal OS drive letters and drivers.

Should be able to just write the FAT without doing anything to the
actual allocation units being allocated???

Anything like this exist?
Thanks, mike
There are ports of the "dd" unix program for windows. It can be used to
write giant files with minimal effort.

Thanks, but minimal effort is not nearly as important as minimal time.
The version of dd I tried does work,

dd if=infile of=outfile seek=2000000
if is 200bytes.


set a larger block size, bs=65536 etc. The default 512 byte blocks are
slow.


That's clearly not what he wants.



but it's only marginally faster
than copying a file. I need something that's a thousand times or so
faster than a file copy for a 1GB file.

Maybe there's a version that doesn't fill the file with zeros, but just
allocates the space?? Think about copying 16 gigabytes to a
USB1.1 drive...while we're still young.


Unless you want sparse files, what you want isn't going to happen.


I suspect you're wrong: he probably just needs either to write (may
require linking with ntdll.lib from the Platform SDK - see
http://www.informit.com/articles/art...22442&seqNum=3) or to
find a utility that uses NtCreateFile to create the file, and set the
AllocationSize parameter to the size he wants the file to be.

- bill
 




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Undelete Utility for .avi files which works? news.rcn.com Storage (alternative) 6 October 18th 07 10:27 PM
Disappointing hard drive value (was: raw files are HUGE) timeOday Storage (alternative) 35 March 12th 07 12:24 AM
Looking for program/utility to slow down the write speed of a CD/DVD burner. Dan G Cdr 0 April 23rd 06 02:13 PM
Determine write date of files on dvd? Goblin28 Cdr 1 February 16th 05 01:24 PM
Utility to find duplicate files? Ian Roberts Storage (alternative) 5 October 6th 03 05:33 PM


All times are GMT +1. The time now is 12:54 PM.


Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright ©2004-2024 HardwareBanter.
The comments are property of their posters.