What is: block-level and file-level

Hi all,

Can anyone provide or point me to a technical explanation of these
terms? I've read a bunch of papers that use these terms, but they
don't explain them.

Thanks!

On 19 Feb 2004 14:02:46 -0800, (Major
Blunts) wrote:

Hi all,

Can anyone provide or point me to a technical explanation of these
terms? I've read a bunch of papers that use these terms, but they
don't explain them.

Thanks!


You'll probably get alot of lessons on this here ....


File level is exactly what it sounds like, perusing a disk by files.
Files like .xls, .doc, .txt, .tmp, whatever. These are all files that
store data in them in some form. People and applications use files to
store their data.

Blocks are a length of the physical media, be it disk or tape, that
store data. This data is not necessarily coherent to anything but the
controller. Files use blocks to store their data.

I may have made things worse above, let me try another tact.

A file can be of any size within the limits of your filesystem. A 1tb
..cad file is a real possibility. A disk cannot guarentee that there
will be 1tb of contiguous space to just drop this file down on the
platter so all space on a disk is broken into blocks. They can be of
several sizes but I believe the default is 512k.
So the 1tb file can be broken up on the physical medium to fit just
about anywhere, as long as there are enough 512k blocks to make up
1tb.

HTH,

~F

(Major Blunts) wrote in message . com...
Hi all,

Can anyone provide or point me to a technical explanation of these
terms? I've read a bunch of papers that use these terms, but they
don't explain them.

Thanks!

Just to be clear, I'm asking in relation to SAN and NAS devices.
Anything specific to database environments would be great.

Major Blunts wrote:
Hi all,

Can anyone provide or point me to a technical explanation of these
terms? I've read a bunch of papers that use these terms, but they
don't explain them.

Thanks!

It's pretty simple really.

Block level I/O sees the disk as a sequential sequence from blocks with no
additional structure, i.e. you ask to read or write a sequence of block
addresses.

File level I/O sees the disk as a structured device with directories and
files, so your application opens a directory or a file and performs I/O
within the constraints of the way the filesystem data is layed out on the
disk. Of course a request to read "file x" ultimately becomes a block level
request to read certain disk blocks on the drive, but the translation of the
file I/O request into corresponding block I/O requests is handled inside the
operating system.


--
Nik Simpson

"Major Blunts" wrote in message
om...
(Major Blunts) wrote in message
. com...
Hi all,

Can anyone provide or point me to a technical explanation of these
terms? I've read a bunch of papers that use these terms, but they
don't explain them.

Thanks!

Just to be clear, I'm asking in relation to SAN and NAS devices.
Anything specific to database environments would be great.

A block-level interface (between a 'host' system and storage) views the
storage as a linear range of equal-sized 'disk blocks': the host asks the
storage to read or write one or more consecutive blocks per request.
Traditional storage maps these blocks linearly to the actual disk blocks (or
perhaps 'stripes' them across multiple disks with a simple algorithm);
'virtualized' storage may map the storage blocks seen by the host
arbitrarily to disk locations, even across multiple storage boxes.

A file-level interface (between a 'client' system and a 'file server') views
the storage as a group of files, organized into the traditional hierarchical
structure by directories. Aside from maintenance operations (open, close,
list directory, etc.), the client asks the server to read or write a
consecutive range of bytes within a particular file in each request, and the
server maps this request to its storage much like a client system maps local
requests to its local storage.

With a block-level interface, you need to use a 'SAN file system' (or a
similarly-integrated database mechanism such as Oracle Parallel Server) on
the hosts if multiple hosts want to share the same files on the storage,
because the host/storage interface is too low-level to be aware of the
file-level coordination required and the hosts must thus communicate among
themselves to provide it (if no portion of the storage is shared among
multiple hosts, they all can just treat their portion like a local disk,
with no inter-host coordination required). With a file-level interface,
that coordination can take place on the server (very much like access
coordination on a stand-alone system between applications that access a
single file concurrently occurs); however, that precludes caching any data
locally on the clients, so high-performance NAS implementations extend the
interface to allow this and thus make it more complex, some even requiring
similar inter-host coordination to that required to share files on a
block-level SAN - and sophisticated NAS approaches can even map files across
multiple NAS servers.

Thus a simple SAN maps what appear to the host to be 'disk blocks' quite
directly to actual disk blocks at the storage, with little loss of bandwidth
or increase in latency over comparable local storage. A simple NAS just
exports client requests to a server where the requested file-level operation
is performed, incurring somewhat greater latency than would occur accessing
local storage (and possibly compromising bandwidth as well). But a simple
NAS allows easy file-sharing among multiple hosts, while a simple SAN does
not.

- bill

"Nik Simpson" wrote in message
File level I/O sees the disk as a structured device with directories and
files, so your application opens a directory or a file and performs I/O
within the constraints of the way the filesystem data is layed out on the
disk.

Isn't that how most applications work? I thought that applications
don't know if the file they are accessing is on SCSI (block) storage
or NAS (NFS, SMB, etc.) storage?

Of course a request to read "file x" ultimately becomes a block level
request to read certain disk blocks on the drive, but the translation of the
file I/O request into corresponding block I/O requests is handled inside the
operating system.

Using NFS as an example, what do you mean by:

the translation of the
file I/O request into corresponding block I/O requests is handled inside the
operating system.

Thanks!

Major Blunts wrote:
"Nik Simpson" wrote in message
File level I/O sees the disk as a structured device with directories
and files, so your application opens a directory or a file and
performs I/O within the constraints of the way the filesystem data
is layed out on the disk.

Isn't that how most applications work? I thought that applications
don't know if the file they are accessing is on SCSI (block) storage
or NAS (NFS, SMB, etc.) storage?



Correct, *most* applications work via the filesystem. In the past, (and
perhaps still today) some applications, notably databases worked at the
block level for performance reasons.



Of course a request to read "file x" ultimately becomes a block level
request to read certain disk blocks on the drive, but the
translation of the file I/O request into corresponding block I/O
requests is handled inside the operating system.

Using NFS as an example, what do you mean by:

the translation of the
file I/O request into corresponding block I/O requests is handled
inside the operating system.


Application X running on the NFS client opens a file reads 20bytes from a
file called "foo" on the NFS mounted directory.

The NFS client software turns this into NFS protocl requests to the NFS
server. The requests (viewed somewhat simply) would be:

1. Open file "foo" and give me back an open file descriptor for future use

2. Return the first 20bytes of the file referenced by the aforementioned
open file descriptor

On the NFS The Open file request means going through the file system to find
the file "foo" and its associated directory entry. The directory entry in
the filesystem contains a list of physical disk blocks that contain file
"foo" this enables the OS to find the correct block addresses needed to
access the contents of file "foo"

The request for the first twenty bytes of the file is satisfied by find the
block address (from the directory entry) of the first disk block used by the
file and then performing block level I/O to read that block into memory.
Once the block is in memory the the requested bytes are packaged up and sent
back to the NFS client which passes them on to the application.

So, in actuality, *ALL* disk I/O is block I/O at some point, you just don't
have to deal with keeping track of which blocks are used each file because
the OS and the filesystem hide that complexity from you by overlaying the
abstract concept of a filesystem on top of the block device we call a disk.


--
Nik Simpson

(Major Blunts) wrote in message . com...
"Nik Simpson" wrote in message
File level I/O sees the disk as a structured device with directories and
files, so your application opens a directory or a file and performs I/O
within the constraints of the way the filesystem data is layed out on the
disk.

Isn't that how most applications work? I thought that applications
don't know if the file they are accessing is on SCSI (block) storage
or NAS (NFS, SMB, etc.) storage?


That's correct. Typically applications just see the files as
presented by the OS's open()/read()/whatever interface (substitute the
correct functions for your favorite OS).


Of course a request to read "file x" ultimately becomes a block level
request to read certain disk blocks on the drive, but the translation of the
file I/O request into corresponding block I/O requests is handled inside the
operating system.

Using NFS as an example, what do you mean by:

the translation of the
file I/O request into corresponding block I/O requests is handled inside the
operating system.


Somebody has to translate a collection of disk blocks into a file
system. That's an OS somewhere.

For a block (SAN) device, the local OS does it, just as it would for a
locally attached disk drive.

For a file (NAS) device, the OS physically associated with the storage
device does the translation, and some other protocol, like NFS, is
used from the actual client machines to request data from the file
system. An NFS request will typically be a packet containing
something like, "read 100 bytes from offset 12345 from file #6789 on
volume xyz". The response is the data requested. The NFS server on
the NAS device nominally translates that into a series of open() and
read() calls and sends back a packet with the results (in practical
terms, the file service software may use a lower level set of
interfaces into the file system for better performance).

In the simplest terms, a block (SAN) device is a disk drive (or
several) in a fancy and excessively smart box. From the OS's point of
view, no different than a few disk drives at the end of an IDE or SCSI
cable. A file (NAS) device is just a file server, not really any
different from a share on a machine in a Windows workgroup. And of
course the distinction (from a *sales* perspective) is not clear-cut.
Many SAN devices can also be NAS devices - it just takes some place to
run the code (in fact, at least one of the big SAN vendors will sell
you a plug in board which runs a complete OS to provide the "NAS"
function).