shared disk, two hosts (*without clustering* ) in linux

Hello,

i have been searching for a solution to have a shared disk (f.ex. SAN )
visible across two (or perhaps more) hosts, both of them able to read
or write on the disk [better if it is a filesystem in it, though raw
disk is also good]. The data must be perfectly synchronized between
both hosts. (If no f.s. this should be managed by my application, but
then i think that perhaps an option for no-cacheing data is needed from
the O.S.).
I have 'googled' for it but i am unable to find anything useful. O.S.
is linux.

i think "lustre", "cluster file system (CFS)" or other clustering
schema, is not an option here, since i would need the two
(LAN-connected ) hosts sharing the disk be one apart the other and
running different processes, because of different functionality.

any thoughts about it?.
TIA, any help appreciated.

--
Fco. Javier Cobas-Seoane
Madrid (Spain)

javier_cobas wrote:
Hello,

i have been searching for a solution to have a shared disk (f.ex. SAN )
visible across two (or perhaps more) hosts, both of them able to read
or write on the disk [better if it is a filesystem in it, though raw
disk is also good]. The data must be perfectly synchronized between
both hosts. (If no f.s. this should be managed by my application, but
then i think that perhaps an option for no-cacheing data is needed from
the O.S.).
I have 'googled' for it but i am unable to find anything useful. O.S.
is linux.

i think "lustre", "cluster file system (CFS)" or other clustering
schema, is not an option here, since i would need the two
(LAN-connected ) hosts sharing the disk be one apart the other and
running different processes, because of different functionality.

any thoughts about it?.
TIA, any help appreciated.

--
Fco. Javier Cobas-Seoane
Madrid (Spain)

this is a fairly common problem, and occurs absolutely and every day in
any shared database application, solved by breaking data writes into
atomic units, locking access whilst these atomic unit writes are carried
out, and keeping a log of all atomic transactions on a separate disk, so
you can roll back to a backup and roll forward the transactions.

Without knowing what is going ON to the shared data area, its hard to
say exactly what approach to take, but implementing a database server
add accessing it through normal APIs from your applications would be one
way.

Thanks.
The shared data area can be modified by some processes running on both
hosts.
They will be "normal" files, though somewhat big.
The idea of using a shared disk (via SAN [with HBA, controllers, ...
etc every path redundant]) is for having a means of quick and reliable
data communication between processes in both hosts, instead of "shared
memory". I have done it with "raw" shared disk on an hp's EVA3000 under
"Tru64 Unix" O.S. The SAN put the data quickly on the other host. It
was fast and efficient for our needs.

Now i wonder if that can be done under Linux also, *better* if it is
under a f.s. structure instead of raw disk. Linux distribution will be
(perhaps) Suse L. Ent. Server or Redhat E. S.

Javier.
--
F. Javier Cobas-Seoane
Madrid (Spain)

javier_cobas wrote:
Thanks.
The shared data area can be modified by some processes running on both
hosts.
They will be "normal" files, though somewhat big.
The idea of using a shared disk (via SAN [with HBA, controllers, ...
etc every path redundant]) is for having a means of quick and reliable
data communication between processes in both hosts, instead of "shared
memory". I have done it with "raw" shared disk on an hp's EVA3000 under
"Tru64 Unix" O.S. The SAN put the data quickly on the other host. It
was fast and efficient for our needs.

Now i wonder if that can be done under Linux also, *better* if it is
under a f.s. structure instead of raw disk. Linux distribution will be
(perhaps) Suse L. Ent. Server or Redhat E. S.

Javier.
--
F. Javier Cobas-Seoane
Madrid (Spain)

My only point is that you should not write to a shared disk and expect
to make it bombproof. Use of a decent protocol with file locking may
make this work, but its better to build a shim over the file system that
handles access to it, and dream up a protocol to 'write' through it via
networked connections, so you can ensure file system integrity by having
just one system finally wrote to the file.

In comp.os.linux.misc javier_cobas :
Thanks.
The shared data area can be modified by some processes running on both
hosts.
They will be "normal" files, though somewhat big.
The idea of using a shared disk (via SAN [with HBA, controllers, ...
etc every path redundant]) is for having a means of quick and reliable
data communication between processes in both hosts, instead of "shared
memory". I have done it with "raw" shared disk on an hp's EVA3000 under
"Tru64 Unix" O.S. The SAN put the data quickly on the other host. It
was fast and efficient for our needs.

It's called TruCluster. IMHO the most advanced *nix cluster
technology, we'll see what HP makes out of it?

Now i wonder if that can be done under Linux also, *better* if it is
under a f.s. structure instead of raw disk. Linux distribution will be
(perhaps) Suse L. Ent. Server or Redhat E. S.

Checkout GFS, sounds like what you want. IIRC it has been
completely GPL'ed by rh, so one could probably roll out his own.
But if you need a certified solution you should get it from rh.

Good luck

--
Michael Heiming (X-PGP-Sig GPG-Key ID: EDD27B94)
mail: echo | perl -pe 'y/a-z/n-za-m/'
#bofh excuse 128: Power Company having EMP problems with
their reactor

On 30 Jan 2006 09:56:47 -0800, "javier_cobas"
wrote:

Hello,

i have been searching for a solution to have a shared disk (f.ex. SAN )
visible across two (or perhaps more) hosts, both of them able to read
or write on the disk [better if it is a filesystem in it, though raw
disk is also good]. The data must be perfectly synchronized between
both hosts. (If no f.s. this should be managed by my application, but
then i think that perhaps an option for no-cacheing data is needed from
the O.S.).
I have 'googled' for it but i am unable to find anything useful. O.S.
is linux.

i think "lustre", "cluster file system (CFS)" or other clustering
schema, is not an option here, since i would need the two
(LAN-connected ) hosts sharing the disk be one apart the other and
running different processes, because of different functionality.

any thoughts about it?.
TIA, any help appreciated.


I may not be understanding exactly what you're asking but it seems to
me a clustered file system is absolutely required. In the case of a
single node not controlling data access (like NAS for instance) the
hosts involved in writes have to be able to share lock information.
I'm not aware of anything other than a filesystem that will do this,
and a clustered filesystem at that.
Even in NFS the locks are advisory only.

Running different processes should not be an issue for any clustered
file system out there today, and as long as they're LAN connected they
can easily share lock and meta data.

~F

The Natural Philosopher wrote:
javier_cobas wrote:

i have been searching for a solution to have a shared disk (f.ex. SAN )
visible across two (or perhaps more) hosts, both of them able to read
or write on the disk [better if it is a filesystem in it, though raw
disk is also good]. The data must be perfectly synchronized between
both hosts. (If no f.s. this should be managed by my application, but
then i think that perhaps an option for no-cacheing data is needed from
the O.S.). I have 'googled' for it but i am unable to find anything
useful. O.S. is linux.

i think "lustre", "cluster file system (CFS)" or other clustering
schema, is not an option here, since i would need the two
(LAN-connected ) hosts sharing the disk be one apart the other and
running different processes, because of different functionality.

this is a fairly common problem, and occurs absolutely and every day in
any shared database application, solved by breaking data writes into
atomic units, locking access whilst these atomic unit writes are carried
out, and keeping a log of all atomic transactions on a separate disk, so
you can roll back to a backup and roll forward the transactions.

Without knowing what is going ON to the shared data area, its hard to
say exactly what approach to take, but implementing a database server
add accessing it through normal APIs from your applications would be one
way.

This is known as the readers and writers problem. See:

http://www.acm.org/classics/feb96/

and you will get an idea of how long ago it was solved.

--
"The power of the Executive to cast a man into prison without
formulating any charge known to the law, and particularly to
deny him the judgement of his peers, is in the highest degree
odious and is the foundation of all totalitarian government
whether Nazi or Communist." -- W. Churchill, Nov 21, 1943

javier_cobas wrote:

Hello,

i have been searching for a solution to have a shared disk (f.ex. SAN )
visible across two (or perhaps more) hosts, both of them able to read
or write on the disk [better if it is a filesystem in it, though raw
disk is also good]. The data must be perfectly synchronized between
both hosts. (If no f.s. this should be managed by my application, but
then i think that perhaps an option for no-cacheing data is needed from
the O.S.).
I have 'googled' for it but i am unable to find anything useful. O.S.
is linux.

i think "lustre", "cluster file system (CFS)" or other clustering
schema, is not an option here, since i would need the two
(LAN-connected ) hosts sharing the disk be one apart the other and
running different processes, because of different functionality.

any thoughts about it?.
TIA, any help appreciated.

--
Fco. Javier Cobas-Seoane
Madrid (Spain)

I think that the SCSI command set includes Reserve and Release,
which can be used as a volume-level lock mechanism. The mainframe
OS MVS and its successors used this in the past. I think you'll
have to do without disk caching, though.

--
-- Marten Kemp
(Fix name and ISP to reply)
-=-=-
.... "There are no problems that cannot be solved by the judicious use of
high explosives" -- British Commando quote, circa WWII.
* TagZilla 0.059 * http://tagzilla.mozdev.org

I don't understand why you don't consider a cluster file system an
option here ? It sounds like a cfs is just what you need. On linux
I would consider the major options:

o Oracle's ocfs2 http://oss.oracle.com/projects/ocfs2/ which
was recently added to the standard linux kernel, should soon
be certified for Oracle 10g, and probably will be a major player
in this area.

o IBM GPFS http://www-03.ibm.com/servers/eserve...ware/gpfs.html
which I'm sure is the most mature option on linux.. (but I
guess I might be biased).

o GFS from Red Hat / Sistina, which I haven't yet tested ...
mostly because it looks too complicated.

Lustre I think is more of a file system for larger clusters / high
performance computing, and not for a 2-node server cluster.



-jf

Thanks, and also to all other responses so far.

Michael Heiming wrote
..../...
I have done it with "raw" shared disk on an hp's EVA3000 under
"Tru64 Unix" O.S. The SAN put the data quickly on the other host. It
was fast and efficient for our needs.

It's called TruCluster. IMHO the most advanced *nix cluster
technology, we'll see what HP makes out of it?
..../...
That's right, but for that particular case it was done without
TruCluster. At that time it was only needed transferring "big" data
structures between hosts (as an alternative to IPC messaging).
Then there were two possibilities, one using filesystem (UFS, AdvFs)
and the other using "raw" disk, write the big data structures to disk
and then signaling the data transfer completion to the other host (via
sys V or POSIX IPC messages with mention to the size and checksum of
the data transferred).
Since it was less overhead without f.s., "raw" disk was selected then.

This time however there are more issues, i think f.s. will be needed
eventually, but each host should maintain its own processes for itself,
apart from the other hosts. Suse Linux or Red Hat L. surely will be the
OS.

Checkout GFS, .../...
I am doing it now. I was said that GFS could also be used without
clustering...