Re: OpenGFS with OpenDLM problem

Hi Ted,

First, please make sure that you are using the lastest cvs source.
Second, before mount OpenGFS, make sure DLM works properly (use
OpenDLM/src/user/test/simpleclient)
Third, "modprobe opendlm debug=3D1", then mount OpenGFS. And mail me the
output of "dmesg".

Best Regards,
Stan

=20
On Thu, 2004-03-25 at 17:57, =E5=BC=B5=E5=AE=B6=E7=A2=A9 wrote:
> Hi Stan
>=20
> dlmdu  can work after adding the following lines you suggest
>=20
> apiauth	heartbeat gid=3Droot uid=3Droot
> apiauth default gid=3Droot uid=3Droot
>=20
> and cust-a emitted the following message
> =3D=3D=3D
> [root@cust-a root]# /usr/sbin/dlmdu -C /etc/dlm.conf
> DLMK Major/Minor(0/1/0x1) Time(17:35:42) Date(Mar 24 2004)
> Opening configuration file [/etc/dlm.conf]
> Opened configuration file.
> [root@cust-a root]# Attempt to load DLM module, cmd [modprobe -a -k haD=
LM haDLM_major_number=3D250 haDLM_name=3DhaDLM haDLM_admin_minor=3D0 haDL=
M_locks_minor=3D1 haDLM_poll_minor=3D2]
> Loaded DLM module!
> Opened [/dev/haDLM/admin], filedes [0]
> 1st thread, pid [1265]
> Created thread, id [16386]
> 2nd thread, pid/tid [1269/16386]
> Node have [1] nodeq elements queued
> Local node is [cust-a]
> Node have [2] nodeq elements queued
> Node have [3] nodeq elements queued
> Wrote code/size[0/12], blocks so far [1]
> compare [cust-a] and [cust-a]
> found number/name[1/cust-a/192.168.0.201]
> compare [cust-a] and [cust-b]
> compare [cust-b] and [cust-b]
> found number/name[2/cust-b/192.168.0.202]
> compare [cust-a] and [cust-a]
> found number/name[1/cust-a/192.168.0.201]
> Wrote code/size[2/12], blocks so far [2]
> =3D=3D=3D
>=20
> LDM seems working fine now.
> thanks alot .
>=20
> after "modprobe opendlm" , I can  use my share storage with OpenGFS.
> But I met another problem.
>=20
> when cust-b mounted the shared storage, I tried to use "df" to see the=20
> partiton useage ,
> the command "df" just hang there.
> And on cust-a, I tried to mount the shared storage and "mount" also han=
g=20
> there too.
>=20
> this is really strange.
>=20
> Best Regards
> Ted Chang
>=20
>=20
> Stanley Wang wrote:
>=20
> >add following lines to ha.cf:
> >
> >apiauth	heartbeat gid=3Droot pid=3Droot
> >apiauth default gid=3Droot pid=3Droot
> >
> >And try again.
> >
> >Hope that helps,
> >Stan
> >
> >
> >On Thu, 2004-03-25 at 16:49, =E5=BC=B5=E5=AE=B6=E7=A2=A9 wrote:
> > =20
> >
> >>Hi Stan
> >>
> >>Thank you for spending time help me to solve this problem.
> >>
> >>I tried to "tail -f /var/log/ha-log" while starting " /usr/sbin/dlmdu=
 -C=20
> >>/etc/dlm.conf"
> >>and  two errors log in this file ,
> >>=3D=3D=3D=3D
> >>heartbeat: 2004/03/25_16:16:49 WARN: Client [12494] pid 12494 failed=20
> >>authorization [no default client auth]
> >>heartbeat: 2004/03/25_16:16:49 ERROR: api_process_registration_msg:=20
> >>cannot add client(1)
> >>=3D=3D=3D=3D
> >>
> >>and attached my config files : ha.cf, authkeys, dlm.conf
> >>and the spec files for rpm ,maybe my "./configure " optitions are not=
=20
> >>correct enough.
> >>
> >>Best Regrads,
> >>Ted Chang
> >>
> >>
> >>Stanley Wang wrote:
> >>
> >>   =20
> >>
> >>>Ted,
> >>>
> >>>I haven't tried to run OpenDLM on RH9, but I met some simliar proble=
m on
> >>>RHEL3, and resolved by disabling NPTL.
> >>>
> >>>And I read the error log again, I found that there might be some iss=
ues
> >>>in you config file (ha.cf). Please double check the API Authenticati=
on
> >>>part (at the end of this file) of ha.cf. Or you can mail your config
> >>>file to me, I could help you to verify it.
> >>>
> >>>Hope that helps,
> >>>Stan
> >>>
> >>>On Thu, 2004-03-25 at 15:41, =E5=BC=B5=E5=AE=B6=E7=A2=A9 wrote:
> >>>=20
> >>>
> >>>     =20
> >>>
> >>>>Hi Stan
> >>>>
> >>>>thanks for your suggetions.
> >>>>But it seems still not working .
> >>>>
> >>>>if "export LD_ASSUME_KERNEL=3D2.2"
> >>>>I met such message about shared library
> >>>>=3D=3D=3D=3D=3D
> >>>>[root@cust-a opendlm-0.9.2]# /usr/sbin/dlmdu -C /etc/dlm.conf
> >>>>/usr/sbin/dlmdu: error while loading shared libraries: libpthread.s=
o.0:=20
> >>>>cannot open shared object file: No such file or directory
> >>>>[root@cust-a opendlm-0.9.2]#
> >>>>=3D=3D=3D=3D=3D
> >>>>
> >>>>Best Regards
> >>>>Ted Chang
> >>>>
> >>>>
> >>>>
> >>>>Stanley Wang wrote:
> >>>>
> >>>>  =20
> >>>>
> >>>>       =20
> >>>>
> >>>>>OK, you problem caused by NPTL in RH9. Disable NPTL by "export
> >>>>>LD_ASSUME_KERNEL=3D2.2".
> >>>>>
> >>>>>Hope that helps,
> >>>>>Stan
> >>>>>
> >>>>>On Wed, 2004-03-24 at 18:22, =E5=BC=B5=E5=AE=B6=E7=A2=A9 wrote:
> >>>>>
> >>>>>
> >>>>>    =20
> >>>>>
> >>>>>         =20
> >>>>>
> >>>>>>Hi  Stan
> >>>>>>
> >>>>>>thanks for your responding so soon.
> >>>>>>sorry I did not describe the packages installed in the machine, c=
ust-a.
> >>>>>>
> >>>>>>here is the detail:
> >>>>>>nodes: cust-a, cust-b , all installed RedHat 9.0
> >>>>>>opengfs and opendlm from the cvs tree
> >>>>>>heartbeat-stonith-1.2.0-1.rh.9
> >>>>>>heartbeat-1.2.0-1.rh.9
> >>>>>>heartbeat-pils-1.2.0-1.rh.9
> >>>>>>libnet-1.1.1
> >>>>>>
> >>>>>>and the following is "ps ax" about heartbeat
> >>>>>>30599 ?        SL     0:00 heartbeat: heartbeat: master control p=
rocess
> >>>>>>30602 ?        SL     0:00 heartbeat: heartbeat: FIFO reader
> >>>>>>30603 ?        SL     0:00 heartbeat: heartbeat: write: bcast eth=
0
> >>>>>>30604 ?        SL     0:00 heartbeat: heartbeat: read: bcast eth0
> >>>>>>
> >>>>>>and attached the files, ha.cf and dlm.conf[root@cust-a tmp]#=20
> >>>>>>/usr/sbin/dlmdu -C /etc/dlm.conf
> >>>>>>when stating dlmdu, it emiited such message
> >>>>>>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> >>>>>>DLMK Major/Minor(0/1/0x1) Time(17:35:42) Date(Mar 24 2004)
> >>>>>>Opening configuration file [/etc/dlm.conf]
> >>>>>>Opened configuration file.
> >>>>>>[root@cust-a tmp]# Attempt to load DLM module, cmd [modprobe -a -=
k haDLM=20
> >>>>>>haDLM_major_number=3D250 haDLM_name=3DhaDLM haDLM_admin_minor=3D0=
=20
> >>>>>>haDLM_locks_minor=3D1 haDLM_poll_minor=3D2]
> >>>>>>Loaded DLM module!
> >>>>>>Opened [/dev/haDLM/admin], filedes [0]
> >>>>>>1st thread, pid [30803]
> >>>>>>2nd thread, pid/tid [30807/16386]
> >>>>>>Created thread, id [16386]
> >>>>>>Cannot get status for node cust-a from Heartbeat
> >>>>>>dlmdu: ../../../src/user/common/dlm_heartbeat.c:428: find_local:=20
> >>>>>>Assertion `!"Cannot get node status"' failed.
> >>>>>>Caught signal [6] count [1]
> >>>>>>Closed admin file [0, /dev/haDLM/admin]
> >>>>>>Shutting down cluster interface.
> >>>>>>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> >>>>>>
> >>>>>>and also find some errors in ha-log , and attached ha-log and ha-=
debug
> >>>>>>
> >>>>>>below is "lsmod" result
> >>>>>>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D
> >>>>>>[root@cust-a tmp]# lsmod
> >>>>>>Module                  Size  Used by    Not tainted
> >>>>>>dlmdk.core            199616   0  (autoclean)
> >>>>>>cccp                   33624   0  (autoclean) [dlmdk.core]
> >>>>>>dlmdk.base             10324   0  (autoclean) [dlmdk.core cccp]
> >>>>>>nfsd                   81328   8  (autoclean)
> >>>>>>lockd                  59472   1  (autoclean) [nfsd]
> >>>>>>sunrpc                 88124   1  (autoclean) [nfsd lockd]
> >>>>>>e1000                  60576   1
> >>>>>>3c59x                  30800   1
> >>>>>>qla2200               229536   0
> >>>>>>sd_mod                 12688   0
> >>>>>>scsi_mod              102804   2  [qla2200 sd_mod]
> >>>>>>[root@cust-a tmp]#
> >>>>>>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
> >>>>>>
> >>>>>>Thanks for your help.
> >>>>>>
> >>>>>>Regards
> >>>>>>Ted Chang
> >>>>>>
> >>>>>>
> >>>>>>Stanley Wang wrote:
> >>>>>>
> >>>>>> =20
> >>>>>>
> >>>>>>      =20
> >>>>>>
> >>>>>>           =20
> >>>>>>
> >>>>>>>Yes, there are some bugs in old version of heartbeat. You could =
try heartbeat 1.2 or later.
> >>>>>>>
> >>>>>>>Best Regards,
> >>>>>>>Stan
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>   =20
> >>>>>>>
> >>>>>>>        =20
> >>>>>>>
> >>>>>>>             =20
> >>>>>>>
> >>>>>>_________________________________________________________________=
_____
> >>>>>>NODECOUNT 2
> >>>>>>1 cust-a 192.168.0.201
> >>>>>>2 cust-b 192.168.0.202
> >>>>>>DLMNAME haDLM
> >>>>>>DLMMAJOR 250
> >>>>>>DLMCMGR heartbeat
> >>>>>>DLMADMIN admin 0
> >>>>>>DLMLOCKS locks 1
> >>>>>>
> >>>>>>_________________________________________________________________=
_____
> >>>>>>debugfile /var/log/ha-debug
> >>>>>>logfile /var/log/ha-log
> >>>>>>logfacility local0
> >>>>>>bcast eth0
> >>>>>>node cust-a
> >>>>>>node cust-b
> >>>>>>
> >>>>>>_________________________________________________________________=
_____
> >>>>>>heartbeat: 2004/03/24_18:07:55 WARN: Deprecated 'legacy' auto_fai=
lback option selected.
> >>>>>>heartbeat: 2004/03/24_18:07:55 WARN: Please convert to 'auto_fail=
back on'.
> >>>>>>heartbeat: 2004/03/24_18:07:55 WARN: See documentation for conver=
sion details.
> >>>>>>heartbeat: 2004/03/24_18:07:55 info: **************************
> >>>>>>heartbeat: 2004/03/24_18:07:55 info: Configuration validated. Sta=
rting heartbeat 1.2.0
> >>>>>>heartbeat: 2004/03/24_18:07:55 info: heartbeat: version 1.2.0
> >>>>>>heartbeat: 2004/03/24_18:07:55 info: Heartbeat generation: 22
> >>>>>>heartbeat: 2004/03/24_18:07:55 info: UDP Broadcast heartbeat star=
ted on port 694 (694) interface eth0
> >>>>>>heartbeat: 2004/03/24_18:07:55 info: pid 30784 locked in memory.
> >>>>>>heartbeat: 2004/03/24_18:07:55 info: Local status now set to: 'up=
'
> >>>>>>heartbeat: 2004/03/24_18:07:56 info: pid 30787 locked in memory.
> >>>>>>heartbeat: 2004/03/24_18:07:56 info: pid 30788 locked in memory.
> >>>>>>heartbeat: 2004/03/24_18:07:56 info: pid 30789 locked in memory.
> >>>>>>heartbeat: 2004/03/24_18:07:56 info: Link cust-b:eth0 up.
> >>>>>>heartbeat: 2004/03/24_18:07:56 info: Status update for node cust-=
b: status active
> >>>>>>heartbeat: 2004/03/24_18:07:56 info: Local status now set to: 'ac=
tive'
> >>>>>>heartbeat: 2004/03/24_18:07:56 info: Link cust-a:eth0 up.
> >>>>>>heartbeat: 2004/03/24_18:07:56 info: Running /etc/ha.d/rc.d/statu=
s status
> >>>>>>heartbeat: 2004/03/24_18:07:56 info: No local resources [/usr/lib=
/heartbeat/ResourceManager listkeys cust-a] to acquire.
> >>>>>>heartbeat: 2004/03/24_18:08:16 WARN: Client [30807] pid 30807 fai=
led authorization [no default client auth]
> >>>>>>heartbeat: 2004/03/24_18:08:16 ERROR: api_process_registration_ms=
g: cannot add client(1)
> >>>>>>
> >>>>>>_________________________________________________________________=
_____
> >>>>>>heartbeat: 2004/03/24_18:07:56 debug: notify_world: setting SIGCH=
LD Handler to SIG_DFL
> >>>>>>heartbeat: 2004/03/24_18:07:56 debug: StartNextRemoteRscReq(): ch=
ild count 1
> >>>>>>heartbeat: 2004/03/24_18:08:16 debug: SO_PEERCRED returned [30807=
, (0:0)]
> >>>>>>heartbeat: 2004/03/24_18:08:16 debug: Verifying authentication: c=
red.uid=3D0 cred.gid=3D0
> >>>>>>heartbeat: 2004/03/24_18:08:16 debug: Verifying authentication: u=
idptr=3D0x80929d8 gidptr=3D0x0
> >>>>>>heartbeat: 2004/03/24_18:08:16 debug: SO_PEERCRED returned [30807=
, (0:0)]
> >>>>>>heartbeat: 2004/03/24_18:08:16 debug: Verifying authentication: c=
red.uid=3D0 cred.gid=3D0
> >>>>>>heartbeat: 2004/03/24_18:08:16 debug: Verifying authentication: u=
idptr=3D0x0 gidptr=3D0x80929f8
> >>>>>> =20
> >>>>>>      =20
> >>>>>>
> >>>>>>           =20
> >>>>>>
> >>_____________________________________________________________________=
_
> >>debugfile /var/log/ha-debug
> >>logfile /var/log/ha-log
> >>logfacility local0
> >>bcast eth0
> >>node cust-a
> >>node cust-b
> >>
> >>_____________________________________________________________________=
_
> >>auth 1
> >>1 crc
> >>
> >>_____________________________________________________________________=
_
> >>NODECOUNT 2
> >>1 cust-a 192.168.0.201
> >>2 cust-b 192.168.0.202
> >>DLMNAME haDLM
> >>DLMMAJOR 250
> >>DLMCMGR heartbeat
> >>DLMADMIN admin 0
> >>DLMLOCKS locks 1
> >>
> >>_____________________________________________________________________=
_
> >>%define debug_package %{nil}
> >>%define KERNEL_VERSION %(uname -r)
> >>Name: opendlm
> >>Version: 0.9.2
> >>Release: 1
> >>Summary: OpenDLM is a distributed lock manager
> >>Copyright: GPL
> >>Vendor: The OpenDLM Project
> >>Group: System Environment/Kernel
> >>Source: %{name}-%{version}.tar.gz
> >>BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}
> >>URL: http://opendlm.sourceforge.net/
> >>BuildRequires: gcc, sed, perl
> >>Packager: Ted Chang <[email protected]>
> >>
> >>%description
> >>  OpenDLM is a true distributed lock manager.  Knowledge of each lock=
 is
> >>distributed among "interested" compute nodes within a cluster of comp=
uters,
> >>and each compute node contains and executes a complete set of logic f=
or
> >>requesting and granting locks among the nodes.
> >>
> >>%package utils
> >>Group: System Environment/Base
> >>Summary: OpenDLM utilities
> >>
> >>%description utils
> >>  OpenDLM utilities.
> >>
> >>
> >>%package modules-%{KERNEL_VERSION}
> >>Group: System Environment/Kernel
> >>Summary: OpenDLM kernel modules for %{KERNEL_VERSION}
> >>
> >>%description  modules-%{KERNEL_VERSION}
> >>  OpenDLM kernel modules for %{KERNEL_VERSION}
> >>
> >>
> >>%prep
> >>%setup -q=20
> >>./bootstrap
> >>
> >>
> >>%build
> >>./configure --prefix=3D/usr \
> >>             --with-linux_srcdir=3D/lib/modules/%{KERNEL_VERSION}/bui=
ld \
> >>             --with-heartbeat-includes=3D/usr/src/redhat/BUILD/heartb=
eat-1.2.0/include \
> >>             --enable-smp
> >>make
> >>
> >>
> >>             #--with-heartbeat_includes=3D/usr/include/heartbeat \
> >>%install
> >>make DESTDIR=3D$RPM_BUILD_ROOT docdir=3D%{_docdir}/%{name}-%{version}=
-%{release} install
> >>
> >>
> >>%post modules-%{KERNEL_VERSION}
> >>/sbin/depmod -a
> >>
> >>%postun modules-%{KERNEL_VERSION}
> >>/sbin/depmod -a
> >>
> >>
> >>%clean
> >>rm -rf $RPM_BUILD_ROOT
> >>#rm -rf $RPM_BUILD_DIR/%{name}
> >>
> >>
> >>%files utils
> >>%defattr(-,root,root,755)
> >>/usr/include/*
> >>/usr/lib/*
> >>/usr/sbin/*
> >>%doc %{_docdir}/*
> >>
> >>
> >>%files modules-%{KERNEL_VERSION}
> >>%defattr(-,root,root)
> >>/lib/modules/%{KERNEL_VERSION}/*
> >>
> >>
> >>%changelog
> >>* Wed Mar 24 2004 Ted Chang <[email protected]>
> >>- initial RPM release
> >>- from opendlm cvs tree
> >>
> >>_____________________________________________________________________=
_
> >>%define debug_package %{nil}
> >>%define KERNEL_VERSION %(uname -r)
> >>Name: opengfs
> >>Version: 0.3.0
> >>Release: 1
> >>Summary: OpenGFS is a clustered filesystem
> >>Copyright: GPL
> >>Vendor: The OpenGFS Project
> >>Group: System Environment/Kernel
> >>Source: %{name}-%{version}.tar.gz
> >>BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}
> >>URL: http://opengfs.sourceforge.net/
> >>BuildRequires: gcc, sed, perl
> >>Packager: Ted Chang <[email protected]>
> >>
> >>%description
> >>  OpenGFS is a clustered filesystem. It's usefulness is thought to be=
 mostly in SAN's, but goes well beyond that. It can be used with iSCSI, H=
yperSCSI, and even firewire.
> >>
> >>%package utils
> >>Group: System Environment/Base
> >>Summary: OpenGFS utilities
> >>Requires: perl,perl-Net-Telnet
> >>
> >>%description utils
> >>  OpenGFS utilities.
> >>
> >>
> >>%package modules-%{KERNEL_VERSION}
> >>Group: System Environment/Kernel
> >>Summary: OpenGFS kernel modules for %{KERNEL_VERSION}
> >>
> >>%description  modules-%{KERNEL_VERSION}
> >>  OpenGFS kernel modules for %{KERNEL_VERSION}
> >>
> >>
> >>%prep
> >>%setup -q=20
> >>./bootstrap
> >>
> >>%build
> >>./configure --with-linux_srcdir=3D/lib/modules/%{KERNEL_VERSION}/buil=
d --mandir=3D%{_mandir} --enable-opendlm --with-opendlm_includes=3D/usr/s=
rcc/redhat/BUILD/opendlm-0.9.2/src/include
> >>make
> >>
> >>
> >>%install
> >>#make DESTDIR=3D$RPM_BUILD_ROOT install
> >>make DESTDIR=3D$RPM_BUILD_ROOT docdir=3D%{_docdir}/%{name}-%{version}=
-%{release} install
> >>
> >>
> >>%post modules-%{KERNEL_VERSION}
> >>/sbin/depmod -a
> >>
> >>%postun modules-%{KERNEL_VERSION}
> >>/sbin/depmod -a
> >>
> >>
> >>%clean
> >>rm -rf $RPM_BUILD_ROOT
> >>#rm -rf $RPM_BUILD_DIR/%{name}-%{version}
> >>
> >>
> >>%files utils
> >>%defattr(-,root,root,755)
> >>/sbin/*
> >>%doc %{_mandir}/*
> >>%doc %{_docdir}/*
> >>
> >>
> >>%files modules-%{KERNEL_VERSION}
> >>%defattr(-,root,root)
> >>/lib/modules/%{KERNEL_VERSION}/*
> >>
> >>
> >>%changelog
> >>* Wed Mar 24 2004 Ted Chang <[email protected]>
> >>- initial RPM release
> >>   =20
> >>
--=20
Opinions expressed are those of the author and do not represent Intel
Corporation
"gpg --recv-keys --keyserver wwwkeys.pgp.net E1390A7F"
{E1390A7F:3AD1 1B0C 2019 E183 0CFF  55E8 369A 8B75 E139 0A7F}



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click