RE: errors

I'm working with Alan Cox on a solution.  I have to put /var/log on
another disk on another scsi card so when it craps out, the driver will
write to /var/log/messages.  Then send him the log.

			Adam

On Tue, 8 Apr 2003 sean.upton-lttx/[email protected] wrote:

> I'm getting these about 6 days apart.  This is on a box that has a lot of
> ram, so swap is rarely used on this volume, and the ext3 filesystems are
> written to sparingly because the only thing that gets changed is stuff on
> /var/log (which is on this volume) - writes to this are very minimal
> (syslog, etc), as I have most of my application data and logging on another
> volume.  I think the thing that likely triggered that last problem I had
> like this was a cron'd logrotate.  This box is in very limited capacity, but
> will be heavily loaded in production soon, which scares me at the moment.
>
> Sean
>
> -----Original Message-----
> From: Adam Williams [mailto:[email protected]]
> Sent: Monday, April 07, 2003 8:24 AM
> To: sean.upton-lttx/[email protected]
> Cc: [email protected]; [email protected]
> Subject: RE: errors
>
>
> Yes that looks like exactly the same problem I am having on my poweredge
> 2500 with a perc 3/di.  Have you found a solution to it?  I am going to
> email the ext3 mailling list about it.  How often does yours give that
> error?  Mine is anywhere from 1 - 4 days.
>
> 			Adam
>
> On Mon, 7 Apr 2003 sean.upton-lttx/[email protected] wrote:
>
> > I'm pretty sure I am having the same problem with a mirror on a Adaptec
> > 2200S.  Running several ext3 volumes across a single RAID 1 of 2 Seagate
> > ST373405LC drives. I'm using the Debian 2.4.20 kernel, which is just a
> > compiled/packaged kernel.org kernel.  This mirror volume holds several
> > filesystems (and swap) that are infrequently written to.  The following
> > types of stuff show up on the console, while the box becomes unable to use
> /
> > /usr and /var (which are on this mirror):
> >
> >  I/O error: dev 08:05, sector 4456504
> > EXT3-fs error (device sd(8,5)): ext3_get_inode_loc: unable to read inode
> > block -
> >  inode=278533, block=557063
> >  I/O error: dev 08:05, sector 0
> > EXT3-fs error (device sd(8,5)) in ext3_reserve_inode_write: IO failure
> >  I/O error: dev 08:05, sector 0
> >  I/O error: dev 08:05, sector 4456504
> > EXT3-fs error (device sd(8,5)): ext3_get_inode_loc: unable to read inode
> > block -
> >  inode=278533, block=557063
> >  I/O error: dev 08:05, sector 0
> > EXT3-fs error (device sd(8,5)) in ext3_reserve_inode_write: IO failure
> >  I/O error: dev 08:05, sector 0
> >  I/O error: dev 08:05, sector 4456504
> > EXT3-fs error (device sd(8,5)): ext3_get_inode_loc: unable to read inode
> > block -
> >  inode=278533, block=557063
> >  I/O error: dev 08:05, sector 0
> > EXT3-fs error (device sd(8,5)) in ext3_reserve_inode_write: IO failure
> >
> >
> > [snip]
> >
> >
> >  I/O error: dev 08:02, sector 1076760
> >  I/O error: dev 08:02, sector 546920
> >  I/O error: dev 08:02, sector 546920
> >  I/O error: dev 08:02, sector 546920
> >  I/O error: dev 08:02, sector 1835040
> > EXT3-fs error (device sd(8,2)): ext3_get_inode_loc: unable to read inode
> > block -
> >  inode=114037, block=229380
> >  I/O error: dev 08:02, sector 0
> > EXT3-fs error (device sd(8,2)) in ext3_reserve_inode_write: IO failure
> >  I/O error: dev 08:02, sector 0
> >  I/O error: dev 08:02, sector 546920
> >  I/O error: dev 08:02, sector 546920
> >  I/O error: dev 08:01, sector 1585432
> >  I/O error: dev 08:01, sector 1585432
> >
> >
> >
> > Sean
> >
> > -----Original Message-----
> > From: Salyzyn, Mark
> > To: 'Adam Williams'; [email protected]
> > Sent: 4/7/03 7:12 AM
> > Subject: RE: errors
> >
> > Being that this is a RAID-0, sounds like you are getting a developing
> > drive
> > failure. ext3 is a logging filesystem, and slowly crawls to the
> > offending
> > blocks, rebooting justifies and resets the logging no doubt creating the
> > feeling of stability for a few days.
> >
> > Can you inspect the adapter event logging (excuse my ignorance regarding
> > the
> > tools that report said log please ...)?
> >
> > The killing of the interrupt handler is most likely a separate and
> > serious
> > issue. The modifications for the 2.5.x scsi system changes are an
> > ongoing
> > work in progress.
> >
> > Sincerely -- Mark Salyzyn
> >
> > -----Original Message-----
> > From: Adam Williams [mailto:[email protected]]
> > Sent: Monday, April 07, 2003 9:01 AM
> > To: [email protected]
> > Subject: errors
> >
> >
> > Hello,
> >
> > I have a Dell Poweredge 2600 with a Perc3/Di containing a 100GB raid 0.
> > Frequently, anywhere from 1 to 4 days, the server can't write to disk
> > anymore and prints on TTY1:
> >
> > EXT3-fs error (device sd(8,4)) in ext3_reserve_inode_write: IO Failure
> > I/O error: dev 08:04, sector 0
> > EXT3-fs error (device sd(8,4)) in ext3_new_inode:  IO failure
> > I/O error: dev 08:04 secotor 0
> > I/O error: dev 08:04 secotor 1050528
> > EXT3-fs error (device sd(8,4)): ext3_get_inode_loc: unable to read inode
> > block - inode=73040, block=131316
> > I/O dev 08:04 sector 0
> > EXT3-fs error (device sd(8,4)) in ext3_reserve_inode_write: IO Failure
> >
> > and on and on, with a lot of sector 0's and other sectors, which numbers
> > seem to be random, as well as the inodes and blocks.  But, all of the
> > I/O errors are always from dev 08:04.  The server is
> > running the stock 2.4.20 kernel (hand compiled) and redhat 8.0 i386.  I
> > can reboot the server and i'll run again fine anywhere from 1-4 days
> > until
> > this error happens again.  I upgraded to 2.5.66 as a test, and it gave
> > me
> > a different error message after a couple of days.  I don't have that
> > error
> > message written down, but the last line I do remember, which was
> > "AAAAAAIIIIIIEEEEEEEE! killing interrupt handler".  With 2.4.20 I can
> > still ssh into the server, but when I do any command, like df, iostat,
> > etc...it says "-bash: /bin/df Input/output error".  With 2.4.20 I can
> > hit
> > the rest button, but with 2.5.66 the reset button doesn't work once the
> > box prints the error message on tty1, and I have to inplug the power
> > cables from back.  I can't ssh or ping it either.
> >
> > Does anyone have suggestions I can try?  Downgrading the kernel to
> > older 2.4 versions still generates the same error.
> >
> > 			Thanks!
> > 			Adam
> >
> > _______________________________________________
> > Linux-aacraid-devel mailing list
> > [email protected]
> > http://lists.us.dell.com/mailman/listinfo/linux-aacraid-devel
> > Please read the FAQ at http://lists.us.dell.com/faq or search the list
> > archives at http://lists.us.dell.com/htdig/
> >
> > _______________________________________________
> > Linux-aacraid-devel mailing list
> > [email protected]
> > http://lists.us.dell.com/mailman/listinfo/linux-aacraid-devel
> > Please read the FAQ at http://lists.us.dell.com/faq or search the list
> > archives at http://lists.us.dell.com/htdig/
> >
>
> _______________________________________________
> Linux-aacraid-devel mailing list
> [email protected]
> http://lists.us.dell.com/mailman/listinfo/linux-aacraid-devel
> Please read the FAQ at http://lists.us.dell.com/faq or search the list archives at http://lists.us.dell.com/htdig/
>

_______________________________________________
Linux-aacraid-devel mailing list
[email protected]
http://lists.us.dell.com/mailman/listinfo/linux-aacraid-devel
Please read the FAQ at http://lists.us.dell.com/faq or search the list archives at http://lists.us.dell.com/htdig/