raid5 troubles on an e250

Hey folks,

	I'm trying to get an e250 with an external tower
stable, and am running into some problems. This is the first time
I've ever used linux on a sparc and suspect that there's some
major detail I am not aware of that I am not making allowances for.

Summary of problem:

I need to create a software raid, raid 5.
I partitoned my drives, created the devices and things worked
swimingly until I had to reboot. when the box came back up,
some of the drives contained their old partition table.
I've re-partitoned, sync'ed, rebooted this box dozens
of times now and I can't get the new paritioning to "stick"

If anyone can suggest what the problem might be or how I may
gather additional information I'd appreciate it. Please
let me know if there is any additional information I can provide.

Thanks!

Jen B

[jlb@shaft jlb]$ uptime
 11:30am  up 6 days, 23:51,  1 user,  load average: 0.00, 0.00, 0.00
[jlb@shaft jlb]$ cat /proc/mdstat
Personalities : [raid1] [raid5]
read_ahead 1024 sectors
md1 : active raid5 sdj2[5] sdj1[4] sdi3[3] sdh3[2] sdg2[1] sdg1[0]
      35438336 blocks level 5, 32k chunk, algorithm 2 [5/5] [UUUUU]

md0 : active raid5 sdf2[4] sdf1[3] sde3[2] sdd2[1] sdd1[0]
      26525952 blocks level 5, 32k chunk, algorithm 2 [4/4] [UUUU]

md2 : active raid5 sdl2[1] sdl1[4] sdk2[3] sdk1[2] sdm1[0]
      34705152 blocks level 5, 32k chunk, algorithm 2 [5/5] [UUUUU]

unused devices: <none>

***after several reboots & writing a few G of data to the devices:

Personalities : [raid1] [raid5]
read_ahead 1024 sectors
md2 : active raid5 sdl1[4] sdk2[3] sdk1[2] sdl2[1] sdm1[0]
      34705152 blocks level 5, 32k chunk, algorithm 2 [5/5] [UUUUU]

md0 : active raid5 sdf2[2] sdf1[3] sdd2[1] sdd1[0]
      26525952 blocks level 5, 32k chunk, algorithm 2 [4/4] [UUUU]

unused devices: <none>

***errors from running raidstart on /dev/md1:

Aug  9 10:23:38 shaft kernel:  [events: 00000006]
Aug  9 10:23:38 shaft kernel:  [events: 00000006]
Aug  9 10:23:38 shaft kernel: md: could not lock sdh3, zero-size? Marking
faulty.
Aug  9 10:23:38 shaft kernel: md: could not import sdh3, trying to run
array nevertheless.
Aug  9 10:23:38 shaft kernel: md: could not lock sdi3, zero-size? Marking
faulty.
Aug  9 10:23:38 shaft kernel: md: could not import sdi3, trying to run
array nevertheless.
Aug  9 10:23:39 shaft kernel: md: could not lock sdj1, zero-size? Marking
faulty.
Aug  9 10:23:39 shaft kernel: md: could not import sdj1, trying to run
array nevertheless.
Aug  9 10:23:39 shaft kernel: md: could not lock sdj2, zero-size? Marking
faulty.
Aug  9 10:23:39 shaft kernel: md: could not import sdj2, trying to run
array nevertheless.
Aug  9 10:23:39 shaft kernel: md: autorun ...
Aug  9 10:23:39 shaft kernel: md: considering sdg2 ...
Aug  9 10:23:39 shaft kernel: md:  adding sdg2 ...
Aug  9 10:23:39 shaft kernel: md:  adding sdg1 ...
Aug  9 10:23:39 shaft kernel: md: created md1
Aug  9 10:23:39 shaft kernel: md: bind<sdg1,1>
Aug  9 10:23:39 shaft kernel: md1: WARNING: sdg2 appears to be on the same
physical disk as sdg1. True
Aug  9 10:23:39 shaft kernel:      protection against single-disk failure
might be compromised.
Aug  9 10:23:39 shaft kernel: md: bind<sdg2,2>
Aug  9 10:23:39 shaft kernel: md: running: <sdg2><sdg1>
Aug  9 10:23:39 shaft kernel: md: sdg2's event counter: 00000006
Aug  9 10:23:39 shaft kernel: md: sdg1's event counter: 00000006
Aug  9 10:23:39 shaft kernel: md1: former device sdh3 is unavailable,
removing from array!
Aug  9 10:23:39 shaft kernel: md1: former device sdi3 is unavailable,
removing from array!
Aug  9 10:23:39 shaft kernel: md1: former device sdj1 is unavailable,
removing from array!
Aug  9 10:23:39 shaft kernel: md1: former device sdj2 is unavailable,
removing from array!
Aug  9 10:23:39 shaft kernel: md1: max total readahead window set to 960k
Aug  9 10:23:39 shaft kernel: md1: 4 data-disks, max readahead per
data-disk: 240k
Aug  9 10:23:39 shaft kernel: raid5: device sdg2 operational as raid disk
1
Aug  9 10:23:39 shaft kernel: raid5: device sdg1 operational as raid disk
0
Aug  9 10:23:39 shaft kernel: raid5: not enough operational devices for
md1 (3/5 failed)
Aug  9 10:23:39 shaft kernel: RAID5 conf printout:
Aug  9 10:23:39 shaft kernel:  --- rd:5 wd:2 fd:3
Aug  9 10:23:39 shaft kernel:  disk 0, s:0, o:1, n:0 rd:0 us:1 dev:sdg1
Aug  9 10:23:39 shaft kernel:  disk 1, s:0, o:1, n:1 rd:1 us:1 dev:sdg2
Aug  9 10:23:39 shaft kernel:  disk 2, s:0, o:0, n:2 rd:2 us:1 dev:[dev
00:00]
Aug  9 10:23:39 shaft kernel:  disk 3, s:0, o:0, n:3 rd:3 us:1 dev:[dev
00:00]
Aug  9 10:23:39 shaft kernel:  disk 4, s:0, o:0, n:4 rd:4 us:1 dev:[dev
00:00]
Aug  9 10:23:39 shaft kernel: raid5: failed to run raid set md1
Aug  9 10:23:39 shaft kernel: md: pers->run() failed ...
Aug  9 10:23:39 shaft kernel: md :do_md_run() returned -22
Aug  9 10:23:39 shaft kernel: md: md1 stopped.
Aug  9 10:23:39 shaft kernel: md: unbind<sdg2,1>
Aug  9 10:23:39 shaft kernel: md: export_rdev(sdg2)
Aug  9 10:23:39 shaft kernel: md: unbind<sdg1,0>
Aug  9 10:23:39 shaft kernel: md: export_rdev(sdg1)
Aug  9 10:23:39 shaft kernel: md: ... autorun DONE.


***hmm, what happened to  sdg,j,i?

[jlb@shaft jlb]$ sudo /sbin/fdisk /dev/sdj
Detected sun disklabel with wrong checksum.
Probably you'll have to set all the values,
e.g. heads, sectors, cylinders and partitions
or force a fresh label (s command in main menu)

***ran through the fresh disklabel process using custom/defaults

Command (m for help): p

Disk /dev/sdj (Sun disk label): 64 heads, 32 sectors, 17499 cylinders
Units = cylinders of 2048 * 512 bytes

   Device Flag    Start       End    Blocks   Id  System
/dev/sdj1             0     17449  17867776   83  Linux native
/dev/sdj2  u      17449     17499     51200   82  Linux swap
/dev/sdj3             0     17499  17918976    5  Whole disk

Command (m for help):

***the above is not how I partitioned the drive. It was partitioned into
one giant linux raid partition. same thing for the other devices. These
drives
have been partitioned at least a dozen times to fit this schema.

my raid devices were 'in tact' for several reboots, and didn't corrupt
until i moved data onto them.

The drives that revert to their old partitioning aren't consistent.


The actual hardware in this setup is a sun e250 with 2 scsi cards
in it. A Qlogic and an LSI using the sym53cxx driver.
drives a-e are in the e250 and hanging off the LSI. The Qlogic
has a seperate tower hanging off of it with the remaining disks.

Here's my /etc/raidtab:

raiddev /dev/md0
        raid-level      5
        nr-raid-disks  4
        nr-spare-disks  1
        persistent-superblock 1
        parity-algorithm        left-symmetric
        chunk-size 32
        device /dev/sdd1
        raid-disk 0
        device /dev/sdd2
        raid-disk 1
        device /dev/sde3
        raid-disk 2
        device /dev/sdf1
        raid-disk 3
        device /dev/sdf2
        spare-disk 0

raiddev /dev/md1
        raid-level      5
        nr-raid-disks   5
        nr-spare-disks  1
        persistent-superblock   1
        parity-algorithm        left-symmetric
        chunk-size      32
        device  /dev/sdg1
        raid-disk 0
        device  /dev/sdg2
        raid-disk 1
        device  /dev/sdh3
        raid-disk 2
        device  /dev/sdi3
        raid-disk 3
        device  /dev/sdj1
        raid-disk  4
        device  /dev/sdj2
        spare-disk 0


raiddev /dev/md2
        raid-level      5
        nr-raid-disks   5
        nr-spare-disks  1
        persistent-superblock   1
        parity-algorithm        left-symmetric
        chunk-size      32
        device  /dev/sdm1
        raid-disk 0
        device  /dev/sdm2
        raid-disk 1
        device  /dev/sdk1
        raid-disk 2
        device  /dev/sdk2
        raid-disk 3
        device  /dev/sdl1
        raid-disk 4
        device  /dev/sdl2
        spare-disk 0





. . . . . . . . . . . . . . . . . .
[email protected]
[email protected]
[email protected]