Re: Amanda 3.5.1 - "ERROR: amcheck-device terminated with signal 11"
Chris Hoogendyk <[email protected]>
| Newsgroups | gmane.comp.archivers.amanda.user |
|---|---|
| Message-ID | <[email protected]> |
From the other server:
amandabackup@eclogite:~/daily$ cat
/usr/local/var/amanda/chg-robot-dev-tape-by-id-scsi-1BDT-FlexStor-II-00MX64100589-LL0
$STATE = {
'drive_lru' => [
'0',
'1'
],
'last_status' => 1602694135,
'last_operation_delay' => 0,
'last_operation_time' => 1602694089,
'slots' => {
'3' => {
'loaded_in' => undef,
'label' => 'geo-daily-062',
'state' => 1,
'ie' => 0,
'f_type' => bless( {
'sign' => '+',
'value' => [
1
]
}, 'Math::BigInt' ),
'device_error' => undef,
'barcode' => '000062L7',
'device_status' => bless( {
'value' => [
0
],
'sign' => '+'
}, 'Math::BigInt' )
},
'4' => {
'device_status' => bless( {
'sign' => '+',
'value' => [
0
]
}, 'Math::BigInt' ),
'barcode' => '000063L7',
'device_error' => undef,
'f_type' => bless( {
'value' => [
1
],
'sign' => '+'
}, 'Math::BigInt' ),
'ie' => 0,
'state' => 1,
'loaded_in' => undef,
'label' => 'geo-daily-063'
},
'17' => {
'device_status' => bless( {
'value' => [
4
],
'sign' => '+'
}, 'Math::BigInt' ),
'barcode' => '000076L7',
'device_error' => 'Tape device /dev/nst1 is not ready or is
empty',
'ie' => 0,
'f_type' => undef,
'state' => 1,
'loaded_in' => undef,
'label' => undef
},
'16' => {
'barcode' => '000075L7',
'device_status' => bless( {
'sign' => '+',
'value' => [
4
]
}, 'Math::BigInt' ),
'device_error' => 'Tape device /dev/nst1 is not ready or is
empty',
'ie' => 0,
'f_type' => undef,
'state' => 1,
'loaded_in' => undef,
'label' => undef
},
'10' => {
'ie' => 0,
'f_type' => undef,
'state' => 1,
'loaded_in' => undef,
'label' => undef,
'barcode' => '000069L7',
'device_status' => bless( {
'sign' => '+',
'value' => [
4
]
}, 'Math::BigInt' ),
'device_error' => 'Tape device /dev/nst1 is not ready or is
empty'
},
'13' => {
'device_error' => 'Tape device /dev/nst1 is not ready or is
empty',
'device_status' => bless( {
'value' => [
4
],
'sign' => '+'
}, 'Math::BigInt' ),
'barcode' => '000072L7',
'loaded_in' => undef,
'label' => undef,
'f_type' => undef,
'ie' => 0,
'state' => 1
},
'7' => {
'state' => 1,
'ie' => 0,
'f_type' => undef,
'loaded_in' => undef,
'label' => undef,
'barcode' => '000066L7',
'device_status' => bless( {
'value' => [
4
],
'sign' => '+'
}, 'Math::BigInt' ),
'device_error' => 'Tape device /dev/nst1 is not ready or is empty'
},
'8' => {
'device_error' => 'Tape device /dev/nst1 is not ready or is
empty',
'device_status' => bless( {
'sign' => '+',
'value' => [
4
]
}, 'Math::BigInt' ),
'barcode' => '000067L7',
'label' => undef,
'loaded_in' => undef,
'ie' => 0,
'f_type' => undef,
'state' => 1
},
'19' => {
'label' => undef,
'loaded_in' => undef,
'f_type' => undef,
'ie' => 0,
'state' => 1,
'device_error' => 'Tape device /dev/nst1 is not ready or is
empty',
'barcode' => '000078L7',
'device_status' => bless( {
'value' => [
4
],
'sign' => '+'
}, 'Math::BigInt' )
},
'23' => {
'ie' => 0,
'state' => 1,
'f_type' => undef,
'loaded_in' => undef,
'label' => undef,
'barcode' => '000050L7',
'device_status' => bless( {
'value' => [
4
],
'sign' => '+'
}, 'Math::BigInt' ),
'device_error' => 'Tape device /dev/nst1 is not ready or is
empty'
},
'6' => {
'device_error' => undef,
'label' => 'geo-daily-065',
'loaded_in' => '1',
'device_status' => bless( {
'value' => [
0
],
'sign' => '+'
}, 'Math::BigInt' ),
'barcode' => '000065L7',
'state' => 1,
'f_type' => bless( {
'sign' => '+',
'value' => [
1
]
}, 'Math::BigInt' )
},
'2' => {
'f_type' => bless( {
'value' => [
1
],
'sign' => '+'
}, 'Math::BigInt' ),
'ie' => 0,
'state' => 1,
'loaded_in' => undef,
'label' => 'geo-daily-061',
'barcode' => '000061L7',
'device_status' => bless( {
'sign' => '+',
'value' => [
0
]
}, 'Math::BigInt' ),
'device_error' => undef
},
'12' => {
'barcode' => '000071L7',
'device_status' => bless( {
'value' => [
4
],
'sign' => '+'
}, 'Math::BigInt' ),
'device_error' => 'Tape device /dev/nst1 is not ready or is
empty',
'f_type' => undef,
'ie' => 0,
'state' => 1,
'loaded_in' => undef,
'label' => undef
},
'14' => {
'label' => undef,
'loaded_in' => undef,
'ie' => 0,
'state' => 1,
'f_type' => undef,
'device_error' => 'Tape device /dev/nst1 is not ready or is
empty',
'barcode' => '000073L7',
'device_status' => bless( {
'sign' => '+',
'value' => [
4
]
}, 'Math::BigInt' )
},
'22' => {
'device_status' => bless( {
'sign' => '+',
'value' => [
4
]
}, 'Math::BigInt' ),
'barcode' => '000049L7',
'device_error' => 'Tape device /dev/nst1 is not ready or is
empty',
'ie' => 0,
'state' => 1,
'f_type' => undef,
'loaded_in' => undef,
'label' => undef
},
'20' => {
'device_status' => bless( {
'value' => [
4
],
'sign' => '+'
}, 'Math::BigInt' ),
'barcode' => '000079L7',
'device_error' => 'Tape device /dev/nst1 is not ready or is
empty',
'ie' => 0,
'state' => 1,
'f_type' => undef,
'label' => undef,
'loaded_in' => undef
},
'21' => {
'ie' => 0,
'state' => 1,
'f_type' => undef,
'loaded_in' => undef,
'label' => undef,
'device_status' => bless( {
'value' => [
4
],
'sign' => '+'
}, 'Math::BigInt' ),
'barcode' => '000048L7',
'device_error' => 'Tape device /dev/nst1 is not ready or is
empty'
},
'15' => {
'device_error' => 'Tape device /dev/nst1 is not ready or is
empty',
'device_status' => bless( {
'sign' => '+',
'value' => [
4
]
}, 'Math::BigInt' ),
'barcode' => '000074L7',
'loaded_in' => undef,
'label' => undef,
'ie' => 0,
'f_type' => undef,
'state' => 1
},
'1' => {
'device_status' => undef,
'barcode' => undef,
'device_error' => undef,
'ie' => 0,
'state' => 2,
'f_type' => undef,
'label' => undef,
'loaded_in' => undef
},
'11' => {
'barcode' => '000070L7',
'device_status' => bless( {
'value' => [
4
],
'sign' => '+'
}, 'Math::BigInt' ),
'device_error' => 'Tape device /dev/nst1 is not ready or is
empty',
'f_type' => undef,
'ie' => 0,
'state' => 1,
'label' => undef,
'loaded_in' => undef
},
'18' => {
'label' => undef,
'loaded_in' => undef,
'ie' => 0,
'state' => 1,
'f_type' => undef,
'device_error' => 'Tape device /dev/nst1 is not ready or is
empty',
'barcode' => '000077L7',
'device_status' => bless( {
'value' => [
4
],
'sign' => '+'
}, 'Math::BigInt' )
},
'5' => {
'device_error' => undef,
'barcode' => '000064L7',
'device_status' => bless( {
'value' => [
0
],
'sign' => '+'
}, 'Math::BigInt' ),
'label' => 'geo-daily-064',
'loaded_in' => undef,
'ie' => 0,
'state' => 1,
'f_type' => bless( {
'value' => [
1
],
'sign' => '+'
}, 'Math::BigInt' )
},
'24' => {
'device_error' => undef,
'device_status' => undef,
'barcode' => 'CLN031CU',
'loaded_in' => undef,
'label' => undef,
'ie' => 0,
'f_type' => undef,
'state' => 1
},
'9' => {
'device_error' => 'Tape device /dev/nst1 is not ready or is
empty',
'barcode' => '000068L7',
'device_status' => bless( {
'sign' => '+',
'value' => [
4
]
}, 'Math::BigInt' ),
'label' => undef,
'loaded_in' => undef,
'ie' => 0,
'state' => 1,
'f_type' => undef
}
},
'drives' => {
'1' => {
'label' => 'geo-daily-065',
'res_info' => undef,
'state' => 1,
'barcode' => '000065L7',
'orig_slot' => 6
},
'0' => {
'barcode' => undef,
'state' => 2,
'orig_slot' => undef,
'label' => undef
}
},
'bc2lb' => {
'000100L7' => 'geo-daily-100',
'000091L7' => 'geo-daily-091',
'000063L7' => 'geo-daily-063',
'000090L7' => 'geo-daily-090',
'000086L7' => 'geo-daily-086',
'000061L7' => 'geo-daily-061',
'000083L6' => 'geo-daily-223',
'000064L7' => 'geo-daily-064',
'000086L6' => 'geo-daily-226',
'000065L7' => 'geo-daily-065',
'000087L6' => 'geo-daily-227',
'000082L7' => 'geo-daily-082',
'000061L6' => 'geo-daily-201',
'000066L6' => 'geo-daily-206',
'000062L6' => 'geo-daily-202',
'000064L6' => 'geo-daily-204',
'000091L6' => 'geo-daily-231',
'000095L7' => 'geo-daily-095',
'000088L7' => 'geo-daily-088',
'000080L7' => 'geo-daily-080',
'000092L7' => 'geo-daily-092',
'000084L7' => 'geo-daily-084',
'000089L7' => 'geo-daily-089',
'000082L6' => 'geo-daily-222',
'000098L7' => 'geo-daily-098',
'000099L7' => 'geo-daily-099',
'000080L6' => 'geo-daily-220',
'000092L6' => 'geo-daily-232',
'000084L6' => 'geo-daily-224',
'000077L6' => 'geo-daily-217',
'000097L7' => 'geo-daily-097',
'000081L7' => 'geo-daily-081',
'000062L7' => 'geo-daily-062',
'000063L6' => 'geo-daily-203',
'000085L6' => 'geo-daily-225',
'000083L7' => 'geo-daily-083',
'000088L6' => 'geo-daily-228',
'000089L6' => 'geo-daily-229',
'000078L6' => 'geo-daily-218',
'000065L6' => 'geo-daily-205',
'000085L7' => 'geo-daily-085',
'000093L7' => 'geo-daily-093',
'000081L6' => 'geo-daily-221',
'000079L6' => 'geo-daily-219',
'000094L7' => 'geo-daily-094',
'000096L7' => 'geo-daily-096',
'000090L6' => 'geo-daily-230',
'000087L7' => 'geo-daily-087'
},
'current_slot' => {
'config' => {
'daily' => {
'storage' => {
'daily' => {
'changer' => {
'NEO200' => 6
}
}
}
}
}
}
};
On 10/14/20 1:32 PM, Nathan Stratton Treadway wrote:
> On Wed, Oct 14, 2020 at 12:20:33 -0400, Chris Hoogendyk wrote:
>> Both lexical.pm and ScanInventory.pm restored to original. New fix
>> only patch applied to ScanInventory.pm. amanda.conf restored to use
>> oldest.
>>
>> amanda@marlin:~/daily$ amcheck daily
>>
>> Amanda Tape Server Host Check
>> -----------------------------
>> NOTE: Holding disk '/amanda3': 139784192 KB disk space available, using 34926592 KB
>> NOTE: Holding disk '/amanda4': 170082304 KB disk space available, using 65224704 KB
>> NOTE: Holding disk '/amanda5': 240713728 KB disk space available, using 135856128 KB
>> ***** Authorized Use Only *****
>>
>> snapper
>> slot 25: volume 'Bio-Research-028'
>> Will write to volume 'Bio-Research-028' in slot 25.
>> NOTE: skipping tape-writable test
>> Server check took 50.059 seconds
>> Amanda Backup Client Hosts Check
>> --------------------------------
>> Client check: 4 hosts checked in 6.692 seconds. 0 problems found.
>> (brought to you by Amanda 3.5.1)
>>
>> amanda@marlin:~/daily$
> [...]>
>> Launched a flush on that. Then the following seems to set up a tape on the second tape drive.
>>
>> amanda@marlin:~/daily$ amtape daily taper
>>
>> slot 31: volume 'Bio-Research-032'
>> Will write to volume 'Bio-Research-032' in slot 31.
>>
>> amanda@marlin:~/daily$
> Okay, sounds like things are back to working "normally" on that server,
> right?
>
> So, do you still have a second server which is getting coredumps (at
> least with the oldest taperscan)?
>
> Based on the investigation so far, it seems like the crash is caused by
> tape-inventory records which have no label text along with some specific
> other data field values.
>
> If you post the /usr/local/var/amanda/chg-robot-dev-tape-by-id-scsi*
> changer state file that other server, we can double check that such
> entries exist over there, too.
>
> (Assume they do, then I guess the question will be whether you want to
> apply the same ScanInventory.pm patch there, or if you instead want to
> try clearing that/those bad inventory record(s) without changing the
> installed code on that box....)
>
> Nathan
>
> ----------------------------------------------------------------------------
> Nathan Stratton Treadway - [email protected] - Mid-Atlantic region
> Ray Ontko & Co. - Software consulting services - http://www.ontko.com/
> GPG Key: http://www.ontko.com/~nathanst/gpg_key.txt ID: 1023D/ECFB6239
> Key fingerprint = 6AD8 485E 20B9 5C71 231C 0C32 15F3 ADCD ECFB 6239
--
---------------
Chris Hoogendyk
-
O__ ---- Systems Administrator, Retired
c/ /'_ --- Biology & Geosciences Departments
(*) \(*) -- 315 Morrill Science Center III
~~~~~~~~~~ - University of Massachusetts, Amherst
<[email protected]>
---------------
Erdös 4