Ghost jobs?!

Marco Gaiarin <[email protected]>
Newsgroups gmane.comp.bacula.user
Organization Il gaio usa sempre TIN per le liste, fallo anche tu!!!
Message-ID <[email protected]> 
Deciding a title to this post is a bit tricky... ;-)


Firs of all: i need to chage a set of tapes in a tape autochanger; to be
able to release the tapes, the library and change the tape i have to
'umount' the tape in the tape unity(es); but if i use umount, then i need to
do an explicit 'mount'. I'm doing something wrong?


After that, a collegues forgot to do a 'mount'. This morning the situation
is:

 *status dir
 [...]
  JobId  Type Level     Files     Bytes  Name              Status
 ======================================================================
  33832  Back Full          0         0  LOM-BP-BPBKPLOMShares1 is running
  33833  Back Full          0         0  LOM-BP-BPBKPLOMShares2 is waiting on Storage "BPBKPLOMAutochanger"
  33834  Back Full          0         0  LOM-BP-BPBKPLOMHomes is waiting on Storage "BPBKPLOMAutochanger"
  33835  Back Full          0         0  LOM-BP-BPBKPLOMServers is waiting on Storage "BPBKPLOMAutochanger"
  33836  Back Full          0         0  LOM-BP-BPBKPLOMSEDI is waiting on Storage "BPBKPLOMAutochanger"


and mount simply does not work, does nothing. But if i see at storage:

*status storage=BPBKPLOMAutochanger
Connecting to Storage daemon BPBKPLOMAutochanger at bpbkplom.bp.lnf.it:9103

bpbkplom-sd Version: 9.4.2 (04 February 2019) x86_64-pc-linux-gnu debian 10.5
Daemon started 22-Nov-24 14:45. Jobs: run=168, running=10.
 Heap: heap=196,608 smbytes=37,405,481 max_bytes=39,571,259 bufs=565 max_bufs=1,034
 Sizes: boffset_t=8 size_t=8 int32_t=4 int64_t=8 mode=0,0 newbsr=0
 Res: ndevices=1 nautochgr=1

Running Jobs:
Writing: Full Backup job LOM-BP-BPBKPLOMServers JobId=33423 Volume=""
    pool="LOM-BP-BPBKPLOMLTOPool" device="LTO9Storage0" (/dev/tape/by-id/scsi-10WT020147-nst)
    spooling=0 despooling=1 despool_wait=0
    Files=8,285,953 Bytes=591,062,000,287 AveBytes/sec=743,618 LastBytes/sec=222,231
    FDReadSeqNo=79,962,545 in_msg=55164530 out_msg=6 fd=13
Writing: Incremental Backup job LOM-BP-BPBKPLOMShares1 JobId=33705 Volume="AAJ614L9"
    pool="LOM-BP-BPBKPLOMLTOPool" device="LTO9Storage0" (/dev/tape/by-id/scsi-10WT020147-nst)
    spooling=0 despooling=0 despool_wait=1
    Files=14,369 Bytes=4,008,128,015 AveBytes/sec=10,933 LastBytes/sec=3,300
    FDReadSeqNo=128,259 in_msg=105243 out_msg=10 fd=10
Writing: Incremental Backup job LOM-BP-BPBKPLOMSEDI JobId=33709 Volume="AAJ614L9"
    pool="LOM-BP-BPBKPLOMLTOPool" device="LTO9Storage0" (/dev/tape/by-id/scsi-10WT020147-nst)
    spooling=0 despooling=0 despool_wait=1
    Files=40,931 Bytes=11,841,582,904 AveBytes/sec=32,307 LastBytes/sec=9,752
    FDReadSeqNo=412,767 in_msg=333921 out_msg=10 fd=11
Writing: Incremental Backup job LOM-BP-BPBKPLOMShares2 JobId=33770 Volume="AAJ614L9"
    pool="LOM-BP-BPBKPLOMLTOPool" device="LTO9Storage0" (/dev/tape/by-id/scsi-10WT020147-nst)
    spooling=0 despooling=0 despool_wait=0
    Files=0 Bytes=0 AveBytes/sec=0 LastBytes/sec=0
    FDReadSeqNo=6 in_msg=6 out_msg=5 fd=12
Writing: Incremental Backup job LOM-BP-BPBKPLOMHomes JobId=33771 Volume="AAJ614L9"
    pool="LOM-BP-BPBKPLOMLTOPool" device="LTO9Storage0" (/dev/tape/by-id/scsi-10WT020147-nst)
    spooling=0 despooling=0 despool_wait=0
    Files=0 Bytes=0 AveBytes/sec=0 LastBytes/sec=0
    FDReadSeqNo=6 in_msg=6 out_msg=5 fd=15
Writing: Incremental Backup job LOM-BP-BPBKPLOMServers JobId=33772 Volume="AAJ614L9"
    pool="LOM-BP-BPBKPLOMLTOPool" device="LTO9Storage0" (/dev/tape/by-id/scsi-10WT020147-nst)
    spooling=0 despooling=0 despool_wait=0
    Files=0 Bytes=0 AveBytes/sec=0 LastBytes/sec=0
    FDReadSeqNo=6 in_msg=6 out_msg=5 fd=14
Writing: Full Backup job LOM-BP-BPBKPLOMShares1 JobId=33832 Volume="AAJ614L9"
    pool="LOM-BP-BPBKPLOMLTOPool" device="LTO9Storage0" (/dev/tape/by-id/scsi-10WT020147-nst)
    spooling=0 despooling=0 despool_wait=0
    Files=0 Bytes=0 AveBytes/sec=0 LastBytes/sec=0
    FDReadSeqNo=6 in_msg=6 out_msg=5 fd=19
Writing: Full Backup job LOM-BP-BPBKPLOMShares2 JobId=33833 Volume="AAJ614L9"
    pool="LOM-BP-BPBKPLOMLTOPool" device="LTO9Storage0" (/dev/tape/by-id/scsi-10WT020147-nst)
    spooling=0 despooling=0 despool_wait=0
    Files=0 Bytes=0 AveBytes/sec=0 LastBytes/sec=0
    FDSocket closed
Writing: Full Backup job LOM-BP-BPBKPLOMHomes JobId=33834 Volume="AAJ614L9"
    pool="LOM-BP-BPBKPLOMLTOPool" device="LTO9Storage0" (/dev/tape/by-id/scsi-10WT020147-nst)
    spooling=0 despooling=0 despool_wait=0
    Files=0 Bytes=0 AveBytes/sec=0 LastBytes/sec=0
    FDSocket closed
Writing: Full Backup job LOM-BP-BPBKPLOMSEDI JobId=33836 Volume="AAJ614L9"
    pool="LOM-BP-BPBKPLOMLTOPool" device="LTO9Storage0" (/dev/tape/by-id/scsi-10WT020147-nst)
    spooling=0 despooling=0 despool_wait=0
    Files=0 Bytes=0 AveBytes/sec=0 LastBytes/sec=0
    FDSocket closed
Writing: Full Backup job LOM-BP-BPBKPLOMServers JobId=33835 Volume="AAJ614L9"
    pool="LOM-BP-BPBKPLOMLTOPool" device="LTO9Storage0" (/dev/tape/by-id/scsi-10WT020147-nst)
    spooling=0 despooling=0 despool_wait=0
    Files=0 Bytes=0 AveBytes/sec=0 LastBytes/sec=0
    FDSocket closed
====

Jobs waiting to reserve a drive:
   3609 JobId=33833 Max concurrent jobs=7 exceeded on Tape device "LTO9Storage0" (/dev/tape/by-id/scsi-10WT020147-nst).
   3609 JobId=33834 Max concurrent jobs=7 exceeded on Tape device "LTO9Storage0" (/dev/tape/by-id/scsi-10WT020147-nst).
   3609 JobId=33836 Max concurrent jobs=7 exceeded on Tape device "LTO9Storage0" (/dev/tape/by-id/scsi-10WT020147-nst).
   3609 JobId=33835 Max concurrent jobs=7 exceeded on Tape device "LTO9Storage0" (/dev/tape/by-id/scsi-10WT020147-nst).
====

Terminated Jobs:
 JobId  Level    Files      Bytes   Status   Finished        Name 
===================================================================
 33421  Full    862,595    5.668 T  OK       13-Jan-25 12:38 LOM-BP-BPBKPLOMShares2
 33422  Full   2,444,617    2.387 T  OK       14-Jan-25 00:29 LOM-BP-BPBKPLOMHomes
 33580  Incr        894    1.307 G  OK       14-Jan-25 07:04 LOM-BP-BPBKPLOMHomes
 33579  Incr     41,153    400.2 G  OK       14-Jan-25 08:16 LOM-BP-BPBKPLOMShares2
 33420  Full   3,396,198    3.568 T  OK       15-Jan-25 05:42 LOM-BP-BPBKPLOMShares1
 33644  Incr        517    133.4 M  OK       15-Jan-25 05:45 LOM-BP-BPBKPLOMHomes
 33643  Incr     16,642    53.10 G  OK       15-Jan-25 05:57 LOM-BP-BPBKPLOMShares2
 33424  Full   5,456,571    5.466 T  OK       16-Jan-25 02:23 LOM-BP-BPBKPLOMSEDI
 33706  Incr        559    2.732 G  OK       16-Jan-25 05:25 LOM-BP-BPBKPLOMShares2
 33707  Incr        949    1.194 G  OK       16-Jan-25 06:01 LOM-BP-BPBKPLOMHomes
====

Device status:
Autochanger "Autochanger" with devices:
   "LTO9Storage0" (/dev/tape/by-id/scsi-10WT020147-nst)

Device Tape: "LTO9Storage0" (/dev/tape/by-id/scsi-10WT020147-nst) is not open.
   Device is BLOCKED waiting for media.
   Drive 0 is not loaded.
==
====

Used Volume status:
Reserved volume: AAJ614L9 on Tape device "LTO9Storage0" (/dev/tape/by-id/scsi-10WT020147-nst)
    Reader=0 writers=3 reserves=4 volinuse=0 worm=0
====

Data spooling: 3 active jobs, 91,840,969,609 bytes; 165 total jobs, 1,137,764,115,807 max bytes/job.
Attr spooling: 3 active jobs, 4,166,890,103 bytes; 165 total jobs, 9,143,599,805 max bytes.
====


Seems to me that i've reached 'Max concurrent jobs' so the storage are stuck
and don't receive more 'mount' command; but still if i try to cancel stuck
incremental jobs:

 *cancel jobid=33705
 Warning Job JobId=33705 is not running.

How can i recover from a situation like this, if jobs can occupy a 'slot'
but cannot be canceled?


Thanks.

--
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.