rexecv, ssh, nohup, reop_export_path

Jack de Valpine <[email protected]>
Newsgroups gmane.linux.cluster.ssic.user
Message-ID <[email protected]>
Hi,

I have been through the archives a bit and seen that this topic has come 
up in part but it is not clear if there is any resolution. Following is 
my description of what we are seeing.

I have added rexecv into some code to start jobs on selected nodes. 
Normally I login into the openSSI (1.2.2 as reported by cluster -r) 
cluster via ssh. The job is started as follows:

    /usr/bin/nohup my_distrubuted_job &

We use nohup on non ssi nodes so that the job will continue to process 
even if the user logs out of their ssh session or the ssh session goes 
down for some reason. So the theory is that it should work on the 
cluster. However, after exiting my ssh session and having to kill the 
xterm the session was running in as the exit just hangs on execution, 
upon logging back in I have discovered the following:

    * processes formerly running on other nodes are gone
    * only process running on node 1 continues
    * dmesg reports - reop_export_path: Can't export unlinked file
      /cluster/dev/pts1 (deleted)
    * lsof | grep pts - shows /dev/pts/1 (deleted) - repeated 4 times
    * ls /proc/<pid of main process>/fd - shows that:
          o 0 -> /dev/pts/1 (deleted)
          o 1 -> nohup.out
          o 2 -> nohup.out
          o 3 -> nohup.out
          o 4 -> pipe:[17098126]
          o 5 -> input_file
          o 6 -> pipe:[17098129]

    where fd's 0, 4 and 6 are shown in red, which I think means they are
    broken links.

In an earlier thread on "Migration pains," it seems the problem relates 
to sshd disconnecting the pty which then frees up the pty when in fact 
it should not be. It sounded like there was a potential patch/resolution 
to this. Did  this make it into 1.2.2? Does it need to be applied by hand?

Note that if I do note logout of the ssh session then the job runs fine 
to completion.

Thanks for your time and please let me know if there is any additional 
information that I can add to clarify the issue.

-Jack de Valpine

-- 
# Jack de Valpine
# president
#
# visarc incorporated
# http://www.visarc.com
#
# channeling technology for superior design and construction
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.