SMF Manifest for HTTP Server started by a forking job runner

Hi there,

I'm trying to write an SMF manifest to track the lifecycle of a Ruby
HTTP server (thin).  For various reasons, this has a few
complications:

1) It needs to be started by a meta starter called 'bundle exec'
2) It needs to be started with a set of paths and environments which
set up a particular version of Ruby

I'm pretty confident I can handle (2), but I'm really struggling with (1).

Bundle exec seems to start the server and then exits quickly, which
causes SMF to try to restart it, which causes the service to go into
maintenance.

Log snippet:

[ Jan 31 12:34:30 Method "start" exited with status 0 ]
[ Jan 31 12:34:30 Stopping because all processes in service exited. ]
[ Jan 31 12:34:30 Executing stop method (:kill) ][ Jan 31 12:34:30
Executing start method ("/lib/svc/method/thin start") ]cd
/data/apps/old-statements/current && rvm use ree@statements && bundle
exec thin -e production -p 4000 -c /data/apps/old-statements/current
-l /data/apps/old-statements/current/log/thin-4000.log -P
/data/apps/old-statements/current/tmp/pids/thin-4000.pid -u webservd
-g webservd start
/data/apps/old-statements/releases/6f8b585ceeec44c0a3ca80bbc6acc5743c561446
PATH=/usr/sbin:/usr/binSMF_FMRI=svc:/application/ruby/thin:defaultSMF_METHOD=/lib/svc/method/thin
start
SMF_RESTARTER=svc:/system/svc/restarter:default
TZ=PST8PDT

I've experimented with different service models (per
http://www.c0t0d0s0.org/archives/4145-Solaris-Features-Service-Management-Facility-Part-2-The-foundations-of-SMF.html)
and tried specifying a contract-based model, and asking SMF to not
worry about core or signal:

 <property_group name='startd' type='framework'>
<propval name='duration' type='astring' value='contract' />
<propval name='ignore-error' type='astring' value='core,signal'/>
</property_group>

But whatever combination I try seems to have the same result.  My
manifest is here:

<?xml version='1.0'?>
<!DOCTYPE service_bundle SYSTEM '/usr/share/lib/xml/dtd/service_bundle.dtd.1'>
<!--
//
// This file and its contents are supplied under the terms of the
// Common Development and Distribution License ("CDDL)". You may
// only use this file in accordance with the terms of the CDDL.
//
// A full copy of the text of the CDDL should have accompanied this
// source. A copy of the CDDL is also available via the Internet at
// http://www.illumos.org/license/CDDL.
//
//
// Copyright 2012 Atalanta Systems Ltd. All rights reserved.
//
-->

<service_bundle type='manifest' name='thin'>
  <service name='application/ruby/thin'
           type='service'
           version='1'>

    <!--
        Wait for network interfaces to be initialized.
    -->

    <dependency name='network'
                grouping='require_all'
                restart_on='error'
                type='service'>
      <service_fmri value='svc:/milestone/network:default'/>
      <!--<service_fmri value='svc:/network/nfs/client'/>-->
    </dependency>

    <!--
        Wait for all local filesystems to be mounted.
    -->

    <dependency name='filesystem-local'
                grouping='require_all'
                restart_on='none'
                type='service'>
      <service_fmri
          value='svc:/system/filesystem/local:default'/>
    </dependency>

    <exec_method
        type='method'
        name='start'
        exec='/lib/svc/method/thin start'
        timeout_seconds='60' />

    <exec_method
        type='method'
        name='stop'
        exec=':kill'
        timeout_seconds='30' />

    <instance name='default' enabled='false'>

      <property_group name='startd' type='framework'>
        <propval name='duration' type='astring' value='contract' />
        <propval name='ignore-error' type='astring' value='core,signal'/>
      </property_group>

      <property_group name='thin' type='application'>
        <propval name='app_home' type='astring' value='' />
        <propval name='user' type='astring' value='' />
        <propval name='group' type='astring' value='' />
        <propval name='environment' type='astring' value='production' />
        <propval name='gemset' type='astring' value='' />
        <propval name='tcp_listen_port' type='astring' value='4000' />
      </property_group>
    </instance>

    <stability value='Evolving' />

    <template>
      <common_name>
        <loctext xml:lang='C'>Yet another web server</loctext>
      </common_name>
      <documentation>
        <doc_link name='thin_docs'
uri='http://code.macournoyer.com/thin/usage/' />
      </documentation>
    </template>

  </service>

</service_bundle>

My helper script is here:

#!/sbin/sh
#
# thin helper script
#

. /lib/svc/share/smf_include.sh

# SMF_FMRI is the name of the target service. This allows multiple
instances
# to use the same script.

# Functions

getproparg() {
    val=`svcprop -c -p $1 $SMF_FMRI`
    [ -n "$val" ] && echo $val
}

# Variables

THIN_APP_HOME=`getproparg thin/app_home`
THIN_USER=`getproparg thin/user`
THIN_GROUP=`getproparg thin/group`
THIN_ENVIRONMENT=`getproparg thin/environment`
THIN_GEMSET=`getproparg thin/gemset`
THIN_LISTEN_PORT=`getproparg thin/tcp_listen_port`

THIN_PID=${THIN_APP_HOME}/tmp/pids/thin-${THIN_LISTEN_PORT}.pid
THIN_LOG=${THIN_APP_HOME}/log/thin-${THIN_LISTEN_PORT}.log

# THIN_PATH=`cd $THIN_APP_HOME; rvm use ree@${THIN_GEMSET}; echo
$GEM_HOME`

THIN_STARTER="cd $THIN_APP_HOME && rvm use ree@${THIN_GEMSET} && bundle exec"

#THIN_STARTER="cd $THIN_APP_HOME && bundle exec
/usr/local/rvm/gems/ree-1.8.7-2011.12@statements/bin/thin"

# Prerequisites test

## Mainfest properties test

if [ -z "$SMF_FMRI" ]; then
    echo "SMF framework variables are not initialized."
    exit "$SMF_EXIT_ERR"
fi

if [ ! -n "$THIN_APP_HOME" -o "${THIN_APP_HOME}" = "\"\"" ]; then
    echo "thin/app_home property not set"
    exit "$SMF_EXIT_ERR_CONFIG"
fi

if [ ! -n "$THIN_USER" -o "${THIN_USER}" = "\"\"" ]; then
    echo "thin/user property not set"
    exit "$SMF_EXIT_ERR_CONFIG"
fi

if [ ! -n "$THIN_GROUP" -o "${THIN_GROUP}" = "\"\"" ]; then
    THIN_GROUP=$THIN_USER
fi

if [ ! -n "$THIN_GEMSET" -o "${THIN_GEMSET}" = "\"\"" ]; then
    echo "thin/gemset property not set"
    exit "$SMF_EXIT_ERR_CONFIG"
fi

THIN_LISTEN_PORT=${THIN_LISTEN_PORT:-"3000"}

# File test

touch ${THIN_LOG}
touch ${THIN_PID}

if [ ! -f ${THIN_LOG} ] ; then
    echo "Cannot create $THIN_LOG log file"
    exit "$SMF_EXIT_ERR_CONFIG"
fi

if [ ! -f ${THIN_PID} ]; then
    echo "Cannot create $THIN_PID pid file"
    exit "$SMF_EXIT_ERR_CONFIG"
fi

# Clean thin pid file before starting new thin instance
rm -f ${THIN_PID}

# Main

case "$1" in
    'start')
        $THIN_STARTER thin -e $THIN_ENVIRONMENT -p $THIN_LISTEN_PORT
-c $THIN_APP_HOME -l $THIN_LOG -P $THIN_PID -u $THIN_USER -g
$THIN_GROUP start
        echo $THIN_STARTER thin -e $THIN_ENVIRONMENT -p
$THIN_LISTEN_PORT -c $THIN_APP_HOME -l $THIN_LOG -P $THIN_PID -u
$THIN_USER -g $THIN_GROUP start
        ;;
    'stop')
        ;;
    'restart')
        ;;
    *)
        echo $"Usage: $0 {start|stop}"
        exit 1
        ;;
esac

exit "$SMF_EXIT_OK"

I'm afraid I'm all out of ideas, and I'm running out of time before
this service needs to be running in a monitored/audited way...

Any ideas or recommendations gratefully received....

S.
-- 
Stephen Nelson-Smith
Technical Director
Atalanta Systems Ltd
www.atalanta-systems.com