Re: [mh] Is this the expected behaviour of process_item?

Giles Godart-Brown <[email protected]> Fri, 19 Feb 2021 19:17:40 +0000
Newsgroups gmane.comp.misc.misterhouse.user
Message-ID <[email protected]>
Attached is a first go at building a process_item_queue.

Its a single queue, and I'm sure one queue per process_item would be 
better, but I've run out of talent :-)

Giles

On 18/02/2021 18:28, Giles Godart-Brown wrote:
> I was going to build it for myself, then if its good enough (?) and 
> fulfils backwards compatibility I will add a pull
>
> G
>
> On 18/02/2021 17:25, Jeff Siddall via misterhouse-users wrote:
>> Giles,
>>
>> When you say "I'll build a queue" do you mean for your project or 
>> adding it onto process_items?  As long as it was backwards compatible 
>> with the existing behavior of a process_item, it would be nice to add 
>> that onto the process_item so there is a standard way for anyone to 
>> push items onto a process_item queue instead of everyone implementing 
>> it their own way.
>>
>> Jeff
>>
>>
>> On 2021-02-18 11:47 a.m., Giles Godart-Brown wrote:
>>>
>>> Thanks H
>>>
>>> I'll build a queue and update the docs over the weekend
>>>
>>> Giles
>>>
>>> On 18/02/2021 14:24, H Plato wrote:
>>>> As far as I know that’s how it works, the process_item has a single 
>>>> thread. You can check if the process_item is still active by 
>>>> calling$p_longproc->done(), and then queuing 
>>>> the additional commands to an array. That’s how I’ve dealt with 
>>>> this in some of my modules
>>>>
>>>>> On Feb 18, 2021, at 1:22 AM, Giles Godart-Brown 
>>>>> <[email protected] <mailto:[email protected]>> wrote:
>>>>>
>>>>> I use process_items to send messages and urls, occasionally 
>>>>> (usually when I write bad code) I need to send a lot of emails in 
>>>>> quick succession but MisterHouse kills any unfinished ones, rather 
>>>>> than running them in parallel which is what I expected. To test 
>>>>> this I built a minimum reproducible case below where it launches a 
>>>>> process that simply sleeps for 5 seconds twice in succession. As 
>>>>> you can see from the print log, he first one gets killed.
>>>>>
>>>>> Is this normal?
>>>>>
>>>>> Giles
>>>>>
>>>>> <mytest.pl>
>>>>>
>>>>> print_log( "Longproc run 1", "INFORMATIONAL", "p_longproc" );
>>>>> start $p_longproc;
>>>>> print_log( "Longproc run 2", "INFORMATIONAL", "p_longproc" );
>>>>> start $p_longproc;
>>>>> </mytest.pl>
>>>>>
>>>>> <longproc.sh>
>>>>>
>>>>> #!/bin/sh
>>>>> # shell to sleep
>>>>> echo longproc start 5 second sleep pid= $$
>>>>> sleep 5
>>>>> echo "longproc end"
>>>>> exit 0
>>>>> </longproc.sh>
>>>>>
>>>>> <print.log>
>>>>>
>>>>> 18/02/2021 08:10:27 [p_longproc] INFO Longproc run 1
>>>>> 18/02/2021 08:10:27 [p_longproc] INFO Longproc run 2
>>>>> Warning, a previous 'start' on this process has not finished yet
>>>>> Killing unfinished process id 9380 
>>>>> cmd=//home/pi/mh/GGBcode/procs/longproc.sh
>>>>>
>>>>> ...
>>>>>
>>>>> 18/02/2021 08:10:33 [p_longproc] longproc start 5 second sleep 
>>>>> pid= 9381
>>>>> 18/02/2021 08:10:33 [p_longproc] longproc end
>>>>>
>>>>> </print.log>
>>>>>
>>>>> ________________________________________________________
>>>>> To unsubscribe from this list, go to: 
>>>>> https://lists.sourceforge.net/lists/listinfo/misterhouse-users
>>>>>
>>>>
>>>
>>> <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient> 
>>>     Virus-free. www.avg.com 
>>> <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient> 
>>>
>>>
>>> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
>>>
>>>
>>> ________________________________________________________
>>> To unsubscribe from this list, go to: 
>>> https://lists.sourceforge.net/lists/listinfo/misterhouse-users
>>>
>>
>>
>> ________________________________________________________
>> To unsubscribe from this list, go to: 
>> https://lists.sourceforge.net/lists/listinfo/misterhouse-users
>>


-- 
This email has been checked for viruses by AVG.
https://www.avg.com

________________________________________________________
To unsubscribe from this list, go to: https://lists.sourceforge.net/lists/listinfo/misterhouse-users
process_item_queue.pl (text/plain, 4.1 KB)
=head1 B<Process_Item_queue>

=head2 SYNOPSIS

  
=head2 DESCRIPTION

use this object to add process items to a queue. 
This version has a single queue called @process_item_queue, the next item on the queue is forked when the previous one finishes

=head2 INHERITS

<>

=head2 METHODS

add_to_queue add a process item to the queue
check_process_item_queue This is called regularly to see if it has any items to process on the queue

=over

=cut

use strict;
use Data::Dumper;

#-----------------------------------------------------------------------
# tests
#-----------------------------------------------------------------------
# noloop=start
$my_test1   = new Voice_Cmd 'Run test [1,2,3,4]';
$p_longproc = new Process_Item;
set $p_longproc $config_parms{"code_dir"} . "/procs/longproc.sh";
$p_longproc->set_output( $config_parms{data_dir} . "/logs/longproc.txt" );
$p_longproc->set_timeout(10);

# noloop=stop

if ( $state = said $my_test1) {
	if ( $state == 1 ) {
		add_process_item_to_queue("p_longproc");
		check_process_item_queue();
	}
	elsif ( $state == 2 ) {
		print_log( "Longproc run 1", "INFORMATIONAL", "p_longproc" );
		add_process_item_to_queue("p_longproc");
		print_log( "Longproc run 2", "INFORMATIONAL", "p_longproc" );
		add_process_item_to_queue("p_longproc");

	}

	elsif ( $state == 3 ) {

	}
	elsif ( $state == 4 ) {

	}
}

if ( done_now $p_longproc) {
	print_log( "Longproc done", "INFORMATIONAL", "p_longproc" );
	if ( timed_out $p_longproc) {
		print_log( 'longproc - timed out', "ERROR", "p_longproc" );
	}
	print_process_log( "longproc.txt", "p_longproc" );
}

#-----------------------------------------------------------------------
# every  seconds check the queue
#-----------------------------------------------------------------------

if ($New_Second) {
	check_process_item_queue();
}

#-----------------------------------------------------------------------
# globals
#-----------------------------------------------------------------------

#-----------------------------------------------------------------------
# Add a process item to the queue
#-----------------------------------------------------------------------

sub add_process_item_to_queue {
	my $this_process = $_[0];
	print_log( "Adding " . $this_process . " to the queue",
		"INFORMATIONAL", "add_process_item_to_queue" );
	if ( scalar @process_item_queue == 0 ) {
		start_process_name($this_process);
	}

	push @process_item_queue, $this_process;

}

#-----------------------------------------------------------------------
# check the process_item_queue
# if the previous one is done, delete it and action the next
#-----------------------------------------------------------------------
sub check_process_item_queue {
	my ( $this_process, $next_process );
	my $count_items_in_queue = scalar @process_item_queue;
	if ( $count_items_in_queue > 0 ) {
		print_log(
			"there are  " . $count_items_in_queue . " items in the queue",
			"INFORMATIONAL", "check_process_item_queue" );
		$this_process = @process_item_queue[0];
		my $this_process_object = get_object_by_name($this_process);
		if ( !defined( $this_process_object->{done} ) )
		{    # this process is still running
			return;
		}
		else {
			print_log( $this_process . " has finished",
				"INFORMATIONAL", "check_process_item_queue" );
			if ( $count_items_in_queue == 1 ) {
				@process_item_queue = ();
			}
			else {

				$next_process = shift @process_item_queue;
				print_log( "Starting process item " . $next_process,
					"INFORMATIONAL", "check_process_item_queue" );
				start_process_name($this_process);
			}
		}
		return;
	}
	else {
		return;
		print_log( "Process_item_queue is empty",
			"INFORMATIONAL", "check_process_item_queue" );
	}

}

#-----------------------------------------------------------------------
# start a process given its name
#-----------------------------------------------------------------------
sub start_process_name {
	my $this_process = $_[0];
	my $command      = "start \$" . $this_process . ";";
	print_log( "running " . $command, "INFORMATIONAL", "start_process_name" );
	eval $command;
	return;
}
########################################################################
# History:
# 19-02-2021 new code