Re: [mh] Is this the expected behaviour of process_item?
Giles Godart-Brown <[email protected]> Fri, 19 Feb 2021 19:17:40 +0000
| Newsgroups | gmane.comp.misc.misterhouse.user |
|---|---|
| Message-ID | <[email protected]> |
Attached is a first go at building a process_item_queue. Its a single queue, and I'm sure one queue per process_item would be better, but I've run out of talent :-) Giles On 18/02/2021 18:28, Giles Godart-Brown wrote: > I was going to build it for myself, then if its good enough (?) and > fulfils backwards compatibility I will add a pull > > G > > On 18/02/2021 17:25, Jeff Siddall via misterhouse-users wrote: >> Giles, >> >> When you say "I'll build a queue" do you mean for your project or >> adding it onto process_items? As long as it was backwards compatible >> with the existing behavior of a process_item, it would be nice to add >> that onto the process_item so there is a standard way for anyone to >> push items onto a process_item queue instead of everyone implementing >> it their own way. >> >> Jeff >> >> >> On 2021-02-18 11:47 a.m., Giles Godart-Brown wrote: >>> >>> Thanks H >>> >>> I'll build a queue and update the docs over the weekend >>> >>> Giles >>> >>> On 18/02/2021 14:24, H Plato wrote: >>>> As far as I know that’s how it works, the process_item has a single >>>> thread. You can check if the process_item is still active by >>>> calling$p_longproc->done(), and then queuing >>>> the additional commands to an array. That’s how I’ve dealt with >>>> this in some of my modules >>>> >>>>> On Feb 18, 2021, at 1:22 AM, Giles Godart-Brown >>>>> <[email protected] <mailto:[email protected]>> wrote: >>>>> >>>>> I use process_items to send messages and urls, occasionally >>>>> (usually when I write bad code) I need to send a lot of emails in >>>>> quick succession but MisterHouse kills any unfinished ones, rather >>>>> than running them in parallel which is what I expected. To test >>>>> this I built a minimum reproducible case below where it launches a >>>>> process that simply sleeps for 5 seconds twice in succession. As >>>>> you can see from the print log, he first one gets killed. >>>>> >>>>> Is this normal? >>>>> >>>>> Giles >>>>> >>>>> <mytest.pl> >>>>> >>>>> print_log( "Longproc run 1", "INFORMATIONAL", "p_longproc" ); >>>>> start $p_longproc; >>>>> print_log( "Longproc run 2", "INFORMATIONAL", "p_longproc" ); >>>>> start $p_longproc; >>>>> </mytest.pl> >>>>> >>>>> <longproc.sh> >>>>> >>>>> #!/bin/sh >>>>> # shell to sleep >>>>> echo longproc start 5 second sleep pid= $$ >>>>> sleep 5 >>>>> echo "longproc end" >>>>> exit 0 >>>>> </longproc.sh> >>>>> >>>>> <print.log> >>>>> >>>>> 18/02/2021 08:10:27 [p_longproc] INFO Longproc run 1 >>>>> 18/02/2021 08:10:27 [p_longproc] INFO Longproc run 2 >>>>> Warning, a previous 'start' on this process has not finished yet >>>>> Killing unfinished process id 9380 >>>>> cmd=//home/pi/mh/GGBcode/procs/longproc.sh >>>>> >>>>> ... >>>>> >>>>> 18/02/2021 08:10:33 [p_longproc] longproc start 5 second sleep >>>>> pid= 9381 >>>>> 18/02/2021 08:10:33 [p_longproc] longproc end >>>>> >>>>> </print.log> >>>>> >>>>> ________________________________________________________ >>>>> To unsubscribe from this list, go to: >>>>> https://lists.sourceforge.net/lists/listinfo/misterhouse-users >>>>> >>>> >>> >>> <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient> >>> Virus-free. www.avg.com >>> <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient> >>> >>> >>> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> >>> >>> >>> ________________________________________________________ >>> To unsubscribe from this list, go to: >>> https://lists.sourceforge.net/lists/listinfo/misterhouse-users >>> >> >> >> ________________________________________________________ >> To unsubscribe from this list, go to: >> https://lists.sourceforge.net/lists/listinfo/misterhouse-users >> -- This email has been checked for viruses by AVG. https://www.avg.com ________________________________________________________ To unsubscribe from this list, go to: https://lists.sourceforge.net/lists/listinfo/misterhouse-users
process_item_queue.pl
(text/plain, 4.1 KB)
=head1 B<Process_Item_queue>
=head2 SYNOPSIS
=head2 DESCRIPTION
use this object to add process items to a queue.
This version has a single queue called @process_item_queue, the next item on the queue is forked when the previous one finishes
=head2 INHERITS
<>
=head2 METHODS
add_to_queue add a process item to the queue
check_process_item_queue This is called regularly to see if it has any items to process on the queue
=over
=cut
use strict;
use Data::Dumper;
#-----------------------------------------------------------------------
# tests
#-----------------------------------------------------------------------
# noloop=start
$my_test1 = new Voice_Cmd 'Run test [1,2,3,4]';
$p_longproc = new Process_Item;
set $p_longproc $config_parms{"code_dir"} . "/procs/longproc.sh";
$p_longproc->set_output( $config_parms{data_dir} . "/logs/longproc.txt" );
$p_longproc->set_timeout(10);
# noloop=stop
if ( $state = said $my_test1) {
if ( $state == 1 ) {
add_process_item_to_queue("p_longproc");
check_process_item_queue();
}
elsif ( $state == 2 ) {
print_log( "Longproc run 1", "INFORMATIONAL", "p_longproc" );
add_process_item_to_queue("p_longproc");
print_log( "Longproc run 2", "INFORMATIONAL", "p_longproc" );
add_process_item_to_queue("p_longproc");
}
elsif ( $state == 3 ) {
}
elsif ( $state == 4 ) {
}
}
if ( done_now $p_longproc) {
print_log( "Longproc done", "INFORMATIONAL", "p_longproc" );
if ( timed_out $p_longproc) {
print_log( 'longproc - timed out', "ERROR", "p_longproc" );
}
print_process_log( "longproc.txt", "p_longproc" );
}
#-----------------------------------------------------------------------
# every seconds check the queue
#-----------------------------------------------------------------------
if ($New_Second) {
check_process_item_queue();
}
#-----------------------------------------------------------------------
# globals
#-----------------------------------------------------------------------
#-----------------------------------------------------------------------
# Add a process item to the queue
#-----------------------------------------------------------------------
sub add_process_item_to_queue {
my $this_process = $_[0];
print_log( "Adding " . $this_process . " to the queue",
"INFORMATIONAL", "add_process_item_to_queue" );
if ( scalar @process_item_queue == 0 ) {
start_process_name($this_process);
}
push @process_item_queue, $this_process;
}
#-----------------------------------------------------------------------
# check the process_item_queue
# if the previous one is done, delete it and action the next
#-----------------------------------------------------------------------
sub check_process_item_queue {
my ( $this_process, $next_process );
my $count_items_in_queue = scalar @process_item_queue;
if ( $count_items_in_queue > 0 ) {
print_log(
"there are " . $count_items_in_queue . " items in the queue",
"INFORMATIONAL", "check_process_item_queue" );
$this_process = @process_item_queue[0];
my $this_process_object = get_object_by_name($this_process);
if ( !defined( $this_process_object->{done} ) )
{ # this process is still running
return;
}
else {
print_log( $this_process . " has finished",
"INFORMATIONAL", "check_process_item_queue" );
if ( $count_items_in_queue == 1 ) {
@process_item_queue = ();
}
else {
$next_process = shift @process_item_queue;
print_log( "Starting process item " . $next_process,
"INFORMATIONAL", "check_process_item_queue" );
start_process_name($this_process);
}
}
return;
}
else {
return;
print_log( "Process_item_queue is empty",
"INFORMATIONAL", "check_process_item_queue" );
}
}
#-----------------------------------------------------------------------
# start a process given its name
#-----------------------------------------------------------------------
sub start_process_name {
my $this_process = $_[0];
my $command = "start \$" . $this_process . ";";
print_log( "running " . $command, "INFORMATIONAL", "start_process_name" );
eval $command;
return;
}
########################################################################
# History:
# 19-02-2021 new code