Generating constant RPS load for Data cloud

Hi,

We have a data-cloud setup for caching (memory+persistent). Clients talk to
cloud via REST API for key/value storage and retrieval

I have a requirement to load the data-cloud with different RPS(request per
sec) under different infrastructure and system profiles, to finally build a
matrix on performance(response-time).

The approach I have taken to load the system with n-rps in x-duration(secs):

- For each second yield to an event that again creates a new event for every
(time + 1/n-rps) interval within each second interval.

#pool -> max_open=1000
POE::Component::Client::HTTP->spawn(Alias => 'ua', ConnectionManager =>
$pool);


sub _start {
  my ($k,$h) = @_[KERNEL,HEAP];

  $h->{'payload'}      = substr('CLOUD-TEST' x 26, 0, 256);
  $h->{'payload_size'} = 256;

  $k->yield('schedule_work');
}

sub schedule_work {
  my ($k,$s,$h) = @_[KERNEL,SESSION,HEAP];

  my $time = Time::HiRes::time();

  for(my $i=1;$i<=$h->{x-duration'};$i++) {
    $k->alarm_set('gen_work', $time + $i);
  }
}

sub gen_work {
  my ($k,$s,$h) = @_[KERNEL,SESSION,HEAP];

  my $ival = 1/$h->{'rps'};

  my $time = Time::HiRes::time();

  my $_ival = 0;
  for(my $i=0;$i<$h->{'rps'};$i++) {
    $k->alarm_set('some_work', $time + $_ival, $_ival, $time, 'PUT');
    $_ival += $ival;
  }
}

sub some_work {
  my ($k,$s,$h) = @_[KERNEL,SESSION,HEAP];

  my ($frac, $t_ref, $method) = @_[ARG0,ARG1,ARG2];

  if ($method eq 'PUT') {

    $h->{'id_ival'}->{$h->{'id'}} = $frac;
    my $data = gen_header_url('PUT', $h->{'id'}++, $h->{'payload_size'});
    my $req = new HTTP::Request 'PUT' => $data->{'url'};

    $req->header(%{$data->{'header'}});
    $req->content($h->{'payload'});

    $k->post("ua" => "request", "got_response", $req);
  }
}


The above gives perfect response time for loads of 200 RPS and 180 sec
duration. But when the RPS is increased to 1000 with the same duration, the
script takes a long time and 90% of the values do not have the response data
collected. So, this is a limitation as I need to load the system with ~
20000RPS for 5 mins duration with a payload of at-least 100KB (key-value).

Can the group suggest if this is the correct approach or I need to fine tune
my code. I haven't see specfic examples for such need. Any suggestions will
be helpful,

Thanks,
Anil