Re: tcpsvd - Allow only x connections from somebody/hour

On Wed, 2004-12-01 at 23:45 +0100, Clemens Fischer wrote:
> * 2004-11-27 Maurice Lucas:
> 
> > From: "Clemens Fischer"
> > Sent: Saturday, November 27, 2004 4:40 PM
> >
> >> i just thought about this.  what would be the best database-implementation
> >> to keep the IP/connection# tuples?  maurice, do you have less than a few
> >> dozen or more than thousands of IPs connecting?  anyway, it might be easy
> >> to make a little program keeping those tuples in a hash and do the
> >> exec(2)'ing, as the relevant information is readily available:
> 
> > I don't get a lot of connections/hour in a normal event* but I asked because
> > when I'm the victim of a joe job I don't want my server to go to his knees.
> > Nor do I want to waste to much bandwidth.
> 
> i think this is an interesting problem.  the djb-style suggests something like
> this:
> 
>   tcpserver <host> <port> limitip <service-program>
> 
> where "limitip" is a program i'm trying to dream up.  it's supposed to:
> 
> *  keep a persistent database of TCP connections
> *  get its parameters from environment variables set in tcprules
> *  won't grow its database beyond a configuration time computable size
> *  be simple
> 
> problems:  there's no way to get around that database, because "limitip" gets
> invoked for every TCP connection and exec(2)s its argument.  it cannot know if
> the connecting TCP belongs to a session which it's supposed to keep alive,
> because any "state" will be kept only by the server program.
> 
> so for the database i want a fixed size hash table.  the algorithm is
> neccessarily humble in its goals, because it only knows about IPs trying to
> connect, not how any connections get to be handled by the server program.
> 
> please criticize the following (rough outline):
> 
> *  data kept per IP:
>    +  IP (a long number)
>    +  last_seen (timestamp, long int)
>    +  n (number of connections during monitored time period)
> *  data kept global per run:
>    +  this_time (timestamp)
>    +  timeout ($LIMITIPTO, timestamp, when an IP in the db is considered stale)
>    +  table_size ($LIMITIPSIZE)
>    +  max_collisions (how often to try to find a slot for the connection)
>    +  limit_time ($LIMITIPTIME, monitored time period in seconds)
>    +  limit_cons ($LIMITIPCONS, max number of connections during time period)
> 
> per IP data is kept in the database, which is a hash table constant in size
> for a given service.  connections in access of what it can hold will
> neccessarily fail.  do you guys think this will do?
> 
> algorithm:
> 
> (0. if timeout == $LIMITIPTO is zero, don't do anything, just exec.  for your
>     trusted host connections.)
> 1.  load database; this_time = $now;
> 2.  lookup IP in hash.  in case of collision, try max_collisions times to find
>     a better slot.  if slot.last_seen < (this_time - timeout) => squat the slot
>     for the new IP; last_seen = this_time;
> 3.  time_diff = (this_time - last_seen)
> 4.  time_diff < limit_time && n < limit_cons =>
>         ++i; last_seen = this_time; save db; exec;
> 5.  time_diff < limit_time && n > limit_cons =>
>         --i; last_seen = this_time; save db; quit;
> 6.  time_diff > limit_time =>
>         i = 0; last_seen = this_time; save db; exec;
> 
> did i overlook something obvious?  i think i'll try to code this up in "siod",
> which is a small and fast scheme interpreter made for this kind of problem.
> thus i'll be able to experiment.  but don't hold your breath :)
> 
It's been a long time since my last programming action, but I don't see
anything you have forgotten.

A little side program to read the limitip hash table and to sort all
conn for he top 10 servers which connect would be an idea.
As an admin you could run this side script and see that e.g. host
1.2.3.4 does 40% of all the connections so he could "whitelist" this
host and save the cpu-cycle. 
(this data is also available throu the tcpsvd logfiles)

For version 2.0
In the "data kept per IP" a default number for limit_cons with a manual
override throu some admintool.
Reason: From the example above where ip 1.2.3.4 takes 40% of the
connections and limit_cons is normal 100 cons/hour you could limit host
1.2.3.4 to 1000 cons/hour while the server is able to recive an 3000
cons/hour.
Whould somebody (mis)use host 1.2.3.4 this host won't be free to do as
it please.

Maurice Lucas