Re: Dropped inbound packets from stateful allow rule

Paul B. Henson wrote:
> We're running Solaris 10U8, with the latest ipfilter patch. We're having a
> problem where inbound connections that should be allowed by a stateful rule
> are being dropped.
>
> For example:
>
> Apr 12 16:31:53 kyle ipmon[117]: [ID 702911 local0.warning] 16:31:53.771244
> e1000g0 @20:10 b 134.71.247.49,712 -> 134.71.247.14,2049 PR tcp len 20 60
> -S IN
>
> This blocked SYN packet shows it was blocked by rule 20:10:
>
> @10 pass in quick proto tcp from 134.71.0.0/16 to any port = nfsd flags
> S/SA keep state group 20
>
> This packet should have been passed by this rule, not blocked.
>
> I have an open SR with Sun, and based on a non-zero value for lost inbound
> state:
>
> packet state(in):       kept 1213044    lost 1317
>
> They're saying the problem is caused by the state table filling up and
> advise increasing the size of the state table.
>
> We did so:
>
> fr_statemax     min 0x1 max 0x7fffffff  current 72901
> fr_statesize    min 0x1 max 0x7fffffff  current 104147
>
> But were still seeing the problem. I started logging the size of the state
> table every minute, and it would appear it's nowhere near full. The state
> table only had about 300 entries when the above packet was dropped.
>   

So there are a number of different reasons why the packet might be dropped
but unfortuantely the version of code in Solaris 10 U8 is pretty old and 
doesn't
give you any additional insight.

What you might be able to do is use dtrace to provide some additional clues
by using where the return is being made from:

# dtrace -n 'fbt:ipf:fr_addstate:return/arg1==0/{@returns[arg0] = count();}'

The list of causes in 4.1.34 is:
- entry already exists
- hash chain bucket is "full"
- hash table is "full"
- kmem_alloc fails

Darren