Re: FreeBSD 10G forwarding performance @Intel

On 04.07.2012 00:27, Luigi Rizzo wrote:
> On Tue, Jul 03, 2012 at 09:37:38PM +0400, Alexander V. Chernikov wrote:
> ...
>> Thanks, another good point. I forgot to merge this option from andre's
>> patch.
>>
>> Another 30-40-50kpps to win.
>
> not much gain though.
> What about the other IPSTAT_INC counters ?
Well, we should then remove all such counters (total, forwarded) and 
per-interface statistics (at least for forwarded packets).
> I think the IPSTAT_INC macros were introduced (by rwatson ?)
> following a discussion on how to make the counters per-cpu
> and avoid the contention on cache lines.
> But they are still implemented as a single instance,
> and neither volatile nor atomic, so it is not even clear
> that they can give reliable results, let alone the fact
> that you are likely to get some cache misses.
>
> the relevant macro is in ip_var.h.
Hm. This seems to be just per-vnet structure instance.
We've got some more real DPCPU stuff (sys/pcpu.h && kern/subr_pcpu.c) 
which can be used for global ipstat structure, however since it is 
allocated from single area without possibility to free we can't use it 
for per-interface counters.

I'll try to run tests without any possibly contested counters and report 
the results on Thursday.
>
> Cheers
> luigi
>
>>
>> +u_int rt_count  = 1;
>> +SYSCTL_INT(_net, OID_AUTO, rt_count, CTLFLAG_RW,&rt_count, 1, "");
>>
>> @@ -601,17 +625,20 @@ passout:
>>          if (error != 0)
>>                  IPSTAT_INC(ips_odropped);
>>          else {
>> -               ro.ro_rt->rt_rmx.rmx_pksent++;
>> +               if (rt_count)
>> +                       ro.ro_rt->rt_rmx.rmx_pksent++;
>>                  IPSTAT_INC(ips_forward);
>>                  IPSTAT_INC(ips_fastforward);
>>
>>
>>>
>>> cheers
>>> luigi
>>>
>>
>>
>> --
>> WBR, Alexander
>> _______________________________________________
>> [email protected] mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-net
>> To unsubscribe, send any mail to "[email protected]"
> _______________________________________________
> [email protected] mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "[email protected]"
>


-- 
WBR, Alexander
_______________________________________________
[email protected] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[email protected]"