Re: slow AD ldap query or bad filter/algorithm?

--001a11443984796adf05269d720e
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

2015-12-11 12:12 GMT+01:00 Natxo Asenjo <[email protected]>:

> hi,
>
> I need to get mail enabled groups info from Active Directory.
>
> AD mail enabled groups (be it distribution or security groups) keeps the
> member attributes as distinguishednames
>
> Example:
>
> CN=3Dmailgroup,OU=3DGroep,DC=3Ddomain,DC=3Dtld
> objectClass: top
> objectClass: group
> cn: mailgroup
> member: CN=3Duser a,OU=3Dstaff,dc=3Ddomain,dc=3Dtld
> member: CN=3Duser b,OU=3Dstaff,dc=3Ddomain,dc=3Dtld
> member: CN=3Duser c,OU=3Dstaff,dc=3Ddomain,dc=3Dtld
> member: CN=3Duser d,OU=3Dstaff,dc=3Ddomain,dc=3Dtld
>
> etc
>
> So in order to get the mail addresses of those users, I need to launch
> another query.
>
> So I first query the mail enabled groups, fast query:
>
> my $mail_enabled_grps_AD =3D "(&(objectCategory=3Dgroup)(mail=3D*))";
>
> # first find enabled accounts to fill @ad_enabled
> while (1) {
>     my $search_ad =3D $ad_ldap->search(
>         base   =3D> "dc=3Ddomain,dc=3Dtld",
>         scope  =3D> "sub",
>         filter =3D> $mail_enabled_grps_AD,
>         attr =3D>
>           [ 'cn', 'member', 'mail', 'proxyaddresess', 'distinguishedname'=
,
> ],
>         control =3D> [$page_ad],
>     );
>
>     $search_ad->code && die "error on search ad: $@: " . $search_ad->erro=
r;
>     while ( my $entry =3D $search_ad->pop_entry() ) {
>         my $displayname =3D $entry->get_value('cn');
>         my $dn          =3D $entry->get_value('distinguishedname');
>         my $mail        =3D $entry->get_value('mail');
>         my @members     =3D $entry->get_value('member');
>         my @email_addrs =3D $entry->get_value('proxyaddresses');
>
>         # fill @ad_enabled
>         push @ad_mail_enbld_groups, lc $mail;
>
>         my @ad_mails =3D _from_dn_to_mail(@members);
>
>         # generate hash for hoh_AD
>         my $rec =3D {
>             MAIL          =3D> $mail,
>             MEMBERS       =3D> [@members],
>             MEMBERS_ADDRS =3D> [@ad_mails],
>             PROXYADDRS    =3D> [@email_addrs],
>         };
>
>         # assing $rec to %hoh_AD
>         $hoh_AD{$mail} =3D $rec;
>
>     }
>
>     my ($resp) =3D $search_ad->control(LDAP_CONTROL_PAGED) or last;
>     $cookie_ad =3D $resp->cookie or last;
>     $page_ad->cookie($cookie_ad);
> }
>
> If I do not use   my @ad_mails =3D _from_dn_to_mail(@members); then this
> snippet runs under 2 seconds.
>
> If I use it, it takes 2m30secs.
>
> This is the code in the sub(s):
>
> sub _get_ad_user_mail {
>     my ($ad_user) =3D @_;
>     my $search_ad =3D $ad_ldap->search(
>         base  =3D> "dc=3Ddomain,dc=3Dtld",
>         scope =3D> "sub",
>         filter =3D>
> "(&(objectclass=3Duser)(objectcategory=3Dperson)(distinguishedname=3D$ad_=
user))",
>         attr =3D> ['mail'],
>     );
>
>     $search_ad->code && die "error on search ad: $@: " . $search_ad->erro=
r;
>     for my $entry ( $search_ad->entries ) {
>         my $ad_user_mail =3D $entry->get_value('mail');
>         return $ad_user_mail;
>     }
> }
>
> sub _from_dn_to_mail {
>     my (@members) =3D @_;
>     my @ad_mail;
>     for my $member (@members) {
>         push @ad_mail, _get_ad_user_mail($member);
>     }
>     return @ad_mail;
>
> }
>
>
> I mean, it works, but is it normal that it's so slow or am I missing
> something very obvious?
>
> Thanks for you input.
>



You should do a search directly on the member DN (with scope base) instead
of requestind the suffix in sub _get_ad_user_mail :



sub _get_ad_user_mail {
    my ($ad_user) =3D @_;
    my $search_ad =3D $ad_ldap->search(
        base  =3D> $ad_user,
        scope =3D> "base",
        filter =3D> "(objectclass=3D*)",
        attr =3D> ['mail'],
    );

    $search_ad->code && die "error on search ad: $@: " . $search_ad->error;
    for my $entry ( $search_ad->entries ) {
        my $ad_user_mail =3D $entry->get_value('mail');
        return $ad_user_mail;
    }
}



Cl=C3=A9ment.

--001a11443984796adf05269d720e
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><br><div class=3D"gmail_extra"><br><div class=3D"gmail_quo=
te">2015-12-11 12:12 GMT+01:00 Natxo Asenjo <span dir=3D"ltr">&lt;<a href=
=3D"mailto:[email protected]" target=3D"_blank">[email protected]=
</a>&gt;</span>:<br><blockquote class=3D"gmail_quote" style=3D"margin:0px 0=
px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div =
dir=3D"ltr"><div>hi,<br><br></div><div>I need to get mail enabled groups in=
fo from Active Directory.<br><br></div>AD mail enabled groups (be it distri=
bution or security groups) keeps the member attributes as distinguishedname=
s<br clear=3D"all"><div><div><div><div><div><br>Example:<br><br>CN=3Dmailgr=
oup,OU=3DGroep,DC=3Ddomain,DC=3Dtld<br>objectClass: top<br>objectClass: gro=
up<br>cn: mailgroup<br>member: CN=3Duser a,OU=3Dstaff,dc=3Ddomain,dc=3Dtld<=
br>member: CN=3Duser b,OU=3Dstaff,dc=3Ddomain,dc=3Dtld<br>member: CN=3Duser=
 c,OU=3Dstaff,dc=3Ddomain,dc=3Dtld<br>member: CN=3Duser d,OU=3Dstaff,dc=3Dd=
omain,dc=3Dtld<br><br></div><div>etc<br><br></div><div>So in order to get t=
he mail addresses of those users, I need to launch another query.<br><br></=
div><div>So I first query the mail enabled groups, fast query:<br><br>my $m=
ail_enabled_grps_AD =3D &quot;(&amp;(objectCategory=3Dgroup)(mail=3D*))&quo=
t;;<br><br># first find enabled accounts to fill @ad_enabled<br>while (1) {=
<br>=C2=A0=C2=A0=C2=A0 my $search_ad =3D $ad_ldap-&gt;search(<br>=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 base=C2=A0=C2=A0 =3D&gt; &quot;dc=3Ddomai=
n,dc=3Dtld&quot;,<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 scope=C2=A0=
 =3D&gt; &quot;sub&quot;,<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 fil=
ter =3D&gt; $mail_enabled_grps_AD,<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 attr =3D&gt;<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 [ &#39;cn&#39;, &#39;member&#39;, &#39;mail&#39;, &#39;proxyaddresess&#=
39;, &#39;distinguishedname&#39;, ],<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 control =3D&gt; [$page_ad],<br>=C2=A0=C2=A0=C2=A0 );<br><br>=C2=
=A0=C2=A0=C2=A0 $search_ad-&gt;code &amp;&amp; die &quot;error on search ad=
: $@: &quot; . $search_ad-&gt;error;<br>=C2=A0=C2=A0=C2=A0 while ( my $entr=
y =3D $search_ad-&gt;pop_entry() ) {<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 my $displayname =3D $entry-&gt;get_value(&#39;cn&#39;);<br>=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 my $dn=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0 =3D $entry-&gt;get_value(&#39;distinguishedname&#3=
9;);<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 my $mail=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0 =3D $entry-&gt;get_value(&#39;mail&#39;);<br>=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 my @members=C2=A0=C2=A0=C2=A0=C2=
=A0 =3D $entry-&gt;get_value(&#39;member&#39;);<br>=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0 my @email_addrs =3D $entry-&gt;get_value(&#39;proxyaddre=
sses&#39;);<br><br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # fill @ad_en=
abled<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 push @ad_mail_enbld_gro=
ups, lc $mail;<br><br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 my @ad_mai=
ls =3D _from_dn_to_mail(@members);<br><br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 # generate hash for hoh_AD<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 my $rec =3D {<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0 MAIL=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 =3D&gt; $mail,<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 MEMBERS=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =3D&gt; [@membe=
rs],<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =
MEMBERS_ADDRS =3D&gt; [@ad_mails],<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 PROXYADDRS=C2=A0=C2=A0=C2=A0 =3D&gt; [@email=
_addrs],<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 };<br><br>=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # assing $rec to %hoh_AD<br>=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 $hoh_AD{$mail} =3D $rec;<br><br>=C2=A0=C2=A0=
=C2=A0 }<br><br>=C2=A0=C2=A0=C2=A0 my ($resp) =3D $search_ad-&gt;control(LD=
AP_CONTROL_PAGED) or last;<br>=C2=A0=C2=A0=C2=A0 $cookie_ad =3D $resp-&gt;c=
ookie or last;<br>=C2=A0=C2=A0=C2=A0 $page_ad-&gt;cookie($cookie_ad);<br>}<=
br><br></div><div>If I do not use =C2=A0 my @ad_mails =3D _from_dn_to_mail(=
@members); then this snippet runs under 2 seconds.<br><br></div><div>If I u=
se it, it takes 2m30secs.<br><br></div><div>This is the code in the sub(s):=
<br><br>sub _get_ad_user_mail {<br>=C2=A0=C2=A0=C2=A0 my ($ad_user) =3D @_;=
<br>=C2=A0=C2=A0=C2=A0 my $search_ad =3D $ad_ldap-&gt;search(<br>=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 base=C2=A0 =3D&gt; &quot;dc=3Ddomain,dc=
=3Dtld&quot;,<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 scope =3D&gt; &=
quot;sub&quot;,<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 filter =3D&gt=
;<br>&quot;(&amp;(objectclass=3Duser)(objectcategory=3Dperson)(distinguishe=
dname=3D$ad_user))&quot;,<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 att=
r =3D&gt; [&#39;mail&#39;],<br>=C2=A0=C2=A0=C2=A0 );<br><br>=C2=A0=C2=A0=C2=
=A0 $search_ad-&gt;code &amp;&amp; die &quot;error on search ad: $@: &quot;=
 . $search_ad-&gt;error;<br>=C2=A0=C2=A0=C2=A0 for my $entry ( $search_ad-&=
gt;entries ) {<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 my $ad_user_ma=
il =3D $entry-&gt;get_value(&#39;mail&#39;);<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 return $ad_user_mail;<br>=C2=A0=C2=A0=C2=A0 }<br>}<br><br>s=
ub _from_dn_to_mail {<br>=C2=A0=C2=A0=C2=A0 my (@members) =3D @_;<br>=C2=A0=
=C2=A0=C2=A0 my @ad_mail;<br>=C2=A0=C2=A0=C2=A0 for my $member (@members) {=
<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 push @ad_mail, _get_ad_user_=
mail($member);<br>=C2=A0=C2=A0=C2=A0 }<br>=C2=A0=C2=A0=C2=A0 return @ad_mai=
l;<br><br>}<br></div><div><br></div><div><br></div><div>I mean, it works, b=
ut is it normal that it&#39;s so slow or am I missing something very obviou=
s?<br><br></div><div>Thanks for you input.<span class=3D""><font color=3D"#=
888888"><br></font></span></div></div></div></div></div></div></blockquote>=
<div><br><br><br></div><div>You should do a search directly on the member D=
N (with scope base) instead of requestind the suffix in sub  _get_ad_user_m=
ail :<br><br><br><br>sub _get_ad_user_mail {<br>=C2=A0=C2=A0=C2=A0 my ($ad_=
user) =3D @_;<br>=C2=A0=C2=A0=C2=A0 my $search_ad =3D $ad_ldap-&gt;search(<=
br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 base=C2=A0 =3D&gt; $ad_user,<=
br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 scope =3D&gt; &quot;base&quot=
;,<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 filter =3D&gt; &quot;(obje=
ctclass=3D*)&quot;,<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 attr =3D&=
gt; [&#39;mail&#39;],<br>=C2=A0=C2=A0=C2=A0 );<br><br>=C2=A0=C2=A0=C2=A0 $s=
earch_ad-&gt;code &amp;&amp; die &quot;error on search ad: $@: &quot; . $se=
arch_ad-&gt;error;<br>=C2=A0=C2=A0=C2=A0 for my $entry ( $search_ad-&gt;ent=
ries ) {<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 my $ad_user_mail =3D=
 $entry-&gt;get_value(&#39;mail&#39;);<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 return $ad_user_mail;<br>=C2=A0=C2=A0=C2=A0 }<br>}<br></div><div>=
=C2=A0<br><br><br></div><div>Cl=C3=A9ment.<br></div></div></div></div>

--001a11443984796adf05269d720e--