Re: Comments on Composition and Comportment of the IETF Nominating Committee draft
Dan Harkins <[email protected]> Fri, 24 Jul 2026 23:52:01 -0700
| Newsgroups | gmane.ietf.general |
|---|---|
| Message-ID | <[email protected]> |
This is a multi-part message in MIME format. --Boundary_(ID_mUF3KNhkS/CQ0Uf0aXNTaw) Content-type: text/plain; charset=UTF-8; format=flowed Content-transfer-encoding: 8BIT On 7/6/26 11:10 PM, Mallory Knodel wrote: > Hi Joel, > > On Mon, Jul 6, 2026 at 10:42 PM Joel Halpern <[email protected]> wrote: > > I think without violating the confidentiality requirements, I can > make a couple of observations. Other past nomcom members / chairs > / liaisons can provide comments as they feel appropriate. > > First, one of the difficulties is that the nomcom must treat the > feedback as confidential. Great care is taken to not violate > that. Nonetheless, committees do learn about issues with specific > candidates from the feedback, and it is important to identify > those issues. Whether they decide to ask the candidate about > aspects of that ought, as far as I can tell, be left to the > discretion of the committee. > > In my experience committee members are generally good at > separating substantive feedback issues from feedback driven by > attributes that shouldn't matter. There are many dimensions > besides the ones you are focusing on that the committee needs to > watch for and avoid. I don't think trying to prevent bias in the > feedback is desirable or achievable. > > It is the goal of this section of this document to intervene against > discrimination in leadership appointments by the nomcom. You say "intervene against discrimination" but all you have to appeal to regarding discrimination is representation that is not consistent with some random distribution. This is the classic "residual fallacy": https://www.youtube.com/watch?v=E4LjWMITwXs > > You are validating that, indeed, current processes will change as a > result of non-discrimination efforts. This is inherently good. Also, > diverse teams perform better, which means these process changes will > result in better decisions and better outcomes for the IETF. Can you provide some relevant justification for that statement that "diverse teams perform better" that would apply to the nom-com (and not some arbitrary grouping of people for an unreproducable social science experiment). Please quantify how the changes you are proposing will result in better decisions and outcomes for the IETF. > Win-win. Huh. Not seeing that. Dan. > > Best, > -Mallory > > It is also worth noting that we are interviewing people for many > different jobs. At the very least, even the structured part of > the interview is likely to be different for the different > positions. This then gets even more complicated when the > committee may be evaluating person A based on strength X and > weakness Y, and person B on strength Q and weakness R. The > interview may well want to explore those specific qualities for > that specific candidate. Even when we know the job, we know that > people will do the job differently. And the selection process is > even more complicated by the fact that the job itself changes over > time. In fact, one of the tasks the nomcom undertakes is > deciding what properties they think are important for which positions. > > I suspect that further analysis would reveal even more nuances in > the interview and evaluation process. Yes, it is important to be > clear that we avoid bias in our selection. I don't think tying > the hands of the committee in terms of what they ask candidates is > helpful. > > In some sense, the structured part of the interview presumably > starts with the common questionnaire. I am not asking or > suggesting that the questionnaire be tuned to the individual > candidate. But that is only the starting point. > > Yours, > > Joel > > On 7/6/2026 10:25 PM, Mallory Knodel wrote: >> Hi Joel, >> >> I think specific candidate questions, if not related follow ups >> to the structured interview questions, can invite discrimination. >> >> OTOH, the US OPM goes to the other logical end: Employers should >> use structured interviews in which all candidates are asked the >> same core, job-related questions and are evaluated using the same >> criteria. Any follow-up questions should be limited to clarifying >> a candidate’s responses and should be used consistently to >> maintain fairness and comparability across candidates. >> >> Probably there’s a middle ground. >> >> It would be worth interrogating the role of community feedback. >> Obviously this is a unique element and guidance for most hiring >> processes won’t have much to say about it. If we worry about >> discrimination in nomcom decision making we should probably worry >> about it in candidate feedback too. So what would be the fair >> mitigation? Giving a candidate the chance to rebut a potentially >> discriminatory feedback comment? Or not mentioning the comment >> during the interview at all, and giving the nomcom discretion to >> consider feedback in its own right to supplement its decision? >> >> Sorry I pointed the question back. But I’m genuinely not sure >> beyond my own experiences and intuition, which I’ll keep to >> myself as an author of this draft, which can hopefully encompass >> community consensus on the topic. >> >> -Mallory >> >> On Tue, Jul 7, 2026 at 11:11 Joel Halpern <[email protected]> >> wrote: >> >> With regard to the structured interview item in your draft, I >> was unclear whether your documents intends to say that each >> interview should include the structured questions or were you >> aiming to say that the interviews MUST ONLY consist of the >> structured questions. >> >> As far as I know, the former is common practice, and a good >> idea. The latter would unduly constrain the committee from >> asking about things they think are important to the specific >> candidate, for example based on the feedback they have received. >> >> Yours, >> >> Joel >> >> On 7/6/2026 10:03 PM, Mallory Knodel wrote: >>> Hi, >>> >>> On Tue, Jul 7, 2026 at 01:11 S Moonesamy >>> <[email protected] <mailto:sm%[email protected]>> wrote: >>> >>> Hi Mallory, >>> At 05:13 AM 06-07-2026, Mallory Knodel wrote: >>> >>> >>> >I'm not sure I remember. Could you elaborate this >>> comment please? >>> >>> It's in RFC 7704 (Page 4). >>> >>> >>> Incredible. Thanks a lot for sharing this. I’ll cite it in >>> the next version. >>> >>> >>> >Disclosure of interest to the chair, who then decides >>> if there's a >>> >conflict. Simple recusal. >>> >>> It could be a bit more complicated than that. >>> >>> >>> Certainly but I don’t know if the document can or should try >>> to encompass the complexity or simply describe a process for >>> addressing it. >>> >>> I am open to text suggestions that better elaborate this and >>> any of the comportment sections. >>> >>> >>> >It's here. I've edited the main copy. >>> ><https://urldefense.proofpoint.com/v2/url?u=https-3A__hbr.org_2016_04_how-2Dto-2Dtake-2Dthe-2Dbias-2Dout-2Dof-2Dinterviews&d=DwIBAg&c=slrrB7dE8n7gBJbeO0g-IQ&r=krANNudPSfUTEf2kXiduBUqRjXhDsKNCASr1kibHLfs&m=YIrtQwrRtb_e31ldkUQjV6tcWTEcpf5fCSgJN5EFLRkjqZ3pHWlqlpLKxKQuPKb-&s=MBhR2Q0q-JMD3j71Sm58-ZD61u61e_r2AQAlRgDPkEI&e= >>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__hbr.org_2016_04_how-2Dto-2Dtake-2Dthe-2Dbias-2Dout-2Dof-2Dinterviews&d=DwIBAg&c=slrrB7dE8n7gBJbeO0g-IQ&r=krANNudPSfUTEf2kXiduBUqRjXhDsKNCASr1kibHLfs&m=YIrtQwrRtb_e31ldkUQjV6tcWTEcpf5fCSgJN5EFLRkjqZ3pHWlqlpLKxKQuPKb-&s=MBhR2Q0q-JMD3j71Sm58-ZD61u61e_r2AQAlRgDPkEI&e=> >>> >https://urldefense.proofpoint.com/v2/url?u=https-3A__hbr.org_2016_04_how-2Dto-2Dtake-2Dthe-2Dbias-2Dout-2Dof-2Dinterviews&d=DwIBAg&c=slrrB7dE8n7gBJbeO0g-IQ&r=krANNudPSfUTEf2kXiduBUqRjXhDsKNCASr1kibHLfs&m=YIrtQwrRtb_e31ldkUQjV6tcWTEcpf5fCSgJN5EFLRkjqZ3pHWlqlpLKxKQuPKb-&s=MBhR2Q0q-JMD3j71Sm58-ZD61u61e_r2AQAlRgDPkEI&e= >>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__hbr.org_2016_04_how-2Dto-2Dtake-2Dthe-2Dbias-2Dout-2Dof-2Dinterviews&d=DwIBAg&c=slrrB7dE8n7gBJbeO0g-IQ&r=krANNudPSfUTEf2kXiduBUqRjXhDsKNCASr1kibHLfs&m=YIrtQwrRtb_e31ldkUQjV6tcWTEcpf5fCSgJN5EFLRkjqZ3pHWlqlpLKxKQuPKb-&s=MBhR2Q0q-JMD3j71Sm58-ZD61u61e_r2AQAlRgDPkEI&e=> >>> >>> >>> Thanks. The answers to the questions would not be >>> relevant in my opinion. >>> >>> >>> Fair. My thinking was to provide some authoritative example >>> of how industry thinks about this. I’m open to citing a >>> better and more relevant publication. >>> >>> Thanks, >>> -Mallory >> -- "The object of life is not to be on the side of the majority, but to escape finding oneself in the ranks of the insane." -- Marcus Aurelius --Boundary_(ID_mUF3KNhkS/CQ0Uf0aXNTaw) Content-type: text/html; charset=UTF-8 Content-transfer-encoding: 8BIT <!DOCTYPE html> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> </head> <body> <br> <br> <div class="moz-cite-prefix">On 7/6/26 11:10 PM, Mallory Knodel wrote:<br> </div> <blockquote type="cite" cite="mid:CABMXHrGzRuxZBNXmxtMG8FMu+Wt6v9i=cNyzXXLg9CAC+mN+Ng@mail.gmail.com"> <meta http-equiv="content-type" content="text/html; charset=UTF-8"> <div dir="ltr"> <div dir="ltr">Hi Joel,</div> <br> <div class="gmail_quote gmail_quote_container"> <div dir="ltr" class="gmail_attr">On Mon, Jul 6, 2026 at 10:42 PM Joel Halpern <<a href="mailto:[email protected]" moz-do-not-send="true" class="moz-txt-link-freetext">[email protected]</a>> wrote:<br> </div> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div> <p>I think without violating the confidentiality requirements, I can make a couple of observations. Other past nomcom members / chairs / liaisons can provide comments as they feel appropriate.</p> <p>First, one of the difficulties is that the nomcom must treat the feedback as confidential. Great care is taken to not violate that. Nonetheless, committees do learn about issues with specific candidates from the feedback, and it is important to identify those issues. Whether they decide to ask the candidate about aspects of that ought, as far as I can tell, be left to the discretion of the committee.</p> <p>In my experience committee members are generally good at separating substantive feedback issues from feedback driven by attributes that shouldn't matter. There are many dimensions besides the ones you are focusing on that the committee needs to watch for and avoid. I don't think trying to prevent bias in the feedback is desirable or achievable. </p> </div> </blockquote> <div>It is the goal of this section of this document to intervene against discrimination in leadership appointments by the nomcom.</div> </div> </div> </blockquote> <br> You say "intervene against discrimination" but all you have to appeal to <br> regarding discrimination is representation that is not consistent with some <br> random distribution. This is the classic "residual fallacy":<br> <br> <a class="moz-txt-link-freetext" href="https://www.youtube.com/watch?v=E4LjWMITwXs">https://www.youtube.com/watch?v=E4LjWMITwXs</a><br> <br> <blockquote type="cite" cite="mid:CABMXHrGzRuxZBNXmxtMG8FMu+Wt6v9i=cNyzXXLg9CAC+mN+Ng@mail.gmail.com"> <div dir="ltr"> <div class="gmail_quote gmail_quote_container"> <div><br> </div> <div>You are validating that, indeed, current processes will change as a result of non-discrimination efforts. This is inherently good. Also, diverse teams perform better, which means these process changes will result in better decisions and better outcomes for the IETF.</div> </div> </div> </blockquote> <br> Can you provide some relevant justification for that statement that<br> "diverse teams perform better" that would apply to the nom-com (and<br> not some arbitrary grouping of people for an unreproducable social<br> science experiment). Please quantify how the changes you are<br> proposing will result in better decisions and outcomes for the IETF.<br> <br> <blockquote type="cite" cite="mid:CABMXHrGzRuxZBNXmxtMG8FMu+Wt6v9i=cNyzXXLg9CAC+mN+Ng@mail.gmail.com"> <div dir="ltr"> <div class="gmail_quote gmail_quote_container"> <div>Win-win.</div> </div> </div> </blockquote> <br> Huh. Not seeing that.<br> <br> Dan.<br> <br> <blockquote type="cite" cite="mid:CABMXHrGzRuxZBNXmxtMG8FMu+Wt6v9i=cNyzXXLg9CAC+mN+Ng@mail.gmail.com"> <div dir="ltr"> <div class="gmail_quote gmail_quote_container"> <div><br> </div> <div>Best,</div> <div>-Mallory</div> <div><br> </div> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div> <p>It is also worth noting that we are interviewing people for many different jobs. At the very least, even the structured part of the interview is likely to be different for the different positions. This then gets even more complicated when the committee may be evaluating person A based on strength X and weakness Y, and person B on strength Q and weakness R. The interview may well want to explore those specific qualities for that specific candidate. Even when we know the job, we know that people will do the job differently. And the selection process is even more complicated by the fact that the job itself changes over time. In fact, one of the tasks the nomcom undertakes is deciding what properties they think are important for which positions.</p> <p>I suspect that further analysis would reveal even more nuances in the interview and evaluation process. Yes, it is important to be clear that we avoid bias in our selection. I don't think tying the hands of the committee in terms of what they ask candidates is helpful. </p> <p>In some sense, the structured part of the interview presumably starts with the common questionnaire. I am not asking or suggesting that the questionnaire be tuned to the individual candidate. But that is only the starting point.</p> <p>Yours,</p> <p>Joel</p> <div>On 7/6/2026 10:25 PM, Mallory Knodel wrote:<br> </div> <blockquote type="cite"> <div dir="auto">Hi Joel,</div> <div dir="auto"><br> </div> <div dir="auto">I think specific candidate questions, if not related follow ups to the structured interview questions, can invite discrimination. </div> <div dir="auto"><br> </div> <div dir="auto">OTOH, the US OPM goes to the other logical end: <span style="font-size:inherit;font-family:-apple-system,sans-serif">Employers should use structured interviews in which all candidates are asked the same core, job-related questions and are evaluated using the same criteria. Any follow-up questions should be limited to clarifying a candidate’s responses and should be used consistently to maintain fairness and comparability across candidates.</span></div> <div dir="auto"><span style="font-size:inherit;font-family:-apple-system,sans-serif"><br> </span></div> <div dir="auto"><span style="font-size:inherit;font-family:-apple-system,sans-serif">Probably there’s a middle ground. </span></div> <div dir="auto"><span style="font-size:inherit;font-family:-apple-system,sans-serif"><br> </span></div> <div dir="auto"><font face="-apple-system, sans-serif"><span style="font-size:inherit">It would be worth interrogating the role of community feedback. Obviously this is </span>a unique<span style="font-size:inherit"> element and guidance for most hiring processes won’t have much to say about it. If we worry about discrimination in nomcom decision making we should probably worry about it in candidate feedback too. So what would be the fair mitigation? Giving a candidate the chance to rebut a potentially discriminatory feedback comment? Or not mentioning the comment during the interview at all, and giving the nomcom discretion to consider feedback in its own right to supplement its decision?</span></font></div> <div dir="auto"><font face="-apple-system, sans-serif"><span style="font-size:inherit"><br> </span></font></div> <div dir="auto"><font face="-apple-system, sans-serif"><span style="font-size:inherit">Sorry I pointed the question back. But I’m genuinely not sure beyond my own experiences and intuition, which I’ll keep to myself as an author of this draft, which can hopefully encompass community consensus on the topic. </span></font></div> <div dir="auto"><font face="-apple-system, sans-serif"><span style="font-size:inherit"><br> </span></font></div> <div dir="auto"><font face="-apple-system, sans-serif"><span style="font-size:inherit">-Mallory </span></font></div> <div><br> <div class="gmail_quote"> <div dir="ltr" class="gmail_attr">On Tue, Jul 7, 2026 at 11:11 Joel Halpern <<a href="mailto:[email protected]" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">[email protected]</a>> wrote:<br> </div> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div> <p>With regard to the structured interview item in your draft, I was unclear whether your documents intends to say that each interview should include the structured questions or were you aiming to say that the interviews MUST ONLY consist of the structured questions.</p> <p>As far as I know, the former is common practice, and a good idea. The latter would unduly constrain the committee from asking about things they think are important to the specific candidate, for example based on the feedback they have received.</p> <p>Yours,</p> <p>Joel</p> </div> <div> <div>On 7/6/2026 10:03 PM, Mallory Knodel wrote:<br> </div> <blockquote type="cite"> <div dir="auto">Hi,</div> <div dir="auto"><br> <div class="gmail_quote" dir="auto"> <div dir="ltr" class="gmail_attr">On Tue, Jul 7, 2026 at 01:11 S Moonesamy <<a href="mailto:sm%[email protected]" target="_blank" moz-do-not-send="true">[email protected]</a>> wrote:<br> </div> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" dir="auto">Hi Mallory,<br> At 05:13 AM 06-07-2026, Mallory Knodel wrote:<br> <br> <br> >I'm not sure I remember. Could you elaborate this comment please?<br> <br> It's in RFC 7704 (Page 4).<br> </blockquote> <div dir="auto"><br> </div> <div dir="auto">Incredible. Thanks a lot for sharing this. I’ll cite it in the next version. </div> <div dir="auto"><br> </div> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" dir="auto"><br> >Disclosure of interest to the chair, who then decides if there's a <br> >conflict. Simple recusal.<br> <br> It could be a bit more complicated than that.<br> </blockquote> <div dir="auto"><br> </div> <div dir="auto">Certainly but I don’t know if the document can or should try to encompass the complexity or simply describe a process for addressing it. </div> <div dir="auto"><br> </div> <div dir="auto">I am open to text suggestions that better elaborate this and any of the comportment sections. </div> <div dir="auto"><br> </div> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" dir="auto"><br> >It's here. I've edited the main copy.<br> ><<a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__hbr.org_2016_04_how-2Dto-2Dtake-2Dthe-2Dbias-2Dout-2Dof-2Dinterviews&d=DwIBAg&c=slrrB7dE8n7gBJbeO0g-IQ&r=krANNudPSfUTEf2kXiduBUqRjXhDsKNCASr1kibHLfs&m=YIrtQwrRtb_e31ldkUQjV6tcWTEcpf5fCSgJN5EFLRkjqZ3pHWlqlpLKxKQuPKb-&s=MBhR2Q0q-JMD3j71Sm58-ZD61u61e_r2AQAlRgDPkEI&e=" rel="noreferrer" target="_blank" moz-do-not-send="true">https://urldefense.proofpoint.com/v2/url?u=https-3A__hbr.org_2016_04_how-2Dto-2Dtake-2Dthe-2Dbias-2Dout-2Dof-2Dinterviews&d=DwIBAg&c=slrrB7dE8n7gBJbeO0g-IQ&r=krANNudPSfUTEf2kXiduBUqRjXhDsKNCASr1kibHLfs&m=YIrtQwrRtb_e31ldkUQjV6tcWTEcpf5fCSgJN5EFLRkjqZ3pHWlqlpLKxKQuPKb-&s=MBhR2Q0q-JMD3j71Sm58-ZD61u61e_r2AQAlRgDPkEI&e=</a> ><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__hbr.org_2016_04_how-2Dto-2Dtake-2Dthe-2Dbias-2Dout-2Dof-2Dinterviews&d=DwIBAg&c=slrrB7dE8n7gBJbeO0g-IQ&r=krANNudPSfUTEf2kXiduBUqRjXhDsKNCASr1kibHLfs&m=YIrtQwrRtb_e31ldkUQjV6tcWTEcpf5fCSgJN5EFLRkjqZ3pHWlqlpLKxKQuPKb-&s=MBhR2Q0q-JMD3j71Sm58-ZD61u61e_r2AQAlRgDPkEI&e=" rel="noreferrer" target="_blank" moz-do-not-send="true">https://urldefense.proofpoint.com/v2/url?u=https-3A__hbr.org_2016_04_how-2Dto-2Dtake-2Dthe-2Dbias-2Dout-2Dof-2Dinterviews&d=DwIBAg&c=slrrB7dE8n7gBJbeO0g-IQ&r=krANNudPSfUTEf2kXiduBUqRjXhDsKNCASr1kibHLfs&m=YIrtQwrRtb_e31ldkUQjV6tcWTEcpf5fCSgJN5EFLRkjqZ3pHWlqlpLKxKQuPKb-&s=MBhR2Q0q-JMD3j71Sm58-ZD61u61e_r2AQAlRgDPkEI&e=</a> <br> <br> Thanks. The answers to the questions would not be relevant in my opinion.<br> </blockquote> <div dir="auto"><br> </div> <div dir="auto">Fair. My thinking was to provide some authoritative example of how industry thinks about this. I’m open to citing a better and more relevant publication.</div> <div dir="auto"><br> </div> <div dir="auto">Thanks,</div> <div dir="auto">-Mallory </div> </div> </div> </blockquote> </div> </blockquote> </div> </div> </blockquote> </div> </blockquote> </div> </div> </blockquote> <br> <pre class="moz-signature" cols="72">-- "The object of life is not to be on the side of the majority, but to escape finding oneself in the ranks of the insane." -- Marcus Aurelius</pre> <br> </body> </html> --Boundary_(ID_mUF3KNhkS/CQ0Uf0aXNTaw)--