Re: [docs] [PATCH 1/8] conf.py: add linkcheck builder exclusions

Hi Antonin,

On 7/15/26 2:30 PM, Antonin Godard wrote:
> Hi,
> 
> On Fri Jul 10, 2026 at 5:09 PM CEST, Quentin Schulz via lists.yoctoproject.org wrote:
>> Hi Antonin,
>>
>> On 7/10/26 4:46 PM, Antonin Godard via lists.yoctoproject.org wrote:
>>> The linkcheck builder can be used to scout for broken links. By looking at
>>> the output of:
>>>
>>>     grep -E -r --no-filename -o 'href="http.://[^/"]+' | sort | uniq -c | sort -nr
>>>
>>> from the HTML output directory, exclude links that are too frequent.
>>
>> Because they are too frequent it's impossible they are broken?
>> Considering you fix links that are in this list later in this series, I
>> don't think this holds. At least provide a proper justification because
>> "too frequent" doesn't make sense.
> 
> I see your point. I did not want to hammer those websites, that is all. I'll

And that is absolutely fair!

> drop this patch I think. Maybe I could add this but commented out, in case
> someone also wants to apply those exclusions.
> 

I could see it being disabled by default and having an environment 
variable enabling the check so that we can quickly run a check at least 
once every now and then. Due to AI scrapers, most websites now block 
bots and linkchecker is seen as one so we'll also have a bunch of links 
one will need to verify manually...

Maybe something like:

"""
linkcheck_ignore = []

if os.environ.get('LINKCHECK_NOT_NICE'):
     linkcheck_ignore.extend([r'https?://nvd\.nist\.gov.*', ...])
"""

Cheers,
Quentin