Heads up: The Yarn.social (yarns) search engine is crawling twtxt feeds over gopher://

James Mills <[email protected]> Sat, 18 Sep 2021 12:25:48 +1000
Newsgroups gmane.network.gopher.general
Message-ID <CALGqR9KNwTGQg5=4YZTNjqx=HdrV1xYMPE550v5mStwOnetkMg@mail.gmail.com>
Hi All,

Just a heads up that if you have a twtxt feed you host over gopher:// the
Yarn.social search engine (called yarns) hosted at https://search.twtxt.net/
may possibly crawl your feed.

@Cameron Kaiser <spectre-nb4sah8SP49Wk0Htik3J/[email protected]> I apologize in advance, but _some_
users have started following _some_ feeds over your floodgap proxy. To
address this I've recently added native support for Gopher feeds to
Yarn.social pods (the `yarnd` software) -- If you can help me figure out
who/what is still hitting your proxy we can mitigate this and get them
migrated off.

The search engine currently re-crawls the entire space, once-per-day. No
more at this time.

If you have any concerns or questions don't hesitate to contact me!

Kind regards

James

James Mills / prologic

E: [email protected]
W: prologic.shortcircuit.net.au
Blog:  Read my Blog <https://prologic.blog/>
Twtxt: Follow me on twtxt.net <https://twtxt.net/user/prologic>