Finding Comics on Usenet, Power Searching and Deep Indexing.

Just some notes from PX's independently funded Usenet research. I'm a 
couple of years into this and I'm still getting deeper into deep 
retention.

First a provider update. There is only one block account provider left on 
the omicron backbone, Blocknews. Maximum Usenet went to a different 
backbone with 3000 days retention. And here's something weird. I can 
still download headers and do an index that goes back to 2008 on my 
blocks from Maximum Usenet, Cubenet, Thundernews and NewgroupDirect but 
the older articles won't load so my block accounts are still useful for 
indexing. It seems like their backbone is connected to Omicron but only 
for headers. I added an Eweka account to the collection and their 
retention goes back to August 15, 2008 vs August 8, 2008 for Block News. 
Just one week's and a few file's difference.


File search test, deep indexing with Pan vs paid indexers vs Newslazer 
and Newsbin Pro.

I got some critical remarks about my deep indexing when I brought it up 
in a movie group and someone thought it wasn't necessary at all for older 
files and there were other ways to find files.I think they were using one 
of the paid Usenet indexers to find an old flood of scifi and horror 
movies and some of them do have indexes that go back a lot more than 3 
years. The other possibility is that they were using Newsbin Pro's search 
or that of one of the provider newsreaders that have their own indexes to 
search. I've been trying a few of the paid indexers out on either a free 
or trial plan and seeing how they compare and I have Newsbin Pro and 
Newslazer as well. End result, there's no comparison at all with the paid 
indexers and Newsbin Pro and Newslazer do alright but still aren't as 
complete as searching my own indexes in Pan. Here's the test I'm running. 
I'm doing a search for "Comics History" to see how many of the books and 
magazines on the subject I can find that Symm posted a few years ago with 
"Comics History" in the subject. I'm searching my own indexes of 
a.b.comics.dcp and a.b.pictures.comics with Pan and comparing that to a 
comic search with the same term in NzbGeek, Nzb Finder, Nzb.su and DogNZB 
as example indexers and the built in search in Newsbin Pro and Newslazer. 
None of the indexers give me more than a small fraction of what Symm 
posted, usually one or two pages of search results that mostly contain 
episodes of Robert Kirkman's Secret History of Comics and at the very end 
a tiny fraction of what Symm posted to a.b.comics.dcp. and almost none of 
what he posted to other groups. He was posting Comics History in 
a.b.pictures.comics all the way to 2019 and that group doesn't seem to be 
indexed too well by the paid indexers. Only NZB finders has a browse 
groups option and when I browse a.b.comics.dcp, I do find a lot of Symm 
posts but the search function doesn't seem to cover the group too well. 
Newsbin Pro and Newslazer did much better. I got about 3 pages of Symm's 
posts from both. Neither got all the posts but they did get a lot of them 
with Newslazer and Eweka doing a bit better and catching Symm's last 
comic history uploads in 2019 in a.b.pictures.comics. Only searching my 
own indexes with Pan gives me results that are complete. When I put 
"Comics History" in the Pan search box with a.b.comics.dcp and 
a.b.pictures.comics, I get everything he posted and could download 
anything I want without bothering with an nzb or I could make an nzb for 
any or all of it. I also get a few comic history posts he posted 
occasionally to other groups. Pan lets you keep the same search term as 
you switch groups and in a couple of minutes I can do a search for a 
specific term like "comics history" accross multiple groups. The paid 
indexers do have nzbs for a lot of obfuscated movies. It looks like the 
latest way of avoiding DMCA is to obfuscate posts with ngPost and upload 
the nzbs to paid private indexers. For comics, they are not worth the 
money. Eweka with Newslazer and Newsbin Pro both are better at finding 
comics.



Using Eweka's indexes with Newslazer. 

I just got Eweka a few weeks ago and Eweka comes with a newsreader, 
Newslazer. With it you can browse groups with Eweka's own indexes as well 
as do global usenet searches of it. Eweka's indexes do go back to 2008. 
It is a nice alternative to tying up my computer and internet connection 
for hours to do a complete index on my own for groups that don't have 
enough content that I'm interested in. The main downside to Eweka with 
Newslazer is that the indexes are consolidated into blocks and you get a 
result that's a block of files posted at the same time by the same 
poster. If you right click a result, you get a list of individual files 
that you can select or deselect. The other downside is that Newslazer 
categorizes groups as text or binary and it won't show binary posts in 
groups it classifies as text. The text groups load as in other news 
readers whereas the binary groups load Eweka's own indexes.



Deep Indexing with Newsbin Pro.

While reading Reddit threads on Usenet, I stumbled upon a link for a free 
license for Newsbin Pro. It only had a 7 day search subscription and it's 
a Windows program and I use Linux but for free I decided to give it a try 
with Wine. It works without issues and I've started to try it on indexing 
comic groups with Eweka. It hasn't had any problems handling large 
indexes so far and I just finished an index of 
alt.binaries.pictures.comics. It seems a bit faster than Pan for indexing 
and the databases are 50% smaller. It also has the ability to make nzbs 
out of a list of selected files like Pan. This is an item on a right 
mouse button context menu that is fairly extensive. Newsbin Pro is very 
similar to Pan in that regard. So far, it's the only newsreader I've 
found that compares to Pan for deep indexing. It seems better in some 
ways but each has something it does better. The great thing about Pan is 
that it is native Linux free and open source software that I've compiled 
myself with custom mods. But that sort of thing is not for everyone so I 
think Newsbin Pro is a good newsreader and probably the best one for 
Windows. I'm impressed on how well it's doing in Wine. It's search 
function for indexed groups is good but a bit different parameter wise 
from Pan. Pan does searches by subject and author while Newsbin Pro is by 
subject only but has file size and type parameters. Pan does regex 
filtering for blocking the garbage posts but if your using search you're 
not going to see them anyway. Apples and oranges really and it's best to 
make a fruit salad and use both. This local search of an indexed group is 
different from the internet search function which uses Newsbin Pro's own 
indexes which does allow for searches by either subject or author.. 



Combining newsreaders for searching and downloading.

I only had the full Newsbin Pro search for a week but I found that it's 
still useful after the subscription expired and it only gives me the name 
of a file but not the group it is in. I found that the name was enough. I 
just copy and paste the name into Newslazer's search which uses Eweka's 
indexes and I get a full listing with group and I can download it 
directly from Newslazer. If it's in a group I've got indexed, I just do 
search for the file in the group's headers and download it with either 
Pan or Newsbin Pro. 






-- 
PX=Paradox