Re: The future of the file-sharing service

On 9/19/22 1:42 PM, madmurphy wrote:
> Hi all,
> 
> On Thu, Sep 15, 2022 at 2:55 PM Maxime Devos <[email protected] 
> <mailto:[email protected]>> wrote:
> 
>     Myself, I'm very interested in the file-sharing service and DHT
>     service -- I would like to implement (Guix) substitutes over GNUnet
>     (I already sent a POC implementation previously, but it wasn't a
>     very nice implementation, e.g. there was no fallback to regular http
>     / https, that's why I started scheme-gnunet, to make GNUnet easier
>     to use from Guile.
> 
> I know that probably the FS service is not the priority at the moment, 
> but I was thinking that nothing forbids to imagine now how an ideal FS 
> service could look like. Time ago I had written down some of the 
> features that I believe a future FS module will need to implement. I 
> paste my list here, with the hope of leaving food for thought for a 
> brain storming.
> 
> What FS needs to have:
> 
>  1. Possibility of sharing a file while it is still being downloaded
>     (parts of it, of course)

This is already possible, the GAP routing layer can decide to cache any 
block of data it receives in the datastore.

>  2. Metadata must be editable and sharable

You can always publish new metadata under a keyword. What does sharable 
mean here?

>  3. Search keywords must be visible, editable, sharable (part of the
>     metadata?)

Metadata (along with the URI of the file) is published under keywords, a 
keyword search returns this information. Are you suggesting including 
all the known keywords as part of the metadata?

>  4. Introduction of a rating mechanism for files (against spam)
>  5. Allow reverse search (i.e. chk-URI lookup)

This could be done by using the URI as a keyword when publishing.

>  6. Automatically and fully auto-unindex a file when it is missing
>  7. Autoshare the dynamic content of a directory and update its index in
>     real time (e.g. if I “autoshare” the content of
>     |/srv/filesharing/gnunet/madmurphy/|, when I add |foobar.txt| to
>     that directory it must be automatically indexed – the opposite if I
>     remove it)

This is what gnunet-auto-share does.

>  8. Implement file statistics (download counter? last seen? etc.) – this
>     should allow the network to get rid easily of “lost” content

The gtk gui tries to download parts of search results to display how 
available a file is.

>  9. Implement a NOT operator for search keywords (a tilde, “~”?)
> 10. Implement an OR operator (a vertical bar, “|”? currently not writing
>     any operator equals OR, but we need an explicit OR operator if we
>     want to implement the next point)
> 11. Allow parenthesis parsing for AND/OR/NOT operators; this will
>     require that operators can be followed by spaces (e.g.
>     |gnunet-search +required '+' '(' optional1 '|' optional2 ')'|)
> 12. The output filename must become optional in |gnunet-download|. If
>     not specified, the file/directory must be downloaded using the
>     original filename it was published with – if currently this
>     information is not always saved during publishing, then we must make
>     sure that it is always saved (this is part of the metadata too and
>     must be editable)
> 
> My two cents. Please do not hesitate to post your comments on these 
> proposals or add new ideas.
> 
> --madmurphy
>