Re: Modifying "msg" and "file" files

Dear John,

Thank you for your quick and detailed reply. Everything is clear now.

I need to respond to this question only:

> Yeah the docs need some restructuring. Where did you expect to find
> that info?

I did a "ctrl-f search" in the entire documentation, but I expected to
find the answer somewhere around here:

FileClass subsection of
https://roundup-tracker.org/docs/customizing.html#classes-and-properties-creating-a-new-information-store

or at

https://roundup-tracker.org/docs/design.html#messages
https://roundup-tracker.org/docs/design.html#files

Sincerely,
Gábor Nagy

> Hello Nagy:
> 
> In message <20201125204545.74bedbe3@Dell>,
> Nagy Gabor writes:
> >I am new to roundup, and it seems very impressive;  
> 
> Welcome to roundup.
> 
> >I really like its
> >customizability. I bring up the same question as I asked on IRC today
> >(sorry for the duplication, I think this is the better place for the
> >discussion).  
> 
> Yup. No problem on the duplication, but yes this is a better place for
> the discussion.
> 
> >My question is that if I modify an existing "msg1" or "file1" file in
> >the db directory of the tracker, then the Roundup database will
> >remain consistent? (Or does the database rely on some properties of
> >the created file, like size or md5sum etc.?)  
> 
> By default, there is no validation of the file contents stored in the
> db. The mime type used for the file is stored in the db. You might
> want to change (e.g. to text/plain from application/octet-stream or
> image/jpg) so people can view the edited file rather than being forced
> to download it.
> 
> You can edit the file any way you like. If you delete the file,
> retrieving it will cause an error. Emptying the file is also confusing
> to the user. I recommend something like:
> 
>   echo "File deleted due to spam" > db/files/msg/0/msg344
> 
> so the user who tries to download the file will not be presented with
> an empty download and think something has gone wrong.
> 
> >I want to do this, because sometimes it would be nice to edit
> >messages or wipe out large files (e.g. email attachments) on the disk
> >by replacing them to an empty files (which does not break the
> >"retirement" design of roundup).  
> 
> Yup I have edited messages to remove passwords 8-(.
> 
> Note retiring the file only removes it from indexes. You can still
> download it if you specify the url e.g. tracker/msg334. Retiring the
> item doesn't cause a 404 error or prevent accessing the info.
> 
> >I did not find the answer to this question in the documentation.  
> 
> Yeah the docs need some restructuring. Where did you expect to find
> that info?
> 
> >By inspection, it *seems* that modifying these files does not break
> >anything, and I will give a deeper look to the relevant parts of the
> >code. But I would be secure, if any of the main developers could
> >confirm my "observation" / "conjecture", because probably I would
> >never understand the whole source code like them.   
> 
> You can make these changes. It's not explicitly permitted by roundup
> (i.e. roundup doesn't provide an edit mechanism for messages by
> default). So while roundup expects the messages to be immutable, it
> doesn't require/enforce that. Also file content changes are not
> journaled.
> 
> You can use file mixins as documented at:
> 
>   https://wiki.roundup-tracker.org/MixinClassFileClass
> 
> to do various things with the stored files. The two examples there
> de-duplicate attached files and perform transparent file
> compression.
> 
> You could add a content_md5sum field to the schema. Then write a mixin
> that stores an md5 sum of the file contents in the database. The mixin
> could validate the value on retrieval and report a problem if the
> validation fails. The mixin could also add another journal entry
> reporting that content changed. The journal entries would be shown
> providing some indication that the content was changed.
> 
> >And if the modification of these files are (unofficially) supported,
> >then what is your suggested method for doing that (I mean, using as
> >much Roundup API as possible).  
> 
> You can do it via rest (assuming you are using round 2.0). For
> example:
> 
>    $ curl -u demo:demo -s -H "X-requested-with: rest" \
>                           -H "Referer:
> https://rouilj.example.com/demo/" \ -X GET \
>              https://rouilj.example.com/demo/rest/data/file/30/content
>   {
>       "data": {
>           "id": "30",
>           "type": "<class 'str'>",
>           "link":
> "https://rouilj.dynamic-dns.net/demo/rest/data/file/30/content",
> "data": "hello3", "@etag": "\"3f2f8063dbce5b6bd43567e6f4f3c671\""
>       }
>   }
> 
> Save the etag and use it to overwrite the content:
> 
>    $ curl -u demo:demo -s -H "X-requested-with: rest" \
>                           -H "Referer:
> https://rouilj.example.com/demo/" \ -H 'If-Match:
> "3f2f8063dbce5b6bd43567e6f4f3c671"' \ -X PUT \
>                           -F "data=@hello" \
>                 https://rouilj.dynamic-dns.net/demo/rest/data/file/30/content
> 
> where hello is a file on local disk.
> 
> Also since roundup usually considers the file immutable, you can just
> edit the file on disk as well. (You can create an auditor to prevent
> updating the content property to enforce immutability.)
> 
> Hopefully this helps.
> 
> --
> 				-- rouilj
> John Rouillard
> ===========================================================================
> My employers don't acknowledge my existence much less my opinions.


_______________________________________________
Roundup-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/roundup-users