Re: Some questions about file handling + some other things

Dear John,

Thank you for your quick reply.

> Hi Nagy:
> 
> In message <20210319221008.2249f6a1@Dell>,
> Nagy Gabor writes:
> >My first question is about the web interface (and in fact, more
> >general). My observations after playing with @file and @file-1 etc.
> >form variables. Maybe it will be useful to others, and you can
> >correct me if I am wrong:
> >
> >1. Multiple file upload is supported via @file: You can use a
> ><input type="file" name="@file" multiple> form input,  
> 
> Correct but this does have the problem that multiple files can only be
> chosen from one directory (in windows at least). The file chooser
> can't be used to select different files from diffrent directories
> AFAICT.

Yes. That is why I finally ended up with JavaScript (for handling
multiple file upload). :)

> >or you can
> >add <input type="file" name="@file"> multiple times. (The second
> >solutions seems a bit ugly to me, it may depend on the browser.)
> >This is documented in the source code, but not on the website.  
> 
> Hmm, I didn't realize that @file could be repeated successfully. Can
> you tell me where in the code that is documented?

Oh, here I just meant this for the scenario that multiple files can
be uploaded using @file, and I referred to the following comment in
cgi/form_parser.py (which is ):
""" 
  If "multiple" is turned on for file uploads in the html
  template, multiple links are generated::

      @link@files=file-2
      file-2@content=value
      ...

  depending on how many files the user has attached.
""" 

> >2. By inspecting, it seems that if you want to use @file-1, @file-2,
> >@file-3,... instead of the above method (because you also want to set
> >some properties in the creation step), then some "-N" can be omitted.
> >For example if @file-1 and @file-3 form variables are set, but
> >@file-2 is not then it works as expected. This needs some more
> >testing though. :) [This is handy, when the file upload page is
> >created using JS, and the user is allowed to remove files from the
> >to-be-uploaded list. Then you do not have to renumber everything.]  
> 
> AFAIK this is correct. The -N extension is only used to allow linkage
> between fields in a newly created object in the class. There is no
> need to have them be consecutive.

OK. Thank you for the confirmation, this agrees with my observations.
 
> >3. You can create files (of FileClass) in the database even with
> >roundup-admin, simply just do "create file ..." and setting the
> >content property accordingly. (This was not obvious to me, sorry.)  
> 
> Correct. The one issue with this this is trying to get multiline or
> binary data (jpeg) into the file. I think enhancing roundup-admin to
> special case content= in FileClass objects to recognize
> content=<filename is one way to do it. Another way is to expose the
> hidden property "binary_content" that is used internally. So
> binary_content=filename always reads from a file.

Thank you. (I will also use the non-web interfaces for providing
anonymous users file uploading to the tracker ("initial commit"), but
for security reasons and sensitive data, I do not want to allow them to
use the web interface, even through ReverseProxy::FormFiller.)

> >4. I do not really understand how indexing works yet, and
> >the documentation is a bit terse on that topic. I just vaguely looked
> >into the code, but it seems that
> >The content of files (file, msg, etc.) are indexed (?), but only if
> >their type is text/plain (?). This indexing can (?) be disabled by
> >explicitly setting
> >
> >file = FileClass(db, "file",
> >                name=String(),
> >                content=String(indexme='no'))
> >
> >in schema.py. Btw shall I install Xapian, or not? :)  
> 
> IIRC any String is indexed. So title fields, your memo field above
> etc. should all be indexed. It also strips words < 1 character or
> words > 25 characters I guess it might be a problem for German 8-).
>
> You can always add an indexer later and run reindex. (See
> upgrading.doc for an example). By default it uses native indexing
> based on inserting tokenized words into tables in the backend
> database. I don't think we use the text search capabilities of either
> sqlite (FTS4/5) or postgres (GIN index).
> 
> If you need a porter stemmer, xapian is set up to that IIRC. Woosh is
> supposed to do "Fuzzy" matching. I don't remember exactly what the
> rules are there. The test suite just does the most basic tests of the
> indexers. So Porter stemming, fuzzy matching etc aren't tested.

Thank you.
 
> >5. The other thing I was interested in is journaling: As I see, the
> >the journaling of String() property changes cannot be disabled. (In
> >my db there is a "memo" for issues, which always increasing, but I
> >did not want to use a FileClass for this, and the history looks
> >ugly. But the history needs rewriting, anyway. ;)  
> 
> By memo do you mean
> 
>    memo = String()
> 
> in the index class? If that is what you mean, yeah there is no way to
> disable journaling for the string.

Yes, I meant that. It is not a big problem, I will implement my own
history module as extension, and I will filter out these changes.

> IIRC you can disable journaling on a class (enableJournalling and
> disableJournalling methods added in the backends). I think that also
> disables journaling for all properties of the class. In theory you
> could create a Memo object with one string field. Then disable
> journaling on the Memo class. A new issue would create a new memo
> object and link it to the issue.  Editing the issue would display/edit
> the string in the linked Memo. This might work but seems fragile at
> best.

Thanks, good to know, I will consider this, too.

> However do_journal doesn't quite do what I think you expect. Suppose
> is the issue has an owner and owner is defined as:
> 
>   owner=Link("user", do_journal="no")
> 
> now you assign user1 (admin) as the owner for issue1 replacing user3
> (demo). Issue1 will have in it's history log:
> 
>    owner: demo -> admin
> 
> if do_journal was not set to 'no', the history for user1 would
> have
> 
>   link issue1 owner
> 
> and the history for user3 would have
> 
>   unlink issue1 owner
> 
> this is the journaling that do_journal suppresses. It has no effect on
> the journaling for the link/multilink field in the issue. Which I
> think is what you want for the String property.

Yes, you are I right, I thought do_journal="no" will not journal
anything (in any class) about the modification of the owner Link.
 
> The customizing.txt document for do_journal makes perfect sense after
> you read it oh 10 or so times. (Hence the reason for wanting to
> split/enhance customizing.txt.)

:)
 
> That being said, changing the code to add a no_logging attribute to
> all fields that does what you want do_journal to do for srings could
> be done.
> 
> IIRC the history can be deleted (pack subcommand for
> roundup-admin). So it may be possible to delete the newly added
> history for the string property from a reactor.

Good to know, I will check this, too. Thank you for this information.

> >6. Logical negation (or exclude_spec) is missing from filtering
> >machinery. :) Stuff like "status_notresolved
> >string:-1,1,2,3,4,5,6,7;" looks a bit strange in the page.html of
> >the default templates... :)  
> 
> I was going to say there is a way to do it, but it's not
> documented. However (I just discovered) it doesn't work for links,
> only multilinks. So nevermind.
> 
> >I hope I can deeply understand the code at some day, and I can commit
> >patches at some point. :)  
> 
> That would be helpful.
> 
> Have a great weekend.

Regards,
Gábor Nagy

> --
> 				-- rouilj
> John Rouillard
>
> ===========================================================================
> My employers don't acknowledge my existence much less my opinions.
> 
> 
> _______________________________________________
> Roundup-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/roundup-users


_______________________________________________
Roundup-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/roundup-users