Re: Some questions about file handling + some other things

Hi Nagy:

In message <20210319221008.2249f6a1@Dell>,
Nagy Gabor writes:
>My first question is about the web interface (and in fact, more
>general). My observations after playing with @file and @file-1 etc.
>form variables. Maybe it will be useful to others, and you can correct
>me if I am wrong:
>
>1. Multiple file upload is supported via @file: You can use a
><input type="file" name="@file" multiple> form input,

Correct but this does have the problem that multiple files can only be
chosen from one directory (in windows at least). The file chooser
can't be used to select different files from diffrent directories
AFAICT.

>or you can
>add <input type="file" name="@file"> multiple times. (The second
>solutions seems a bit ugly to me, it may depend on the browser.)
>This is documented in the source code, but not on the website.

Hmm, I didn't realize that @file could be repeated successfully. Can
you tell me where in the code that is documented?

>2. By inspecting, it seems that if you want to use @file-1, @file-2,
>@file-3,... instead of the above method (because you also want to set
>some properties in the creation step), then some "-N" can be omitted.
>For example if @file-1 and @file-3 form variables are set, but @file-2
>is not then it works as expected. This needs some more testing though.
>:) [This is handy, when the file upload page is created using JS,
>and the user is allowed to remove files from the to-be-uploaded list.
>Then you do not have to renumber everything.]

AFAIK this is correct. The -N extension is only used to allow linkage
between fields in a newly created object in the class. There is no
need to have them be consecutive.

>3. You can create files (of FileClass) in the database even with
>roundup-admin, simply just do "create file ..." and setting the content
>property accordingly. (This was not obvious to me, sorry.)

Correct. The one issue with this this is trying to get multiline or
binary data (jpeg) into the file. I think enhancing roundup-admin to
special case content= in FileClass objects to recognize
content=<filename is one way to do it. Another way is to expose the
hidden property "binary_content" that is used internally. So
binary_content=filename always reads from a file.

>4. I do not really understand how indexing works yet, and
>the documentation is a bit terse on that topic. I just vaguely looked
>into the code, but it seems that
>The content of files (file, msg, etc.) are indexed (?), but only if
>their type is text/plain (?). This indexing can (?) be disabled by
>explicitly setting
>
>file = FileClass(db, "file",
>                name=String(),
>                content=String(indexme='no'))
>
>in schema.py. Btw shall I install Xapian, or not? :)

IIRC any String is indexed. So title fields, your memo field above
etc. should all be indexed. It also strips words < 1 character or
words > 25 characters I guess it might be a problem for German 8-).

You can always add an indexer later and run reindex. (See
upgrading.doc for an example). By default it uses native indexing
based on inserting tokenized words into tables in the backend
database. I don't think we use the text search capabilities of either
sqlite (FTS4/5) or postgres (GIN index).

If you need a porter stemmer, xapian is set up to that IIRC. Woosh is
supposed to do "Fuzzy" matching. I don't remember exactly what the rules
are there. The test suite just does the most basic tests of the
indexers. So Porter stemming, fuzzy matching etc aren't tested.

>5. The other thing I was interested in is journaling: As I see, the the
>journaling of String() property changes cannot be disabled. (In my db
>there is a "memo" for issues, which always increasing, but I did not
>want to use a FileClass for this, and the history looks ugly. But the
>history needs rewriting, anyway. ;)

By memo do you mean

   memo = String()

in the index class? If that is what you mean, yeah there is no way to
disable journaling for the string.

IIRC you can disable journaling on a class (enableJournalling and
disableJournalling methods added in the backends). I think that also
disables journaling for all properties of the class. In theory you
could create a Memo object with one string field. Then disable
journaling on the Memo class. A new issue would create a new memo
object and link it to the issue.  Editing the issue would display/edit
the string in the linked Memo. This might work but seems fragile at
best.

However do_journal doesn't quite do what I think you expect. Suppose
is the issue has an owner and owner is defined as:

  owner=Link("user", do_journal="no")

now you assign user1 (admin) as the owner for issue1 replacing user3
(demo). Issue1 will have in it's history log:

   owner: demo -> admin

if do_journal was not set to 'no', the history for user1 would
have

  link issue1 owner

and the history for user3 would have

  unlink issue1 owner

this is the journaling that do_journal suppresses. It has no effect on
the journaling for the link/multilink field in the issue. Which I
think is what you want for the String property.

The customizing.txt document for do_journal makes perfect sense after
you read it oh 10 or so times. (Hence the reason for wanting to
split/enhance customizing.txt.)

That being said, changing the code to add a no_logging attribute to all
fields that does what you want do_journal to do for srings could be
done.

IIRC the history can be deleted (pack subcommand for
roundup-admin). So it may be possible to delete the newly added
history for the string property from a reactor.

>6. Logical negation (or exclude_spec) is missing from filtering
>machinery. :) Stuff like "status_notresolved string:-1,1,2,3,4,5,6,7;"
>looks a bit strange in the page.html of the default templates... :)

I was going to say there is a way to do it, but it's not
documented. However (I just discovered) it doesn't work for links,
only multilinks. So nevermind.

>I hope I can deeply understand the code at some day, and I can commit
>patches at some point. :)

That would be helpful.

Have a great weekend.
--
				-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.