Re: Some questions about file handling + some other things

Hi Nagy:

In message <20210324005512.1134ba81@Dell>,
Nagy Gabor writes:

> [ stuff omitted for Ralf to address ]
>One more question on Dates. The dates must be submitted in
>html form variables in the standard date format 2021-03-24.14:00,
>right? There are methods for printing dates in alternative format (like
>pretty()), but it seems to me that the date submitted to Roundup must
>be in this format. And I guess if my users want a different date format,
>then I have to do the conversions on my own.

Correct. You can intercept hyperdb.Date.from_raw with a function you
write.  Your function converts the date to standard roundup form, and
calls the original from_raw to do the real work. I have intercepted
Interval.from_raw to treat:

  integer: as the number of minutes
  float: as the number of hours and fractional hours

A value of 90 is an interval of 1:30 (1 hour 30 minutes) and a
value of 1.5 is also an interval of 1:30.

You can see the code run from interfaces.py in the root of the tracker
at:

https://rouilj.dynamic-dns.net/fossil/roundup_sysadmin/file?ci=tip&name=interfaces.py&ln=50-83

>The sorted method of Multilink properties (provided by the cgi engine),
>does not always work as expected. First, I got Python error on
>context.files.sorted('filetype'), when the 'files' list had an item
>with 'filetype' unset.

Can you open a ticket at https://issues.roundup-tracker.org/ on
this. I assume your error is:

  TypeError: '<' not supported between instances of 'NoneType' and
  'str'

The question is where should None sort.

Best not to ask the question.  Add an auditor that either sets a sane
value (e.g. application/octet-stream) or raises Reject and has the
user fix the missing value. (In the web interface adding 'required' to
the html input helps, but still need an auditor to make sure.)

(but keep reading..)

>Second, this method does not seem to respect the data type of the
>'order' property of 'filetype' class. It was set to Integer in my
>schema.py for the 'filetype' class, but sorted() treated it as a string.
>It took a while until I realised that the reason for unexpected sorting
>is "12" < "2", for example. :)

Internally sorted uses this call to sort the data:

  value.sort(key=lambda a:a[property], reverse=reverse)

I think a[property] is always a string. Changing the code to
something like:

  cls = self._db.getclass(self._prop.classname)
  to_raw = cls.property.sort_repr # (or maybe cls[property])
  value.sort(key=lambda a:to_raw(cls, a[property], property),
             reverse=reverse)

Ralf, Bern et. al. does the derivation of to_raw to get the hyperdb
Integer property's sort_repr method look right?

Also it looks like this might not work as the sort_repr for the
hyperdb.py::Integer class just returns "val" rather than int(val).  So
it looks like both the Number and Integer classes need a new

   sort_repr(self, cls, val, name)

method returning respectively float(val) and int(val) (in a try block
to handle undefined values). This issue is definitely worth a bug
report. Nagy can you create a report at: issues.roundup-tracker.org
with the python command you tried and what you saw (basically a
reworded copy of your paragraph above).

>Finally, one more minor question. Assume that I want to sort notclosed
>tasks by 'deadline' in increasing order (which seems reasonable). In
>this case the first tasks will be those for which deadline is unset.
>But what if I want to put them to the end of the list? How this can be
>done? (First try: I won't allow unset deadline, it is 9999-12-31 behind
>the scenes.

You would want to enforce that with an auditor. However I assume you
don't really want that value in the database. You just want it to sort
like that. Right?

>Second try: I do two database query for tasks (the first
>one filters on 'deadline is set', and an other one filters on 'deadline
>is not set', and join these list. But this is just an example, I am more
>interested in how these filter/sort cgi request can be customized, or
>the obtained "batch list" modified?)

The only customization I know of is controlling sort order using the
orderprop and filtering. Maybe somebody else knows how to do it.

Going back to the question of where the null value should sort.

The default_value property is defined for all hyperdb types.  It is
referenced in cgi/templating.py (with a call to get_default_value()).
I don't know exactly where it shows up. The default_value AFAIK
doesn't show up in the db (to replace null). It may only be present in
the html interface or accessible by python code (e.g. an auditor).  I
wonder if default_value could be used somehow to control sorting
order?

Consider a sort_repr for Integer of:

   sort_repr(self, cls, val, name):
      if val:
         return int(val)
      else:
         cls[name].get_default_value()

If you have an Integer, you use:

    workorder=Integer(default_value=9999)

to make any undefined value sort at the end. Changing to 0 (assuming
range is 0-9999)makes it sort at front. Similarly for a string: a
default_value of '' sorts at one end and default_value of
'zzzzzzzzzzzzzzzzzz' sorts at the other end.

The downside of this is that you can't change it. It's hard-coded in
the schema. (Well you can change __default_value but no don't do
that.)

Maybe better to make:

   sorted(self, property, reverse=False, nullval=None)

then change the sort call in sorted() from:

  value.sort(key=lambda a:to_raw(cls, a[property], property),
             reverse=reverse)

to:

  def keyfunc(a):
      # define this inside of sorted()
      # to_raw, cls, property etc. defined as above
      # in sorted()
      v = to_raw(cls, a[property], property)
      if v:
         return v
      elif nullval is not None:
         return nullval
      else:
         return val # and possibly raise an exception if None cant be sorted.

  value.sort(key=keyfunc, reverse=reverse)

where nullval must have the same type as returned by to_raw.
Alternatively nullval could always be a string and we return:

        return = to_raw(cls, nullval, property)

which will probably be easier for the user but takes some extra
computation. Thoughts?

(Note all of this was done with no residual caffeine so may be gibberish.)

--
				-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.