Re: Some questions about file handling + some other things

On Tue, Mar 23, 2021 at 11:13:20PM -0400, John P. Rouillard wrote:
> Internally sorted uses this call to sort the data:
> 
>   value.sort(key=lambda a:a[property], reverse=reverse)
> 
> I think a[property] is always a string. Changing the code to
> something like:
> 
>   cls = self._db.getclass(self._prop.classname)
>   to_raw = cls.property.sort_repr # (or maybe cls[property])
>   value.sort(key=lambda a:to_raw(cls, a[property], property),
>              reverse=reverse)
> 
> Ralf, Bern et. al. does the derivation of to_raw to get the hyperdb
> Integer property's sort_repr method look right?

Looks good (without having tested anything) but I'm not sure to_raw is
the correct method to use here in sorting.

> Also it looks like this might not work as the sort_repr for the
> hyperdb.py::Integer class just returns "val" rather than int(val).  So
> it looks like both the Number and Integer classes need a new
> 
>    sort_repr(self, cls, val, name)
> 
> method returning respectively float(val) and int(val) (in a try block
> to handle undefined values). This issue is definitely worth a bug
> report. Nagy can you create a report at: issues.roundup-tracker.org
> with the python command you tried and what you saw (basically a
> reworded copy of your paragraph above).

It is also worth a regression test: If sorting of numbers and/or dates
works in SQL (for everything but Multilink properties) I'm quite sure
that it would break for anydbm which always sorts in python.

> >Second try: I do two database query for tasks (the first
> >one filters on 'deadline is set', and an other one filters on 'deadline
> >is not set', and join these list. But this is just an example, I am more
> >interested in how these filter/sort cgi request can be customized, or
> >the obtained "batch list" modified?)
> 
> The only customization I know of is controlling sort order using the
> orderprop and filtering. Maybe somebody else knows how to do it.

The sorting regarding the handling of NULL values (aka None in Python)
is currently fixed and cannot be influenced via the API.

> Going back to the question of where the null value should sort.
> 
> The default_value property is defined for all hyperdb types.  It is
> referenced in cgi/templating.py (with a call to get_default_value()).
> I don't know exactly where it shows up. The default_value AFAIK
> doesn't show up in the db (to replace null). It may only be present in
> the html interface or accessible by python code (e.g. an auditor).  I
> wonder if default_value could be used somehow to control sorting
> order?

Hmm, using default_value would be too surprising when it influences the
sort order. I'd put this into the declaration of the property (similar
to 'do_journal' for Link/Multilink properties. But this would mean it
cannot be changed by the user in the index template input. A name I'd
suggest would be 'sort_null_last' which by default is 'no'.

> to make any undefined value sort at the end. Changing to 0 (assuming
> range is 0-9999)makes it sort at front. Similarly for a string: a
> default_value of '' sorts at one end and default_value of
> 'zzzzzzzzzzzzzzzzzz' sorts at the other end.

.. until 'zzzzzzzzzzzzzzzzzzz' comes along .-)


> The downside of this is that you can't change it. It's hard-coded in
> the schema. (Well you can change __default_value but no don't do
> that.)
> 
> Maybe better to make:
> 
>    sorted(self, property, reverse=False, nullval=None)

Hmm, I would make this a boolean property (NULL sorted either in front
or to the back), this would also make it work for all values, you cannot
find a "nullval" value in the example above that works across all data.
And I do not see a use-case where you want NULL-values to be sorted in
the middle somewhere.

If we really need this to be specified by the user, which I doubt (note
that the filter method already has a lot of parameters that are
currently not exposed to html, I'm reluctant at this point to add more).

Currentl sort/group parameters to filter are specified as lists of
tuples of the sort direction and the property name (I always forget
which one comes first, just looked it up, the direction is first). We
*could* add another character to the direction, e.g. '~' to specify the
handling of NULL values. Something like 

db.someclass.filter (None, filterspec, sort = [('+~', propname)]
But this is ugly :-)
Or an optional third parameter sort = [('+', propname, 'null-last')]

Ralf
-- 
Dr. Ralf Schlatterbeck                  Tel:   +43/2243/26465-16
Open Source Consulting                  www:   www.runtux.com
Reichergasse 131, A-3411 Weidling       email: [email protected]