Short note for the season

Hello all:

Things have been busy in the Roundup workshop (not located at the
north pole). In the past few weeks, the following notable
improvements have been made:

  * adding a filter() to permissions to dramatically speed up
    index/search operations.

  * serve unencoded content data for files/messages using the REST
    interface.

  * reject a request if the client requests a response in an
    unsupported format without changing the database.

  * the format for RPN property expressions has been documented

These will all be in the 2.5 release.

Details follow. If you have questions about these, feel free to follow
up on the mailing list.

  * Ralf added a feature to verify permissions using the
    database. This significantly speeds up the display of index pages
    and search results.

    For example, you have a 'company' property associated with both a
    user and an issue. You define a permission that allows a user to
    only see issues from their company.

    This can be implemented by defining a check() for the permission.
    The check() has to retrieve each item from the database and then
    check the item's properties. This retrieves a lot of data for
    items that will not meet the permission criteria.

    The new filter() for permissions filters the ids returned by a
    search using the database. The output of the filter() method is
    turned into a SQL where clause and applied to the list of ids by
    the database server. The return from the filter() is a shorter
    list of ids. Continuing the example above, filter() would return a
    value that creates a "WHERE company is <the user's company>"
    clause. This can dramatically reduce the number of ids that
    check() has to retrieve and test.

    In SQLite, this isn't as big of a problem because data retrieval
    is quick, but filtering in C instead of Python is still
    beneficial. For MySQL/Maria or PostgreSQL, this process is much
    faster.  For the dbm back end, the filtering is still done in
    Python.

  * Getting the content of a file or message through the REST
    interface involved pulling the encoded content from a JSON
    response. While this method worked, it wasn't great because the
    data size could increase by four times. The data could also be
    accessed without encoding by using the same endpoint as the HTML
    interface. This method worked too, but it wasn't very tidy.

    I added code that lets you get the binary content of an item as a
    raw data stream from the REST interface. By adjusting the 'Accept'
    header to the right mime type for the data (or using the wildcard
    '*/*'), the REST client will receive the raw data directly from
    the '/binary_content' endpoint. There's no need for the server to
    encode and the client to decode the data. This is also more
    efficient than using the HTML endpoint.

  * I fixed a long standing bug in the REST interface. Roundup would
    give a '406' error if the REST client asked for an 'Accept' value
    that Roundup couldn't provide. For example, a request for
    'text/plain' when the endpoint returns only json or xml.

    However the '406' error was produced after the operation had
    completed. If you were only reading data, it wasn't a problem. If
    you were using POST to create an item, PUT to update it, or PATCH
    to change it, the modification would happen despite the '406'
    error. This left the client unaware that the request was
    successfully completed.

    The REST interface now checks the Accept values and returns an
    error before starting any tasks. This means Roundup quickly fails
    and won't make a change when it can't reply properly to the
    client.

Hope you all have a happy holiday season.

--
				-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.