Re: REST API

Hi Everybody:

My original concept/use case for the REST interface was to use it
to drive a replacement front end for roundup. The front end could be a
javascript single page application written in react, angular, vue
etc. It could be a hybrid of back end HTML and front end javascript
where all the data comes from the REST endpoint.

In this case, some of the functionality we currently build into the
html templates needs to be exposed via the rest interface. I will
return to this thought below. In general I will probably bring up more
questions than answers 8-). Sorry for the length of this email.

However to answer a few questions...

In message <[email protected]>,
Ralf Schlatterbeck writes:
>I'm now taking the private discussion I had with John to the list so
>that more people are informed about what is done.
>
>As far as I can tell what is still open is
>- A version Identifier after /rest, e.g. /rest/.../v1 or /rest/.../1 for
>  the initial release. Your latest commit is ambiguos, it says that
>  /rest returns supported version but then goes on with /rest/data --
>  which should then be /rest/.../0/data?

The metadata returned by /rest:

  {
    "data": {
      "links": [
	{
	  "rel": "summary",
	  "uri": "https://example.com/demo/rest/summary"
	},
	{
	  "rel": "self",
	  "uri": "https://example.com/demo/rest"
	},
	{
	  "rel": "data",
	  "uri": "https://example.com/demo/rest/data"
	}
      ],
      "supported_versions": [
	"1"
      ],
      "default_version": "1"
    }
  }

is meant to define possible versions and the version used if no
specific version is requested. How the client specifies the version is
still an open question.

>  Or should we make the initial version v0?
>  Should we have some prefix before the version (indicated by ... above)
>  This could be done in class Routing, which would become more
>  complicated when we add additional versions.

I originally proposed putting the version specifier in the URL,
e.g. /v1/. However the URL in REST is supposed to be a resource, the
url's

   http://foo/tracker/rest/.../v1/issue/20

and

   http://foo/tracker/rest/.../v2/issue/20

are still the same resource. Consider the roundup web
interface. .../tracker/issue14 is a constant. You can make it look
different by adding ?@template to the constant url.  So I really don't
like the /v1/, /v2/ concept, it's just the first implementation I saw
and has the advantage of being browser friendly.


I think implementing a query parameter to control the version is
similar to @template. So .../rest/data/issue/1?@version=1 is
better. It may also be cleaner to implement when we have multiple
versions.

The "correcter" way would be to set the version in the Accept header
in some form. However controlling the HTTP headers from a browser is
difficult.  By supporting ?@version=1 somebody investigating the rest
interface in a browser could enter:

  rest/data/issue/21?@version=1
  rest/data/issue/21?@version=2

and see the different output. (This is the same reason for supporting
.json or .xml at the end of the url to control the returned data format.)

I think version 0 is fine. The question is what form should the
version take? Should it be 0.0, or just 0. In X.y versioning would
mean that any client that understands version X.0 would be able to
understand version X.1 or newer. Compare to HTTP 1.0 vs 1.1. A version
0.3 client could understand a version 0.99 response but might not
understand a 1.0 response.

This would let us refine the response to add fields that can be
ignored by older clients. With the x.y format, the y allows us to
identify the exact format for bug reports etc.

We could use just plain version numbers 0, 1, 2 etc. However a version
0 client wouldn't know if it could understand a version 7 response. It
would have to assume that it can't. We could make each number a
breaking change, so that version 1 could not be understood by version
0 clients. But we would have no way to track minor changes in bug
reports. So there may be 3 or 4 different outputs all labeled with
version 0.

>- Hard-coded /data: Are you working on this? If not I think I know the
>  Roundup-API well enough to fix this

I am not sure what you mean by this. Do you mean the result of accessing:

  /rest/data/

If so, I am not working on it. Returning the list of URL's and objects
that can be accessed via /rest/data/ would be good.

>- Hard-coded class in /summary: This should also be fixed, I have quite
>  some trackers that simply don't have an 'issue' class.
>  The code there (and the permission checks) should probably be a lot
>  more dynamic, we could leave out the status if no permission etc.

Agreed.

This brings up another point, how to customize the data returned based
on the schema.

Given my original concept with REST driving the UI, let's look at how
we handle changing the status of an issue. A GET on rest/data/issue/21
can return (using pseudojson):

{ data: { @etag: "aaaaa",
          attributes: { 
	    title: "title here",
	    nosy: [ "1", "4" ],
	    status: "1",
	    assignedto: "4",	    
	    ...
	  }
          id: "21",
	  link: "https:..../rest/data/issue/21",
	  type: "issue",
  	  ...
  }
}

Now suppose I want to change status. Also let's assume I have a
workflow set up. Using the HTML interface, I would have an option
list like:

  New (1)
  Open (2)
  [Stalled (3)]
  [Closed (4)]
  Rejected (5)

The []'s indicate that the option is disabled and unselectable. So
from the current state of "New", the only change would be to value 2
state Open, or value 5 state Rejected. Stalled and Closed are right out.

How do we represent the same in REST?

One way might be to implement an @allowed_changes property. So a GET on
rest/data/issue/21/status/@allowed_changes would return:

{ 
  data: {
     @etag: "aaaaa",
     @links: {
     	new: { id: "1",
	       label: "new",
	       rel: "self"
	       uri: "rest/data/issue/21/status"
	}  
        open: { id: "2",
	        label: "open",
		uri: rest/data/issue/21/status
 	}
	rejected: { id: "5",
		    label: "rejected",
		    uri: rest/data/issue/21/status
	}		
     }
   }
}   

this would tell the client that the user is allowed to open or retire
the issue, but nothing else. To change to open, PUTting the id (2) to
the uri will make the change. (I assume the label could be localized
into a different language, but the keys of the @links dictionary would
not be translated.)

I think the rest interface can be augmented from the tracker. This
would be similar to the way that new actions can be defined in the web
interface. So it should be possible to construct a handler for new
@properties that are unique per tracker and can return the proper
elements (id, label, uri etc.) to allow the client code to do
something useful.

In a similar way

  http://roundup.sourceforge.net/docs/customizing.html#restricting-the-list-of-users-that-are-assignable-to-a-task

could be implemented to get back a list of users who are allowed to be
the assignedto user.

As another use case, let's look at displaying the activity, assignedto
user, and title for all open issues.

A GET on: rest/data/issue?status=open (note the current syntax is
different and uses ?where_status=open) would return an array like:

{ data: [
    { id: 1, link: rest/data/issue/1 }
    { id: 2, link: rest/data/issue/2 }
    { id: 3, link: rest/data/issue/3 }
  ]
}

Now I need to do a GET on each link/uri to get the title and activity
to display to the user. So I could do a GET on

  rest/data/issue/1?@fields=title,activity,assignedto

to get those three pieces of info (note the current interface uses
?fields=...). I can use the strings returned for title and activity
directly. However the value of assignedto is:

  "4"

Now I need to guess how to use this string to get a link to the user
represented by "4". Magically, I do a GET on
rest/data/user/4?fields=username to see:

   { data: { attributes: { username: fred },
             id: "4",
	     link: rest/data/user/4,
	     type: "user",
	     @etag: ...
	     }

So now I have the display name for the user.  Assuming I have N
issues, this means I need: 2N+1 trips to the rest endpoint to display:
title, activity and owner to the interface user.

We have two issues here:

  1) lots of round trips and the latency delay
  2) the need to figure out a magic url for the assignedto user

Solving 2 would mean that the response for an issue needs to look more like:

{ data: { @etag: "aaaaa",
          attributes: { 
	    title: "title here",
	    nosy: [ {id: "1", link: rest/data/user/1},
	             {id: "4", link: rest/data/user/4} ],
	    status: "1",
	    assignedto: { id: "4", link: rest/data/user/4 }
	    ...
	  }
          id: "21",
	  link: "https:..../rest/data/issue/21"
	  type: "issue"
  	  ...
  }
}

to provide links for every reference.

This removes the need for magic to figure out how to get additional
data.  Also it is cheap to generate since no additional database
lookups are needed to provide the link. It also implements HATEOAS and
decouples the client from the implementation. The nosy property could
be a multilink to a "nosy_users" class. The client would just blindly
follow the link and still work. However, it does bloat the response.

Solving problem 1 would mean embedding fields and objects inside the
response. Theoretically I could do a GET on:

  rest/data/issue?status=open&@attributes=title,activity,assignedto&@embed=assignedto

which returns:

 { data: [
    { id: 1, attributes: {title: "my title", activity: "2018-05-18.01:54:17",
         assignedto: { id: "4", link: rest/data/user/4, label: "fred" },
      link: rest/data/issue/1 }
    { id: 2, attributes: {title: "A title", activity: "2018-07-18.01:54:17",
         assignedto: { id: "5", link: rest/data/user/5, label: "bob" },
      link: rest/data/issue/2 }
    { id: 3, attributes: {title: "B title", activity: "2018-03-18.01:54:17",
         assignedto: { id: "6", link: rest/data/user/6, label: "jim" },
      link: rest/data/issue/3 }
    ]
 }

This returns all the data I need in one transaction. (The value of
label is the labelprop for the class.) But there is a cost to
this. Getting the related data (e.g. label info for users) results in
a number of additional database accesses and is a concern for larger
databases. But is this a good way to do this?

>- Docs :-)

Yup.

So what other use cases and issues should we consider for the rest
interface?

As I said above I have more questions here, but I hope this has given
people something to think about for the rest interface.

While this is a development discussion, does anybody think the use case
discussion would be useful on the users list?

--
				-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.