Re: [Tiki-devel] Tiki API discussion

> On Nov 17, 2021, at 7:33 AM, Victor Emanouilov via TikiWiki-devel <[email protected]> wrote:
> 
> Dear dev community,
> 
> We are adding an API to Tiki 24. Your input will be highly valuable.
> 
> As discussed previously with Jonny, Marc and others, first step was exploring the opportunity of exposing our ajax-related services as an API. This makes sense but also has some drawbacks discussed below. Main benefits:
> - extremely quick start - we have many services accessible via ajax for internal Tiki operations that can be exposed as an API for external systems to retrieve data and manipulate data
> - having one and the same code used by internal components and external services will ensure interoperability in the future and same behavior inside and outside Tiki
> - avoid designing and especially coding a whole new level of services for the API
> - opportunity to enhance existing ajax services with missing pieces like permission enforcement and documentation
> Main drawbacks:
> - lack the possibility of designing an API around an idea (resources/restful or graphql)
> - lack versioning support - changing an ajax service in Tiki automatically changes the API which possibly breaks existing integrations (though we can try to be as much backward-compatible as possible and also provide a list of breaking changes when upgrading Tiki)
> - existing services are not restful, neither graph-based, so the resulting API is non-standard
> - some of the existing services are not designed to be exposed via API - e.g. Cypht ajax controllers
> 
> Having these points in mind, I started a MR with the quick-win method of exposing Tiki ajax services here: https://gitlab.com/tikiwiki/tiki/-/merge_requests/1028
> 
> Solved some of the problems with authentication (currently bypassing all standard Tiki auth methods and allowing bearer token authentication only for now), CSRF (turning it off for API but also disabling session cookies, so CSRF are not possible via the API), ensure requests are stateless and no JS or other funky stuff is going on. Now we have the remaining TODO items like documenting the services, enforcing permissions, consider versioning and restful resources. I consider 3 possible ways to go forward but needed your input before investing more time:
> 
> 1. Leave the API urls like now 100% based on the ajax services. Typical URL is tiki.org/api/controller/action?param1=val1&param2=val2, e.g. tiki.org/api/tracker/view?id=12. Enhance services with permissions (where missing), document endpoints, input parameters and output. Versioning seems impossible in this case. At least the standard versioning we are used to see. Maybe we can do versioning based on Tiki version - API version 24 is one and the same for all Tiki 24 releases, then we have version 25, etc...
> 
> 2. Add an adapter between ajax services and API. Make adapter versionable (so we have API versions - any time an ajax service is changed, we keep old behavior and add a new adapter for the new version). Adapter will also help us come up with more standard restful API endpoints and input/output. It can skip certain ajax services that doesn't make sense to be exposed. Permissions should still be in the services themselves as currently, it is possible to get any object attribute without even logging in Tiki with a URL like this https://tiki.org/tiki-ajax_services.php?controller=attribute&action=get&type=trackeritem&object=2856&attribute=tiki.geo.lat. Documentation will be based on the newly written adapter rather than existing ajax services. Will take more time but solve most of the drawbacks. This is my preferred option.
> 
> 3. Design, write and document an API from scratch. We can make use of existing ajax services in terms of code but don't depend on their controller/action architecture - here we have the utmost flexibility to design a graphql or restful API but also the most time-consuming. Without a specific use-case, a project or a subset of Tiki-stored data and functionality to expose, I think it will be a waste of time for now. Also, one more bit here that's pretty important is the fact that Tiki as a web application is stateful in so many ways - it has extensive use of the PHP session that is not available in an API. Adding a proper stateless API requires refactoring all those interal code elements to depend on incoming params or database rather than the session which is a daunting task to even think about. Tiki will certainly benefit from this refactor but at a price of a huge time commitment.
> 
> I also have a question about API documentation. We have a bunch of options here with most prominent ones being Swagger (OpenAPI), Raml and API blueprint - all of which require a separate set of documentation sources (in yaml, json or markdown format). We will need to go through existing services and document endpoints, required params, input, output, etc. Definitely a big task to do. I think there is also an option to use inline documentation - PHP class and method annotation documentation that can be exported to a visually appealing html doc. Not as robust as the first set of tools I mentioned but having the docs and the code in one place has its benefits (easier updates for example). Anyone has any preferences here?
> 
A couple of thoughts.

I really like the idea of designing an API from scratch. Of course, it comes at a huge cost. Could the existing AJAX services API calls be integrated into an API from scratch, like AJAX calls? Or something of that manner. Perhaps it would be possible to have migration warnings upon upgrade if AJAX calls being made have been moved into the API from scratch.

We do, like in the case of plugin attributes, mark when something was added within tiki with a version number. If maybe something similar could be done with AJAX calls so any time a call is updated the tiki version number is bumped up to its current number. That way one would be able to tell what Ajax services need to be updated and what will not, as it could just refuse the call if the version number is incompatible.

I know I have always relied on the session for my own coding. Could we elaborate a little bit (maybe not here but in a doc somewhere) to help existing devs to change their coding practices to stateless within TIki? This way we can at least move forward in a way that won’t be taking one step back. Could it be possible to design some stateless check that is performed to remind devs to keep it stateless?

As per the documentation. I have spent a good amount of time trying to bring better documentation to Tiki. The preference was one project. So now all the preference documentation online is generated from our source code. It was a bigger project than I expected because much of the project was merging the documentation that we had in our source code with the documentation we had online. The end result was MUCH better documentation online, MUCH better documentation within Tiki AND (after the initial time investment) MUCH less ongoing work. Documentation is something that is difficult to get people to actually do. The previous method of documentation in 2 paces meant that half the documentation was online, half was in the code and volunteers endlessly tried to constantly keep them both in sync, to little avail.

That plugin that generates the online documentation I would like to at some point refractor & enhance in a number of ways, but it's still currently miles better than what came before. The drawbacks to a dual documentation system are enormous. PHP 8 now also has Annotations. Perhaps we could leverage that into creating documentation that could be inline and also online.

I also thought it would be good to mention that the plugin that creates the online documentation first scans our prefs for all the info it needs, then creates JSON files for storing that information, then outputs that information in the online documentation. This was done to allow for old data to be parsed in new ways in the future, provide standard sets of data for making comparisons between versions and archiving old version data for displaying when that information is no longer available (when the system upgrades) Perhaps with a similar intermediary, and perhaps our own internal code, be it markdown in comments that will only get parsed in the online documentation. Or we could use or own non-standard “tags” inside inline documentation that could be parsed into a standard JSON (etc) format that could bring extra functionality to Swagger (etc.)

Whatever is done, I strongly believe splitting the documentation would be repeating past mistakes. Inline, online, or a combination thereof, let's not have duplicate documentation unless generated via some automated process.

Those are my thoughts. Not sure how much of it is helpful, as my knowledge of OpenAPI and such is limited.

Brendan



> Thanks!
> Victor
> 
> 
> 
> _______________________________________________
> TikiWiki-devel mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel



_______________________________________________
TikiWiki-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel