Content
API, OpenAPI V3 Spec and Code Generation
Added by Gunter Ohrner almost 3 years ago
Hi everyone,
TL;DR:
We need to programmatically access the data in our OpenProject instance from a Java application and I experimented somewhat with the available libraries.
It would be really cool to create some open source Java OpenProject API client (which does not yet seem to exist) if the results of our efforts yield a somewhat (re)usable result, but I'm currently still trying to find the best approach.
I'm well versed in basic Java, but so far didn't have any contact with the JSON+HAL standard nor with the OpenAPI V3 spec format.
Approach 1: Using Edison HAL, which is a Java library to simplify working with JSON+HAL APIs.
Edison HAL e.g. transparently deals with Curies, partially cached _embedded
objects and link
navigation.
It works quite well, even though it needs a patch as the OpenProject API does not fully seem to comply with the JSON+HAL draft:
The API returns href
references with the value null
, in which cases Edison HAL crashes with a NullPointerException
. Obviously, that's no too "robust" behaviour, but according to the latest JSON+HAL draft, the href
attribute's value has to be an
which null
is not. This probably stems from the guarantee the OpenProject API gives to fully embed the returned objects in _embedded
, but strictly speaking seems to violate the spec.
I patched Edison HAL to deal with these situations and then it seems to work fine, but requires me to implement my model objects manually, which is cumbersome and error prone, obviously.
Approach 2: Automated Code Generation using the OpenAPI Code Generator or Swagger CodeGen
I'd really like to know more about the background and state of the current OpenProject API OpenAPI V3 specification files.
My current impression is that they are - so far at least - mainly intended / used to generate the nice API documentation. Is this assumption appropriate?
However the mid to long term goal seems to be to create a spec which works for code generation, as this is also a possibility mentioned in the OpenProject Blog. Or is it already readily possible, and I'm just doing something terribly wrong?
An additional doubt I have concerning code generation is how well the support for the JSON+HAL peculiarities will be - like _embedded
everywhere and Curies, for example. Currently _links
seems to be modelled explicitly in the API spec, but ___embedded
is not.
Concerning the spec itself, there seem to be quite a few invalid or incompletely specified definitions in the files and this becomes obvious when trying to actually generate code based on these.
I started to try to fix up / streamline these files, but it would be a waste if I cannot somehow contribute these changes back into the official sources.
So it'd be cool to get some further insight on some of the ideas behind the way the spec is structured, so I'm not walking into the wrong direction.
Some problems in the API are obvious, like e.g. the details
attribute specification in ActivityModel
:
details:
type: array of formattable
This yields a completely empty ArrayOfFormattable
class in the generated code which is not used anywhere else.
For a more formalized specification, this should probably look something as follows:
details:
type: array
readOnly: true
items:
allOf:
- "$ref": "./formattable.yml"
readOnly: true
Actually, this change then seems to generate a proper and working attribute definition.
However less clear (at least to me) examples are e.g. the Work_Package_activitiesModel
. It seems to be completely empty, containing an example
, but no actual spec/schema. After code generation, this (understandably) causes the return value of WorkPackagesApi.listWorkPackageActivities(...)
to either just be Object
or an completely empty WorkPackageActivitiesModel
class (depending on the code generator used).
Adding some actual spec like
#(incomplete, i.e. missing further attributes and probably also somewhat buggy, but just to illustrate)
properties:
_embedded:
type: object
properties:
elements:
type: Collection
readOnly: true
items:
allOf:
- "$ref": "./activity_model.yml"
- description: Collection of Work Package Activities
readOnly: true
changes this to something usable which also actually runs and returns meaningful results.
There are actually quite a few Models like this, several of which I have already adjusted for testing / experimentation purposes.
And then there are things like all of those e.g. attachment_by_*(Model).yml
definitions, which are all completely empty, but which all basically just seem to be lists of attachments. Why so many variants, and what's eg. the intended difference between attachments_by_post_model.yml
and attachments_by_post.yml
?
I'm currently also playing around with these files, but currently lack the understanding how it's supposed to be modelled at this point.
I'd be really great if someone more experienced with the API spec and OpenAPI V3 could shed some light on this, or if there would be someone I could ask about this like this or necessary background info to perform approriate adjustments in the spec files.
I'd like to provide all fixes upstream, if someone's interested, but obviously don't want to publish "junk" and/or break the current API documentation.
Replies (1)
Hello Gunter,
I'm currently having a look at your recent PRs and within one of those I found the link to your topic in here. I hope, I can shed some light on your issues.
First of all: Approach 2 is definitely something we want to achieve, but we are well aware, that the spec is not fulfilling all the requirements to go down this road. I do not want to appologize in here, but the spec is quite young and was ported mostly automatically from grape half a year ago. So, we gladly accept every help we can get to get this spec along and eventually grant your mentioned "Approach 2" to every OpenProject user out there.
Of course, having the nice, fancy API docs was the first benefit to reap from the migration to OpenAPIv3. ;)
The empty models you mentioned are a relic of the migration, too. Eventually we will remove, replace or fill them with schemas. And even rework the examples.
Some of the issue you mentioned (i.e. array of formattable), were already fixed within the last weeks due to some polishing I made to the spec files. I'll have now a look on all of your PRs within the next days, so we both can garantuee, that we do not "publish junk". ;) But I'm confident, that we can significantly improve the spec together for the good of all users.
BG Eric