Some Kind Of Code

Generating an API SDK

Christopher Beloch -

A few weeks ago, I began working on a new project. Instead of working together with a team of web developers and designing all APIs from scratch, the server API was already available as we had to connect to the existing service of our client. A brief look at the API documentation revealed, that I have no desire in writing all the models and calls from scratch. As the documentation also had an interactive version supported by Swagger, including a swagger.json file that is a nice representation of all the models and APIs and the corresponding data that you'll have to send and receive, I took a dive into tools that could generate some kind of an SDK for the project.

I aimed for the following requirements:

  • generate models and API definitions from swagger.json
  • customizable templates
  • optional: multiple templates to support multiple platforms

I was expecting that I have to write my own tool. If I had to, I would probably go with Node.js, TypeScript, and the @apidevtools/swagger-parser package.

But after asking on Twitter I got pointed at the Swagger Codegen project. It is Java-based and I had no intention to install Java on my system. At least you can execute the CLI via Docker and the official container. You only have to write and compile Java code if you want to create a complete custom generator. But as they already provide a lot of templates, including a swift5 template, I gave that one a shot.

Swagger Codegen

The models are fine as they are. Structs conforming to Codable with optionally generated CodingKeys, all properties available, and even an initializer for all the properties. Everything public and ready to be used as a dedicated package. Well... almost.

The API files are what I was most concerned about. These will require customization from my side and I knew this before I started searching for solutions. But I wanted to see what the default output looks like.

The swift5 template at its core requires Alamofire and can be extended to support PromiseKit and/or RxSwift. I don't need any of these dependencies at the moment and the generated code comes with a bit of boilerplate code that I don't need as well. Luckily there is an option to only generate the API and model files.

-Dmodels -Dapis

Customizing the template is as easy as creating my own template directory and naming the templates the same way, the generator (click here for the swift5 generator files) uses them. Swagger Codegen uses mustache templating. The V3 Variant uses the Java implementation of Handlebars which is compatible with existing mustache templates. Documentation on the official sites is fine, but if you don't want to touch the Java code of the generators like me, you are stuck with the features the used libraries provide. And I got stuck pretty early: There is no option to lowercase or uppercase a value. At least I haven't found one without manipulating the generators java files and registering the StringHelpers that are available in the Handlebars Java implementation but have to be enabled manually.

This at the moment keeps me from using the Swagger Codegen project, but luckily there is an alternative by OpenAPI called the OpenAPI Generator. As Swagger makes use of the OpenAPI specifications this could work.

Next try: OpenAPI Generator

It uses Java as well and a Docker container is available, too. And most important: A full-fledged documentation website including a section about modifying and writing templates and even the options of the swift5-generator.

The OpenAPI Generator uses basically the same logic as the Swagger Codegen project, but templates have a few more extensions to use in the mustache templates. These extended functions are called lambdas. These are neatly documented in the templating documentation I linked above.

As lambdas for uppercase and lowercase are available, I began customizing the output. I only had to create my own version of the api.mustache file inside my own template folder. As a reference, I used the original file to get to know about all the properties and the templating syntax.

You can use the same command line parameter as for Swagger Codegen above to only generate API and model files. I ended up adding some of my own support files and customized the output by using a  config.yml and told the CLI to use it. Of course, there is documentation.

-c <path to config.yml>

I disabled all the other files and ended up with a config like this:

templateDir: /local/templates/swift-customized
projectName: MyAPI
useBacktickEscapes: true
useJsonEncodable: false 
files:
  StringRepresentation.swift:
    folder: MyAPI/Classes/OpenAPIs
  JSONEndpoint.swift:
    folder: MyAPI/Classes/OpenAPIs
  XcodeGen.mustache: 
    folder: unused
  APIHelper.mustache:
    folder: unused
  APIs.mustache:
    folder: unused
  CodableHelper.mustache:
    folder: unused
  Cartfile.mustache:
    folder: unused
  Configuration.mustache:
    folder: unused
  Extensions.mustache:
    folder: unused
  JSONDataEncoding.mustache:
    folder: unused
  JSONEncodingHelper.mustache:
    folder: unused
  Models.mustache:
    folder: unused
  OpenISO8601DateFormatter.mustache:
    folder: unused
  Podspec.mustache:
    folder: unused
  README.mustache:
    folder: unused
  SynchronizedDictionary.mustache:
    folder: unused
  URLSessionImplementations.mustache:
    folder: unused 
  gitignore.mustache:
    folder: unused 
  git_push.sh.mustache:
    folder: unused 
  api_doc.mustache:
    folder: unused

What I did was move everything that I don't need into a folder called unused and added my custom files to the target location at the top. As you can see, I will not use the Podspec or Cartfile templates, as we only use Swift Package Manager at this stage.

As I created a small shell script to run the generator, I added a few lines to delete the unused-folder after generating everything and also used SwiftFormat at the end, because writing the templates with a good output formatting was way harder than I thought.

It is only as good as the documentation

Over the time of regenerating and using the generated code in the project, my colleague and I have noticed that sometimes the attributes of the models got moved around which resulted in a different parameter order for the initializer of that model. This of course ended up messing with the code where we created our own instances as input to an API.

This happened because the swagger generator on the server side messed it up on a few models over time. To fix that, I added a python script to sort all the keys of the swagger.json before passing it to the OpenAPI Generator.

Some data types have been messed up like a Date response type was documented as Int, but that could be fixed after editing the documentation header on the server side.

Creating a script

As I have mentioned above, I did a bunch of extra stuff to get an output I was satisfied with.

  1. Download the swagger.json from the server
  2. Sort the JSON
  3. Run the generator via docker
  4. cleanup output
  5. format output

Your mileage may vary depending on your server-side documentation generator and the output SDK language you use.

My shell script for the Swift output looked like this at the end:

LANGUAGE="swift5"
SOURCE="https://raw.githubusercontent.com/OAI/OpenAPI-Specification/main/examples/v3.0/petstore-expanded.json"
SOURCE_DUMP_FILE="swagger.json"

# 1. download swagger.json
curl $SOURCE > $SOURCE_DUMP_FILE

# 2. sort swagger json to sort model properties
python3 jsonSort.py $SOURCE_DUMP_FILE

# 3. generate SDK
docker run 
  --rm \
  -v "${PWD}:/local" \ # link current folder to /local in the container
  openapitools/openapi-generator-cli generate \
  -i /local/$SOURCE_DUMP_FILE \
  -g $LANGUAGE \
  -o "/local/out/$LANGUAGE" \
  -c /local/config-iOS.yaml

# 4. Cleanup unused files
rm -r "${PWD}/out/$LANGUAGE/unused"
rm -r "${PWD}/out/$LANGUAGE/docs"
rm -r "${PWD}/out/$LANGUAGE/.openapi-generator"
rm "${PWD}/out/$LANGUAGE/.openapi-generator-ignore"
rm "${PWD}/out/$LANGUAGE/.swiftformat"

# 5. Swift Format
if which swiftformat >/dev/null; then
  swiftformat "${PWD}/out/$LANGUAGE/"
else
  echo "warning: swiftformat not installed."
  echo "Install via"
  echo "\tbrew install swiftformat"
fi

Feel free to modify it to your needs.

Summary

It was a bit of work to learn the templating syntax and find out about the parameters and properties to use. But looking at the original templates helped a lot.

The project where I used this generator had over 30 Endpoints and over 200(!) models. Writing all these by hand would've been madness. And they changed in some spots over time. Some of the model files are not even used, because abstract model definitions are treated as a regular model. For example, we have an abstract DatabaseObject model, that has an id, createdAt and updatedAt field and it is the base for most of the models. This, in my case, will generate a corresponding Swift model, but the other models are not a subclass of this model. They still have the properties on their own, but in the code, we will never refer to the DatabaseObject model.

In some places, it would've been cool to use the abstract definitions as a protocol, but I didn't find a neat way to realize it. But you can always create a few extensions by hand if you need them. We tried never to hack the generated files and keep the output as clean as possible. Any extension happened separately from the API Package.

All the time I used to modify everything to our needs was nothing compared to the time this saved us now and will in the future as this reconfiguration for our architecture can be reused very easily for any future project.