Automating OpenAPI Documentation Generation

Maintaining up-to-date API documentation can be a significant challenge in fast-paced development environments, particularly for staging environments. Outdated or inaccurate API documentation can lead to confusion, integration issues, and increased development/testing time for API consumers. Recognizing this pain point, an automated workflow for generating and publishing OpenAPI documentation we implemented, significantly improving the reliability and timeliness of our API information. This blog details the previous manual process and the enhanced automated system.

Manual API Documentation Process

Our initial approach to API documentation was largely manual and prone to inconsistencies. Developers were responsible for updating the OpenAPI specification whenever changes were made to the codebase. This often involved:

Manual Updates: Developers would modify the OpenAPI specification files (typically YAML or JSON) and commit them to reflect API changes.
Lack of Validation: There was no documentation validation in place, which meant developers had no way of knowing if their commits would break the API documentation build.
Language Review: Writers manually reviewed the summaries and descriptions added to the API specification in the source YAML files.
Manual Publishing: Once a release branch was finalized, writers would manually upload the source files to the documentation repository and deploy them to the API documentation portal.

This process was time-consuming, error-prone, and often resulted in documentation lagging behind the latest API versions. The lack of automation meant that keeping the documentation current required constant follow-ups and efforts.

Visual representation of the manual process:

%%{init: {'theme':'base', 'flowchart': {'layout': 'elk'}}}%% flowchart LR A["Update OpenAPI Spec in Source Code"] A --> C["Commit & Push YAML/JSON Files"] C --> D["Writer Review"] D --> F["Manual Upload to Docs Repo"] F --> G{"Render HTMLs (Redocly)"} G -- "Fail" --> H["Troubleshoot Errors"] G -- "Pass" --> I["Deploy API Documentation"] H --> I

Automated OpenAPI Documentation Process

To address the shortcomings of the manual process, I designed an automated workflow leveraging Redocly, python scripts, and CICD. This system automatically fetches, validates, and publishes OpenAPI documentation directly from the source code. For the development branch, this is implemented on a 12 hour cycle. The production workflow depends on the release branch and is triggered as needed rather than on a schedule. The key components of this new workflow include:

Scheduled Fetching: A process is scheduled to run every 12 hours, automatically fetching the latest OpenAPI specification from the designated source repository.
Upstream Validation: The API specification undergoes validation using Redocly. An optional validation step is introduced in the source repository where every time an API spec is updated, the commit triggers a Redocly build command to convert all the YAML files to HTML files. Any errors reported here are automatically emailed to the writing team, indicating which commit caused a specific error in a particular YAML file. This has significantly reduced the turnaround time for troubleshooting API documentation generation errors.
Automated Publishing: The source files are fetched into the documentation repository, and a CI/CD pipeline is configured to automatically render the HTML files generated by Redocly. The updated OpenAPI documentation is then automatically published to our documentation portal.
Error Reporting: In case of validation failures or any other errors during the process, detailed error reports are generated.
Email Notifications: The writing team receives automated email notifications about the success or failure of each documentation update, ensuring we are promptly informed of any issues.

This automated approach ensures that our API documentation remains consistently in sync with the latest changes in the source code, without requiring manual intervention.

Visual representation of automated process:

%%{init: {'theme':'base', 'flowchart': {'layout': 'elk'}}}%% flowchart LR A["API Changes in Source Code (Dev Branch)"] A --> B{"Optional Validation in Source Repo</br>(Redocly Build on Every Commit)"} B --> C["Scheduled Job (Every 12 Hours)"] B --> J["Email Notification to Writing Team (Commit Errors)"] C --> D["Fetch OpenAPI Spec into Docs Repo"] D --> E{"Render HTMLs (Redocly)"} E -- Pass --> F["Publish Docs to Portal"] F --> G["Email Notification (Success)"] E -- Fail --> H["Generate Error Report"] H --> I["Email Notification (Failure)"]

Benefits of the Automated Workflow

Implementing this automated OpenAPI documentation generation workflow has yielded significant benefits:

Always Up-to-Date Documentation from the development branch: The 12 hour update schedule ensures that our documentation closely reflects the current state of our APIs in the development branch.
Reduced Manual Effort: Writers are no longer burdened with the manual task of updating, troubleshooting, and publishing documentation, freeing up their time for other critical tasks.
Improved Accuracy: Automated validation reduces the risk of publishing incorrect or invalid OpenAPI specifications. This also means that publishing from the release branch is much faster and more reliable, as the development branch is constantly up-to-date and reviewed for issues.
Proactive Error Detection: Automated error reporting and email notifications allow us to quickly identify and address any issues with the documentation generation process.
Increased Reliability: The automated system provides a more reliable and consistent approach to documentation management compared to the previous manual process.

Keywords

API Documentation OpenAPI Redocly Swagger Postman Automation CI/CD Git Python YAML Technical Writing Docs-as-Code