Skip to content

Commit

Permalink
Merge pull request #6 from cisco-developer/am-updates
Browse files Browse the repository at this point in the history
simplify doc-openness page
  • Loading branch information
coliu19 authored Oct 17, 2022
2 parents d97e003 + 10ffea7 commit 75a5c28
Show file tree
Hide file tree
Showing 5 changed files with 210 additions and 101 deletions.
92 changes: 35 additions & 57 deletions docs/doc-completeness.md
Original file line number Diff line number Diff line change
@@ -1,27 +1,21 @@
# Completeness of an API Contract
# Documentation Completeness Ruleset

How complete is an OpenAPI Specification (OAS) document ?
The Documentation Completeness Ruleset consists of 2 domains:
- [API definition completeness](#api-definition-completeness) (paths, schemas, response status and error codes):
- Must be a well-formed JSON or YAML document
- Must have an OAS version
- Must be a well-formed OpenAPI document
- Must contain meta information about the API itself
- Must contain a schema definition
- Must have response schema defined
- Must have success status code definitions
- Must have error status code definitions
- [Reference documentation completeness](#reference-documentation-completeness) (descriptions, examples):
- Must have descriptions for every attribute
- Must have examples for every schema

Completeness of an OAS document considers 2 domains:

1. API Definition completeness (paths, schemas, response status and error codes)

2. Reference Documentation completeness (descriptions, examples)


1 STATIC - API Definition Completeness
- Must Be a Well-formed JSON or YAML document
- Must have an OAS Version
- Must Be a Well-formed OpenAPI document
- Must contain Meta Information about the API itself
- Must Contain a Schema Definition
- Must Have Response Schema Defined
- Must have Success Status Code Definitions
- Must have Error Status Code Definitions

2. STATIC: Reference Documentation Completeness
- Must have Descriptions for Every Attribute
- Must have Examples for Every Schema
## Rules

| name_id | description | severity | mitigation |
| ---------------------------------------- | ------------------------------------------------------------------------- | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
Expand All @@ -40,96 +34,80 @@ Completeness of an OAS document considers 2 domains:
| examples-for-every-schema | Examples are not provided for some of the schemas. | warning | Please add examples for the schemas identified. [Reference](#examples-for-every-schemaexamples-for-every-schema) |


Details on rules to check the level of completeness of an OAS document

## 1. STATIC - API Definition Completeness
No partial definitions such as no use of objects / array of objects.

## API Definition Completeness
Check the completeness of an OAS document.


### Well-formed JSON or YAML document

- The document must be syntactically correct regarding the JSON or YAML syntax, parser should not complain

- Mark as “Incomplete Definition: “malformed OAS document, not adhering to format” with format assumed (either JSON or YAML)

### oas-version: Version of the OAS is missing

- The document must specify the OAS version it is supporting.

- Mark as "Incomplete Definition: 'malformed OAS document, not adhering to the OpenAPI v %s'” with %s = the version of the spec declared


### oas[2|3]-schema: Well-formed OpenAPI document

- The document must be syntactically correct regarding the version of OAS it declares Mark as “Incomplete Definition: 'malformed OAS document, not adhering to the OpenAPI v %s'” with %s = the version of the spec declared
- The document must be syntactically correct regarding the version of OAS it declares



### Meta Information about the API itself

The following fields must be present (note: list of fields depend on the OAS version) List for OAS v2:
- info
- title
- version
- basepath
- License
- SecurityDefinitions

List of that are implemented for this check.
- oas[2|3]-meta-info
- info-contact
- info-description
- info-license
The following fields must be present (note: list of fields depend on the OAS version)
List for OAS v2:
- info
- title
- version
- basepath
- License
- SecurityDefinitions

List of that are implemented for this check.
- oas[2|3]-meta-info
- info-contact
- info-description
- info-license

### oas[2|3]-missing-schema-definition: Missing Schema Definition

- There is no schema attribute for a component

- mark as “Incomplete Definition: there are %d1 schema definition missing for a total of %d2 objects defined”


### general-schema-definatio: Generic Schema Definition

- Some of the defined schema use object as a final field when describing their object structure.

- Mark as “Incomplete Definition: there are %d1 schemas that partially define their structure for a total of %d2 schemas”


### oas[2-3]-missing-returned-representation: Missing Returned Representation

- 2XX (except 204) and 4xx responses must have a response schema defined.

- Note1: even some 204 may have a body but it can be ommited, so let’s take 204 out of this check

- Note2: generally 5xx will have a body but we don’t need to check this formally since 5xx is generally driven by the operations (and eventually delegate to an API Gateway).


### success-status-code: Missing Success Status Code

- For every operation in the OAS document, there should be at least one success status code defined.

- A successful status code is in the 1xx, 2xx or 3xx range series, and generally a 200, 201 or 204. Missing response schema object properties.

- Mark as “Incomplete Definition: 'missing success status code for %d operations'” with %d = the number of operations that do not declare success status code among their responses

### error-status-code: Missing Error Status Codes There should be at least one error status code either 4xx or 5xx (or default per the OpenAPI spec / search for Default Response)

- Example here: there are no error codes, only 200 OK https://wwwin-github.cisco.com/stsfartz/oas_docs/blob/master/threatresponse/raw/iroh-enrich.20220324.json#L6698

- Mark as “Incomplete Definition: 'missing error status code for %d operations'” with %d = the number of operations that do not declare error status codes among their responses


## 2. STATIC: Reference Documentation Completeness
## Reference Documentation Completeness

### description-for-every-attribute: Descriptions for Every Attribute

- For every attribute that is present in the OAS document, if a description is proposed as optional to complement that attribute, then yes it must be present.

- Mark as "Incomplete Definition: 'missing descriptions for %d1 attributes among %d2 total attributes that should be documented'” with %d2 = the total number of attributes in the OAS document for which the OpenAPI declares an optional description, and %d1= the number of attributes among d2 that have no description


### examples-for-every-schema:Examples for Every Schema

- For every schema provided in the OAS document, at least one example must be present

- Mark as "Incomplete Definition: 'missing examples for %d1 schema among %d2 total number of schemas'“ with %d2 = the total number of schemas in the OAS document and %d1 = the number of schemas among d2 that do not have at least one example provided.

113 changes: 113 additions & 0 deletions docs/rest-additional.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
# REST Additional

This section proposes additional best practices and recommendations for more consistency of APIs.

## Error Handling

Just like an HTML error page shows a useful error message to a visitor, an API should provide a useful error message in a consumable format.

The API definition - typically using OpenAPI specifications - and API reference documentation should include detailed descriptions about errors that may be returned by operations: these descruotuibs are part of your API contract, and provide important information for developers to handle correctly these situations and helps with troubleshooting.

### HTTP Status Codes

As detailled in section [Returning HTTP Status Codes](./rest-style.md#returning-http-status-codes), your API should return standardized HTTP status codes whose values meet the expectations of the developer community given the outcome of operations.

API errors typically break down into 2 types:
- _4xx_ series status codes for client issues,
- and _5xx_ series status codes for server issues.

At a minimum, the API should standardize that all _4xx_ series errors come with defined, machine-consumable error representations.

When possible, this should extend to _5xx_ series status codes (i.e. if load balancers & reverse proxies can create custom error bodies).

### Error Representations

<!-- reco API.REST.PATTERNS.01 -->
<h6 id="API.REST.PATTERNS.01"></h6>

> **Recommendation** \
> Linter Rule - API.REST.PATTERNS.oas-error-message : Error representations include a useful human-readable message.
<!-- recostop -->
The representation of an error should be no different than the representation of any resource, just with its own defined schema, fields and data.

An error representation should provide several items relevant for the consumer:
- certainly a useful human-readable error message;
- possibly a unique error type/code (that can be looked up for more details in the docs);
- ideally, a detailed description/breakdown with hints as to what went wrong and how to fix it.

The simplest example just provides a human-readable `message` field giving a brief description of the error:

```json
{
"message": "Incomplete request object"
}
```

The HTTP status code of the response will indicate the general error category of an error, but it may be appropriate to provide a more detailed machine-readable error code that can be mapped to a table of known error conditions:

```json
{
"code": "3006",
"message": "Incomplete request object"
}
```

Even more helpful responses might include a detailed error description, perhaps including suggestions on how to correct the error:

```json
{
"code": "3006",
"message": "Incomplete request object",
"description": "Required field 'lastName' is missing"
}
```

Validation errors with PUT, PATCH and POST requests may wish to provide a field-level breakdown. For example, this could be modeled using a fixed top-level error code, with detailed errors in an 'errors' array:

```json
{
"code": "1024",
"message": "User object validation failed",
"errors": [
{
"code": "5432",
"field": "firstName",
"message": "firstName cannot have fancy characters"
},
{
"code": "5622",
"field": "password",
"message": "Password cannot be blank"
}
]
}
```

### Processing Errors

A server may choose to stop processing as soon as a problem is encountered, or it may continue processing and encounter multiple problems. For instance, a server might process multiple attributes and then return multiple validation problems in a single response.

When a server encounters multiple problems for a single request, the most generally applicable HTTP error code should be used in the response. For instance, `400 Bad Request` might be appropriate for multiple _4xx_ errors or `500 Internal Server Error` might be appropriate for multiple _5xx_ errors.

### Troubleshooting

<!-- reco API.REST.PATTERNS.02 -->
<h6 id="API.REST.PATTERNS.02"></h6>

> **Recommendation** \
> Linter Rule - API.REST.PATTERNS.error-response-identifier: Error representations include an identifier to help with troubleshooting.
<!-- recostop -->
As troubleshooting errors will commonly involve analyzing trace logs, it is important that your API implements [TrackingID headers](./rest-style.md#trackingid-header).

Such identifiers - or any other identifier which can help pinpoint errors all the way to your API logs - should also be added to error representations.

As an example, this is a Webex error message returned for a misplace `GET https://webexapis.com/v1/people` request:

```json
{
"message": "Email, displayName, or id list should be specified.",
"trackingId": "WEBEX-DEV-PORTAL_4bb4729e-28f3-4484-8435-bc65dd1721be_4"
}
```
27 changes: 14 additions & 13 deletions docs/rest-conventions.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,22 @@
# REST Conventions

This section proposes conventions when designing REST APIs such as naming resources and representations in order to get better consistency across services APIs.
This section proposes conventions when designing REST APIs such as naming resources and representations in order to get better consistency across APIs.

<!-- toc -->

- [REST Resources](#rest-resources)
* [Resource Names](#resource-names)
* [Functional Resources](#functional-resources)
* [Capitalization](#capitalization)
* [Case Sensitivity](#case-sensitivity)
* [Inclusive and Bias-Free Naming](#inclusive-and-bias-free-naming)
- [REST Representations](#rest-representations)
* [Encoding](#encoding)
* [Returned Collections](#returned-collections)
* [Representation Fields](#representation-fields)
* [Data Types](#data-types)
* [Hypermedia Links](#hypermedia-links)
- [REST Conventions](#rest-conventions)
- [REST Resources](#rest-resources)
- [Resource Names](#resource-names)
- [Functional Resources](#functional-resources)
- [Capitalization](#capitalization)
- [Case Sensitivity](#case-sensitivity)
- [Inclusive and Bias-Free Naming](#inclusive-and-bias-free-naming)
- [REST Representations](#rest-representations)
- [Encoding](#encoding)
- [Returned Collections](#returned-collections)
- [Representation Fields](#representation-fields)
- [Data Types](#data-types)
- [Hypermedia Links](#hypermedia-links)

<!-- tocstop -->

Expand Down
Loading

0 comments on commit 75a5c28

Please sign in to comment.