image by stable difussion, prompt by alswl
This issue has bothered me for a long time, starting during my school days, and every time I needed to communicate and discuss with my team members.
From the initial free style to the later REST, I often referenced Github v3 to the project team and the
Foursqure API (no longer accessible, exposing age) documentation.
However, in practice, there are still cases where there is a mismatch with the actual work or the common company specification.
This is when I need to do some additional work. Eventually, I will write a brief DEVELOPMENT.md
document to describe the design solution.
But I’ve always had more ideas for that document, and it’s not quite good enough. So, I wanted to put together a Simple and Pragmatic Web API best practice, which is this article.
The problem seems obvious, but a deeper analysis involves teamwork efficiency and engineering design philosophy.
API (Application Programming Interface) is the bridge between different software systems to interact with each other. When communicating between different software systems, the APIs allow data transfer and processing in a standardized way, enabling the integration of various applications.
When we start writing API documentation, a paradigm (Design Pattern) emerges, whether it is explicit or implicit, whether Whether it is one set for each or a common set. This is just like when we use a unified USB interface, unification reduces costs and avoids possible errors. Specifically, there are several reasons for this:
image by alswl
While it is true that there are some costs associated with the use of a unified specification that requires framework understanding and promotion, I believe that in most scenarios the benefits of a unified specification far outweigh these costs.
However, not all cases require consideration of the API specification. For short lifecycle projects, internal projects and products with a very small impact specifications may not require much attention. In addition, in some special business scenarios, the The underlying protocol may change, and the existing specification may no longer be applicable. But even so, I still recommend redrafting a new specification, rather than abandoning it.
When developing API specifications, we should follow some basic principles to deal with technical differences, and I have summarized three principles that have gained wide acceptance:
image by alswl
The RESTful API has become a popular protocol in the Web API space. Its broad applicability and reach stems from its binding to the HTTP protocol, which allows RESTful APIs to easily interact with existing Web technologies. If you are not familiar with REST, you can check out [Yifeng Ruan’s You can check out Yifeng Ruan’s RESTful API Design Guide and Best Practices for RESTful API Design.
REST is a high maturity protocol, which Leonard Richardson describes it as having four levels of maturity:
image by alswl
rel
links, JSON:API is the pinnacle of performanceThe core strength of REST is that
However, REST is not a specific protocol or specification, but a style philosophy. While REST defines rules and principles such as the identification of resources, unified interfaces, stateless communication, etc., it does not prescribe a specific protocol or specification. it does not prescribe a specific way to implement them. As a result, in practice, different teams may have different understandings and practices, leading to inconsistencies and portability of the API. This leads to inconsistency and reduced maintainability of the API.
In addition, REST has a number of limitations and drawbacks:
/login
) operation, which is very convoluted when converted to session
;
The same problem arises for operations like money transfers, where the limited number of HTTP verbs cannot support all business scenarios.Therefore, although the REST style is a good guideline, the specific implementation needs to be combined with specific business requirements and technical features to make some trade-offs in order to achieve a good API design. Finally, do we need a Web API design specification that follows the REST style? I think REST can solve 90% of the problems, but there is still 10% need to specify the details.
Since our protocol is based on HTTP and REST design, we will base our discussion on the four core parts of an HTTP request: URL, Header, Request and Response. These are: URL, Header, Request and Response.
My inspiration for URL design came from Ruby on Rails. Before that, I always instinctively put model information on top of URLs, but in reality good URL design should be a plan for the information structure of the system. Therefore, URL design should consider not only the API, but also the user-oriented Web URL.
To achieve good URL design, I have summarized the following rules:
Typically, the URL model is as follows:
/$(prefix)/$(module)/$(model)/$(sub-model)/$(verb)?$(query)#${fragment}
where Prefix may be the version of the API, or it may be a special qualification, as some companies rely on for access layer triage; Module is a business module, which can also be omitted; Model is a model; SubModel is a submodel, which can be omitted; Verb is a verb, which can also be omitted; Query is a request parameter; Fragment is an HTTP primitive Fragment.
It is important to note that not all components are required to appear. For example, fields such as SubModel and Verb can be allowed to be hidden in different URL styles.
Design Style Options
Note: Please note that there is no association between Scenario A / B / C and no association above or below each line
Problem | Explanation (see separate analysis below) | Option A | Option B | Option C |
---|---|---|---|---|
Dos API path’s prefix | /apis |
/api |
second-level domain | |
Does the path contain the API version | Version advantage in URLs | ✅ | 🚫 | |
Does path contain a group | ✅ | 🚫 | ||
Whether path contains action | HTTP Verb is not enough | ✅ | 🚫 (pure REST) | Depends (include if HTTP Verb CRUD cannot be satisfied) |
Model ID form | Readable Stable Identity explained | Self-incrementing ID | GUID | Readable Stable ID |
Model singular or plural in URL | Singular | Plural | List Plural, One-Way Singular | |
Is the resource one-level (tiled) or multi-level (nested) | Explanation of one-level and multi-level | One-level (tiled) | Multi-level (nested) | |
How is search implemented, standalone API (/models/search ) or list-based /models/ API |
standalone | list-based | ||
Does it have an Alias URL | Alias URL explanation | ✅ | 🚫 | |
Does the model in the URL allow abbreviations (or refinements) | Model abbreviations explained | ✅ | 🚫 | |
Hyphenation of multiple words of the model in the URL | - |
_ |
Camel | |
Whether to distinguish between Web API and Open API (non-browser oriented) | ✅ | 🚫 |
Version advantage in URLs
We follow the principle of consistency when designing URLs; the same URL is used to access the same resource, regardless of identity or state. This is also the basic principle of Uniform Resource Location. While we can accept different content formats (e.g. JSON / YAML / HTML / PDF / etc.) but we want the location of the resource to be unique.
However, the question is, should the same resource be reflected in the URL for rendering between versions? This depends on whether the designer considers versioning to be in the realm of location information.
According to the RFC design, in addition to the URL there is URN (Uniform Resource Name). The latter is used to identify the resource, while the URL points to the resource address. In practice, URNs are not so widely used that URIs are almost equivalent to URLs.
HTTP Verb is not enough
In REST design, we need to operate on resources using HTTP verbs like GET / POST / PUT / DELETE / PATCH / HEAD.
For example, using the API GET /apis/books
to view book listings is natural and reasonable.
However, when it comes to performing an action like “borrow a book
we don’t have a suitable verb (BORROW) to represent it. In this case, there are two possible options:
POST /apis/books/borrow
, for the action of borrowing a book;POST /apis/books/borrow-log/
;This issue comes up a lot in complex scenarios such as user login (POST /api/auth/login
vs POST /api/session
) and account transfers (vs transfer record creation), etc.
API abstract or concrete, there is always an explanation of the business. We can’t simply generalize all of our business to CRUD.
Rather, we need to divide the business wisely to make it clearer and more understandable to users.
When designing, we can consider whether we need to create a corresponding button for each API to make it easier for users to do so.
If there is only one API named /api/do
and all the services are bound to it, although technically possible, this design does not meet the business requirements.
Each layer of abstraction is designed to standardize the solution to a particular problem, and the TCP L7 design is a reflection of this philosophy.
Readable Stable Identity explained
When marking a resource, we typically have several options:
I have a personal design tip: use a slug of the form ${type}/${type-id}
to describe an identifier.
For example hostname/abc.sqa
or ip/172.133.2.1
.
This design approach provides a good balance between readability and uniqueness.
A slug is a human-readable, unique identifier, used to identify a resource instead of a less human-readable identifier like an id .
from What’s a slug. and why would I use one? | by Dave Sag
PS: At the end of the article I’ll also introduce an Apple Music solution that combines the ID / Readable / Stable features.
Explanation of one-level and multi-level
The hierarchical design of URLs can be based on modeling, or a straightforward single-level structure can be used. specific problem solving approach.
For example, when designing user-owned books, one can choose a multi-level structure /api/users/foo/books
or a one-level structure /api/books?owner=foo
.
Both options are technically possible, the former respecting the attribution of the model, and the latter focusing on the simplicity of the URL structure.
A multi-level structure is more intuitive, but also needs to address the issue of possible multiple ways of organization, such as books in a library organized by author or category?
In this case, consider making the attribution of models explicit in a multi-level structure
For example /api/author/foo/books
(based on authors) or /api/category/computer/books
(based on categories).
Alias URL explanation
For some frequently used URLs, although they can be designed according to URL rules, we can still design a more concise URL that to facilitate user presentation and use. This design is especially common for Web URLs. For example, the API for a library’s most popular books:
# Original URL
https://test.com/apis/v3/books?sort=hot&limit=10
# Alias URL
https://test.com/apis/v3/books/hot
Model abbreviations explained
Often, when modeling resources, longer names are used to name them, for example a book index might be named BookIndex
instead of Index
.
When rendering in URLs, since the URL prefix of /book/book-index
contains Book, we can reduce the layer of description and
We can make the URL more concise, for example by using /book/index
. This technique is very common in Web URL design.
There is also a strategy of model abbreviation, which provides a complete set of alias registration schemes. aliases that are globally unique.
For example, in Kubernetes, Deployment
is a common naming, while apps/v1/Deployment
is the full name by adding a Group restriction.
There is also a shorthand for deploy
. This mechanism relies on Kubernetes’ API Schema system for registration and work.
We often overlook the importance of the Header. In fact, the selection of HTTP verbs, HTTP status codes, and various authentication logic (e.g., cookies / Basic Auth / Berear Token) all depend on the design of the Header. In fact, the selection of HTTP verbs, HTTP status codes, and various identity verification logic (e.g., Cookie / Basic Auth / Berear Token) all depend on the design of the Header.
Design style selection
Problem | Explanation (see separate analysis below) | Option A | Option B | Option C |
---|---|---|---|---|
Are all verbs using POST | About only using POST | ✅ | 🚫 | |
Is the Modify action POST or PATCH? | POST | PATCH | ||
HTTP status return code | 2XX family | Make full use of HTTP Status | Use only core status (200 404 302, etc.) | Use only 200 |
Is using rate limiter system | ✅ 429 | 🚫 | ||
Is using cache system | ✅ ETag / Last Modify | 🚫 | ||
Is validate UserAgent | ✅ | 🚫 | ||
Is validate Referrral | ✅ | 🚫 |
About only using POST
Some novices (or those who consider themselves experienced) may come to the erroneous conclusion that, with the exception of GET requests all HTTP requests should use the POST method. Some even demand that all actions (even read-only requests) should use the POST method. This argument is often supported on the grounds of “simple consistency”, “cache avoidance”, or “operator requirements”.
However, it is important to understand the original design of the HTTP method: it is used to describe the type of resource manipulation that gives rise to a number of issues including caching, security, and idempotency. In relatively simple scenarios, omitting this layer of abstraction really doesn’t pose much of a problem, but once you get into the complexities of the domain, the Using this layer of abstraction for HTTP methods becomes very important. Whether this follows the standard or not will determine whether you get the benefits of standardization. The analogy is like a new phone manufacturer that can choose not to use the USB TypeC interface. It’s technically possible, but it also loses a lot of standardization support and mental agreement.
I especially like one Knowing user’s comment: “Routing hasn’t gone away, it’s just shifted.
2XX family
The purpose of HTTP status codes is to indicate the result of communication between the client and the server. 2XX status code family means that the server has successfully received, understood, and processed the client request, understood and processed the client request, and the response was successful. The following are common status codes in the 2XX family and their meanings:
The 2XX family of status codes indicates that the request has been successfully processed. These status codes let the client know explicitly that the request has been processed correctly so that it can proceed to the next step.
The need to use the 2XX family of status codes across the board depends on whether the client needs to be explicitly/displayed with information that inform it of the next action. If it has been clearly described by other means (including documentation, verbal agreements) then it is indeed possible to use 200 status codes across the board for return. But passing meaning based on behavior. or document-based (or even verbal)? Is it more complex or more concise?
Design style selection
Problem | Explanation (see separate analysis below) | Option A | Option B | Option C | |
---|---|---|---|---|---|
Whether complex parameters are put into Form Fields or a separate JSON Body | Form Fields | Body | |||
Whether the subresource is a one-time query or an independent query | one-time | independent | |||
Paging parameter storage | Header | URL Query | |||
Paging parameter method | Explanation of paging method | Page based | Offset based | Continuation token | |
Paging Controllers | Paging controller explanation | Client | Server |
Explanation of paging methods
The two most common types of paging we see are Page-based and Offset-based, which can be mapped by formula.
In addition, there exists a method called Continuation Token, which is similar in technique to Oracle’s
rownum paging scheme, using the parameter start-from=?
to describe it.
While the advantages and disadvantages of Continuation Token are significant, using this approach allows sequential to be used as an alternative to random.
Paging controls interpretation.
In some cases, we need to distinguish between Client Pagination and Server Pagniation.
Client Pagination means that the parameters of the next page are calculated by the client, while Server Pagination is a protocol such as rel
or JSON.API returned by the server.
Using Server Pagniation avoids some problems, such as bulk blocking of some content, which may result in missing pages or white screens if client-side paging is used.
Design Style Selection
Problem | Explanation (see separate analysis below) | Option A | Option B | Option C |
---|---|---|---|---|
Model presentation types | Several forms of models | Single model | Multiple models | |
How large models contain sub-model models | Linking, sidecar, and embedding of models | Embedding | Core model + multiple linked resource queries | Linking |
Whether field returns are on-demand or grouped or unified | unified | Use the fields field on demand |
||
field presentation format | Snake | Camel | ||
Error code | No customization, use Message | Custom | ||
Error format | global uniform | on-demand | ||
Time Zone | UTC | Local | Local + TZ | |
HATEOAS | ✅ | 🚫 |
Several forms of models
In API design, there are several definitions for how models should be represented. Although this is not a topic that must be discussed in the API specification, it is very important for API design.
I have divided the model presentation that models are often described into the following categories, which are not professionally defined and borrow some definitions from the Java context below. These names will be called differently in different companies and even different teams:
image by alswl
In addition, two other categories are often used: Rich Model and Tiny Model (please ignore the naming, it varies a lot from team to team):
Linking, sidecar, and embedding of models
In API design, we often need to deal with cases where a model contains multiple submodels, such as a Book containing Comments. For this case, there are usually three representations to choose from: Link, Side, and Embed.
image by alswl
Linking (sometimes this URL is also hidden, based on an implicit protocol between client and server for requests):
{
"data": {
"id": 42,
"name": "朝花夕拾",
"relationships": {
"comments": "http://www.domain.com/book/42/comments",
"author": [
"http://www.domain.com/author/鲁迅"
]
}
}
}
Sidecar:
{
"data": {
"id": 42,
"name": "朝花夕拾",
"relationships": {
"comments": "http://www.domain.com/book/42/comments",
"authors": [
"http://www.domain.com/author/鲁迅"
]
}
},
"includes": {
"comments": [
{
"id": 91,
"author": "匿名",
"content": "非常棒"
}
],
"authors": [
{
"name": "鲁迅",
"description": "鲁迅原名周树人"
}
]
}
}
Embedding:
{
"data": {
"id": 42,
"name": "朝花夕拾",
"comments": [
{
"id": 91,
"author": "匿名",
"content": "非常棒"
}
],
"authors": [
{
"name": "鲁迅",
"description": "鲁迅原名周树人"
}
]
}
}
There are some other issues that are not converged in the four elements, but we often encounter them in engineering practice, and I will run them out:
**I’m not in the HTTP protocol, what should I do? ** I’m not in the HTTP protocol, what should I do?
Non-HTTP protocols are less commonly encountered in Web APIs, and the cost of creating a new set of protocols is too high. There are protocols that are introduced in certain areas. For example, MQTT in the IoT domain.
In addition, RPC is a wide-ranging concept that goes far beyond the protocol level. Often we compare the transport protocols of HTTP and RPC, as well as serialization protocols. I believe that much of the discussion in this paper is also of importance to the RPC community.
Some teams or individuals plan to use protocols that they create themselves, but my opinion is that self-built protocols should be avoided as much as possible, because it is very rare that there is a real need to create a protocol. If a strong need does exist, then I would ask two questions: Have you read through the HTTP RFC document and the HTTP/2 RFC document?
**I’m not a remote service (RPC / HTTP etc.), but what about the SDK? ** This article focuses on Web APIs (RPC/ HTTP etc.).
This article focuses on the design specifications of the Web API (HTTP) and some of the rules can be applied to RPC systems. However, the basis of the discussion is based on Remote Service. If you are an SDK developer, you will have two roles, possibly as a client communicating with a remote server, and as an SDK providing developer-oriented services. You will also provide developer-facing interfaces as an SDK. For the latter, the following specifications can be used as a reference:
For the latter, you can refer to these specifications:
Authentication Authentication Scheme
In general, the Web API design will clearly describe the authentication and authentication system to be used. It is important to distinguish between the concepts of “authentication” and “authentication”. The topic of “authentication” can be discussed in a separate section, so this paper will not cover this aspect.
In Web API design, common authentication methods include HTTP Basic Auth, OAuth2, and account password login. In addition, HMac algorithm-based anti-replay and anti-tampering schemes are used for anti-tampering.
Overlooked Topics
In this discussion, I did not cover the following topics: asynchronous protocols (Web Sockets / Long Pulling / Round Trials), CORS, and security issues. Although these topics are important, they will not be covered in this article.
When to break the rules
Some developers believe that rules are there to be broken. The reality is often so complex that it is difficult to discuss the details. If a developer feels that a rule does not meet the actual need, there are two ways to handle it: change the rule or break it. However, I prefer to discuss and update the rules, clarify where the specification is inadequate, and determine if there are special cases. If a special case does need to be created, be sure to describe it in detail in the documentation, informing the successor and the consumer that it is a special case, explaining why the special case was created and how the special case responds.
Github’s API is one that I often refer to. It models its business domain very clearly, provides thorough documentation, and makes communication much less costly. I mainly refer to the following two links: API Definition GitHub REST API documentation and a list of application-oriented APIs [Endpoints available for GitHub Apps](https://docs.github.com/en/rest/overview/endpoints-available-for-github-apps? apiVersion=2022-11-28) This list includes almost all of Github’s APIs.
Problem | Choice | Comment |
---|---|---|
URL | ||
Dos API path’s prefix | second-level domain | https://api.github.com |
Does the path contain the API version | 🚫 | Header X-GitHub-Api-Version API Versions |
Does path contain a group | 🚫 | |
Whether path contains action | Depends (include if HTTP Verb CRUD cannot be satisfied) | sample: /repos/{owner}/{repo}/pulls/{pull_number}/merge POST /repos/{owner}/{repo}/releases/generate-notes |
Model ID form | Readable Stable Identity | |
Model singular or plural in URL | Plural | |
Is the resource one-level (tiled) or multi-level (nested) | multi-level | |
How is search implemented, standalone API (/models/search) or list-based /models/ API | standalone | |
Does it have an Alias URL | ? | |
Does the model in the URL allow abbreviations (or refinements) | 🚫 | No obvious information is seen, and it is not required based on the multilevel model, but it exists GET /orgs/{org}/actions/required_workflows |
Hyphenation of multiple words of the model in the URL | - and _ | GET /repos/{owner}/{repo}/git/matching-refs/{ref} vs GET /orgs/{org}/actions/required_workflows |
Whether to distinguish between Web API and Open API (non-browser oriented) | 🚫 | |
Header | ||
Are all verbs using POST | 🚫 | |
Is the Modify action POST or PATCH? | PATCH | |
HTTP status return code | Leverage HTTP Status | Commonly used, including rate limiter |
Is using rate limiter system | ✅ 429 | |
Is using cache system | ✅ ETag / Last Modify | Resources in the REST API#client-errors |
Is validate UserAgent | ✅ | |
Is validate Referrral | 🚫 | |
Request | ||
Whether complex parameters are put into Form Fields or a separate JSON Body | Body | for example, Pulls#create-a-pull-request |
Whether the subresource is a one-time query or an independent query | one-time | Judging from Pulls |
Paging parameter storage | URL Query | |
Whether complex parameters are put into Form Fields or a separate JSON Body | Page | Using pagination in the REST API |
Whether the subresource is a one-time query or an independent query | Server | Using pagination in the REST API |
Response | ||
Model presentation types | Multi | For example, the details in Commits and Parent Commits |
How large models contain sub-model models | Core Model + Multiple Linked Resource Queries | Not explicitly stated, inferred from several core APIs |
Whether field returns are on-demand or grouped or unified | Unified | |
field presentation format | Snake | |
Error code | None | Resources in the REST API#client-errors |
Error format | Global Harmonization | Resources in the REST API#client-errors |
Time Zone | Multi(ISO 8601 > Time-Zone Header > User Last > UTC) | Resources in the REST API#Timezones |
Azure’s API design follows [api-guidelines/Guidelines.md at master - microsoft/api-guidelines](https://github.com/Microsoft/api-guidelines/blob/ master/Guidelines.md). This article is schematic, and there is also a practical guide at [Best practices in cloud applications](https://learn.microsoft.com/en-us/azure/architecture/best-practices/index -best-practices) and Web API design best practices.
Note that Azure’s product line is much richer than Github, and some APIs do not follow Azure’s own specifications. When looking for examples, I mainly refer to REST API Browser, the Azure Storage REST API Reference. If the specific implementation conflicts with the Guidelines.md, I will use the Guidelines.md conclusion.
Problem | Choice | Comment |
---|---|---|
URL | ||
Dos API path’s prefix | Second level domain | |
Does the path contain the API version | 🚫 | x-ms-version |
Does path contain a group | ✅ | |
Whether path contains action | 🚫? | Not explicitly stated, but there is a tendency to use the comp parameter for actions that keep the URL RESTful Reference Lease Container (REST API) - Azure Storage |
Model ID form | Readable Stable Identity | Guidelines.md#73-canonical-identifier |
Model singular or plural in URL | Plural | Guidelines.md#93-collection-url-patterns |
Is the resource one-level (tiled) or multi-level (nested) | one-level (tiled) / multi-level (nested) | api-design#define-api-operations-in-terms-of-http-methods, note MS has comp=? parameter, which is used to handle special commands |
How is search implemented, standalone API (/models/search) or list-based /models/ API | ? | Tends to be list-based because of the heavy use of comp= this URL Param for subcommands such as Incremental Copy Blob (REST API) - Azure Storage |
Does it have an Alias URL | ? | |
Does the model in the URL allow abbreviations (or refinements) | ? | |
Hyphenation of multiple words of the model in the URL | Camel | Job Runs - List - REST API (Azure Storage Mover) |
Whether to distinguish between Web API and Open API (non-browser oriented) | 🚫 | |
Header | ||
Are all verbs using POST | 🚫 | |
Is the Modify action POST or PATCH? | PATCH | Agents - Update - REST API (Azure Storage Mover) |
HTTP status return code | Leverage HTTP Status | Guidelines.md#711-http-status-codes |
Is using rate limiter system | ? | |
Is using cache system | ✅ | Guidelines.md#75-standard-request-headers |
Is validate UserAgent | 🚫 | |
Is validate Referrral | 🚫 | |
Request | ||
Whether complex parameters are put into Form Fields or a separate JSON Body | Body | Reference Agents - Create Or Update - REST API (Azure Storage Mover) |
Whether the subresource is a one-time query or an independent query | ? | |
Paging parameter storage | ? | No conclusion |
Whether complex parameters are put into Form Fields or a separate JSON Body | Page based | |
Whether the subresource is a one-time query or an independent query | Server | Agents - Create Or Update - REST API (Azure Storage Mover) |
Response | ||
Model presentation types | Sginle | Speculation |
How large models contain sub-model models | ? | Overly complex scenarios with no single conclusion |
Whether field returns are on-demand or grouped or unified | ? | |
field presentation format | Camel | |
Error code | Use the list of custom error codes | At least within their respective products |
Error format | Customization | |
Time Zone | ? | |
HATEOAS | ? | api-design#use-hateoas-to-enable-navigation-to-related-resources |
The overall design style of Azure is more complex than the Github API, and there are multiple versions of the same product, making it a little less uniform. It looks a little less uniform. It is also more difficult to have a single specification that binds all teams in this complex scenario. We can see that the Azaure team is working on the Guidelines, and they are recently launching the vNext specification.
My personal style is basically inherited from the Github API style, with some minor adjustments to make it more suitable for small and medium-sized product development. The reasons for my changes are explained in the notes, and the starting point for changes is: simplify / reduce ambiguity / consider the actual cost. If “Note” is marked in the notes, it follows the Github scheme and adds some ideas.
Problem | Choice | Comment |
---|---|---|
URL | ||
Dos API path’s prefix | /apis | We often have only one system, one domain name to host the API and Web Page |
Does the path contain the API version | ✅ | |
Does path contain a group | ✅ | Do a layer of business module split to isolate certain cooperation boundaries |
Whether path contains action | Depends (include if HTTP Verb CRUD cannot be satisfied) | |
Model ID form | Readable Stable Identity | |
Model singular or plural in URL | Plural | |
Is the resource one-level (tiled) or multi-level (nested) | Multi-level + One-level | Note: 80% of cases follow the model attribution, a few cases (commonly in search) use the first level |
How is search implemented, standalone API (/models/search) or list-based /models/ API | Uniform > Standalone | Low-cost implementation of some (early Github issues were also without /search interface) |
Does it have an Alias URL | 🚫 | Keep it simple. |
Does the model in the URL allow abbreviations (or refinements) | ✅ | Once the streamlining is done, it needs to be marked in the glossary |
Hyphenation of multiple words of the model in the URL | - | |
Whether to distinguish between Web API and Open API (non-browser oriented) | 🚫 | |
Header | ||
Are all verbs using POST | 🚫 | |
Is the Modify action POST or PATCH? | PATCH | |
HTTP status return code | Leverage HTTP Status | |
Is using rate limiter system | ✅ 429 | |
Is using cache system | 🚫 | Keep it simple, use dynamic data, remove caching capabilities |
Is validate UserAgent | ✅ | |
Is validate Referrral | 🚫 | |
Request | ||
Whether complex parameters are put into Form Fields or a separate JSON Body | Body | |
Whether the subresource is a one-time query or an independent query | one-time | |
Paging parameter storage | URL Query | |
Whether complex parameters are put into Form Fields or a separate JSON Body | Page | |
Whether the subresource is a one-time query or an independent query | Client | Reduce server-side costs and tolerate extreme case gaps |
Response | ||
Model presentation types | Multi | BO / VO / Tiny / Rich |
How large models contain sub-model models | Core Model + Multiple Linked Resource Queries | |
Whether field returns are on-demand or grouped or unified | Unified | Tiny Model (optional) / Model (default) / Rich Model (optional) |
field presentation format | Snake | |
Error code | None | Note: Many scenarios require only message |
Error format | Uniform | |
Time Zone | ISO 8601 | Use only one format, no longer support multiple options |
HATEOAS | 🚫 |
image from Apple Music
I was recently using Apple Music and noticed the URL structure of its web page:
/cn/album/we-sing-we-dance-we-steal-things/277635758?l=en
Looking closely at this URL structure, I can see that the Path contains human-readable slug in three parts: alumn/$(name)/$(id)
(which contains the ID).
A question immediately occurred to me: are the intermediate readable names machine-sensitive and purely geared towards natural humans?
So I tested a fabricated address: /cn/album/foobar/277635758?l=en
.
Can you guess if the result is accessible before you try to access it?
This design paradigm is a bit more complex than my current common URL design specification. My specification requires that the resource locale be organized using a two-level slug, $(type)/$(id)
.
Whereas Apple uses $(type)/(type-id)/$(id)
, taking care of both readability and accuracy.
GraphQL is a way to request APIs by using a custom query language, which has the advantage of providing a more flexible way to fetch data. Compared to RESTful APIs that require a single request for all the data needed, GraphQL allows the client to explicitly specify the data needed, thus reducing unnecessary data transfer and processing.
However, GraphQL’s over-flexibility is also one of its drawbacks. Since it does not have some specifications for modeling business scenarios like the REST API, developers need to think about data processing themselves. developers need to think about how to handle the data themselves. This can lead to unreasonable query requests that can put excessive pressure on the back-end database. In addition, GraphQL has relatively few implementations and documentation, which also requires more learning costs.
Therefore, while GraphQL can provide better results in some specific scenarios, it is not suitable for all API design needs. In fact, some companies have even chosen to drop support for GraphQL, such as Github’s some projects.
Complexity is incremental
- John Ousterhout (via)
There is no best style, only the best fit, but it is important to have style. </mark
Building a good rule requires not only a deep understanding of existing mechanisms, but also a thorough grasp of the business domain, and effective collaboration and communication within the team to promote and implement the rule. Once established, however, rules can effectively reduce the complexity of the system, avoiding increasing complexity over time and as the business progresses, and reducing communication costs on the R&D side.
It’s a long-term investment, but one that will pay off in the long run. I hope those with a long-term view will take note of this article.
Key reference documents:
Giving up an old friend like Toodledo, which I have been working with for more than a decade, made me feel a little sad, but the process also prompted me to summarize my attempts at time management and share some of my experiences.
Image frrom Pixelbay
In this fast-paced modern life, we all encounter a large amount of tasks and information, and how to effectively manage time and tasks has become a challenge for many people. During my school days, I also struggled with task management. Starting from 2009, I tried to use tools such as [calendars](https://blog.alswl.com/2010/01/use-google-calendar-to- manage-the-time/) and notes to organize my tasks and information, but the effect was not ideal. It wasn’t until later that I came across DoIt.im and discovered David Allen’s book, “Getting Things Done”, which brought me a new approach to task management - GTD. For decades since then, I have been using this method to manage my time.
GTD is a personal time management method created by David Allen, aimed at helping people manage their time and tasks more effectively, thereby improving work efficiency and productivity. The basic idea of this method is to break down all tasks into specific actionable steps and organize them in a reliable system for follow-up and management</ mark>. Through this method, people can more easily take control of their work and life, reduce stress and anxiety, and thus complete tasks more efficiently with greater focus.
So how do we implement GTD? Here are the basic steps:
Note: There may be slight differences in structure and external tools I organized. I merged categorizing and reviewing into the organizing step because I found that organizing is essentially planning, and planning involves reviewing the collected to-do items as well as ongoing and completed projects. Why not do them together in the organizing step?
I have used several online task management software, including Doit.im, Remember The Milk, OmniFocus and Toodledo. Below is my usage history and comparison of these software.
I started with Doit.im when I first got into GTD. It is a domestic software that I really like. Doit.im supports basic GTD functions, provides clear task lists, classification, priority and other functions, as well as multiple review modes and multi-platform synchronization. However, I later gave it up because I found the more powerful Toodledo.
Image from doit.im
When looking for the next management tool to use after Doit.im, I found Remember The Milk (RTM). RTM provides basic task management functions such as adding tasks, setting reminders and archiving tasks etc., and supports multi-platform synchronization so you can use it on multiple devices such as computers or phones.
However some people may not like RTM’s user interface design because it looks outdated compared to modern designs. In addition some advanced features require payment which was difficult for me at the time since I was still a poor student so I quickly stopped using RTM.
I learned about Toodledo from xbeta.info/gtd-toodledo.htm (a famous website where users share their experiences with different softwares).
Image from toodeldo.com
Soon after that on Productive Life , I found a very complete tutorial introducing advanced techniques in using Toodeldo especially how to use its Search module effectively . Unfortunately this website cannot be accessed anymore but we can get an idea of what Toodleo looked like back then through screenshots taken from other sources:
Image frrom twitter.com/productivelife
Toodleodo quickly became my favorite task management tool due to its comprehensive functionality supporting task classification sorting filtering etc., while also providing various review modes and custom fields. Additionally,Todoedo supports multi-platform synchronization along with a powerful API making integration with other applications easy.
Since Todoedo had slow access speeds without much improvement in product functionality over time,I briefly explored usingOmniFocus, which is a professional Mac platform task management tool offering features such as project decomposition context etc.,and can be extended via AppleScript among others.OmniFocus supports multiple sync services including Omni Sync Server WebDAV FTP etc., allowing users to choose different services according to their needs.All data are stored locally so users can continue using the software even without internet connection.
After paying for it for some time,I realized that OmniFocu’s search function could not compare with Todoedo’s Search function ,so it couldn’t meet my needs.
Image frrom www.omnigroup.com/Omnifocus
I have also tried out some other tools including Trello Asana Basement Tower Teambition etc., but since they were designed mainly for team collaboration rather than personal use,it wasn’t convenient enough for me.Additionally,the customized search feature of these softwares (at that time) was often weak unable to meet my requirements.In the end,I returned back into the arms of Todoedo until now.
I think in the GTD system, the collection and processing stages are relatively mechanical and simple. The real test for people is the organization stage:
Below, I will share some of my personal concepts and operational methods.
In order to better organize tasks, we need to pay attention to important attributes of tasks, including Context (environment), Project, Status, and Due. Context determines what needs to be done in the current environment and serves as a simple filter. I divide Context into Computer/Work/Mobile. Project assigns tasks to different projects with longer cycles and strong relevance among its tasks. Status describes whether a task can be processed immediately (Next) or requires waiting (Waiting). Some recurring tasks are labeled Active. Due describes time-related attributes such as On-time delivery (On), before this time (Due), or optional completion if possible.
Other attributes such as Priority and Folder type also need consideration but are not most important.
To better manage my daily workloads, every morning I open four views: single view, subtask view, project-xxx view,and inbox view.
The following filtering list helps identify which immediate actions should take place:
Tasks on this list are ranked based on their overall priority level combined with deadline dates & execution status considerations. Every time you enter work mode after reviewing your lists above just select top few items from this filtered list for execution.
I’ve used it for over twelve years now paying eight years’ subscription fees while its product philosophy deeply influenced my own strategies for managing time effectively.I’ll introduce some features/details about Toodledo that worked quite well:
Toodledo supports recurring task functionality allowing users set up repeating periods like daily / weekly / monthly etc., so when a completed item reappears automatically in your active list ready for further processing.This feature helps users manage routine activities more efficiently like regular meetings,daily email handling,daily reviews etc..
Search function provided by Toodledo is very powerful supporting complex logical queries(offers conditional combinations of AND/OR searches).Users could search based on multiple criteria including name,label,date,priority etc..This feature helps users organize their workload more efficiently.I was highly dependent upon this feature during organization phase hence couldn’t leave Toodledo at one point.
Overall, I would still recommend Toodledo to everyone. I have developed multiple tools and plugins based on the API provided by Toodledo. Let me introduce them to you:
Toodledo provides calendar subscription capability that displays tasks in various calendar software. However, the original information is very messy, does not consider task time, and also calculates time incorrectly. Therefore, I developed a web service called toodledo_calendar_filter.
It can display Toodledo’s tasks more friendly on the calendar system. Specifically, it can filter Toodledo’s iCal and only display tasks with due dates and durations. If you also have similar needs, you can use my online service directly (https://toodledo-calendar-filter.alswl.com).
Before:
After:
When using Toodeldo for task management, do you feel that the operation is cumbersome and mouse efficiency is low? Do you miss Gmail-style shortcut keys? If so, then my project [alswl/my-toodleodo] (https://github.com/alswl/my-toodleodo) can help solve this problem.
It is a Greasemonkey script for Toodleodo that provides a series of Gmail-style shortcut keys so that you can quickly operate like using Gmail in Toodleodo including moving tasks (j
/ k
), task operations (x
marks completion , enter
starts the task), switching task views (g s
jumps to search), etc.. It runs stably on Chrome/Safari/Firefox browsers; if you want to improve your efficiency when using Toodeldo,you may try it out.
In order to quickly collect tasks,I once used a CLI tool ([wsargent/toodeldo] (https://github.com/wsargent/toodeldo)) on Github。However,it was based on version 1 of API which has been announced by official seven or eight years ago will be offline soon。So I made my own small project called go-toodeldo .
At first,go-toodeoldo only provided basic CLI functions。Later,I gradually added complete SDK、CLI full functionality even an interactive TUI application allowing users to conveniently use Toodeoldo in Terminal。
Toodeoldo does not provide OpenAPI interface officially;I generated a reverse-engineered copy of [go-TOODEOLDO/swagger.yaml] (Https:// github .com / alswl / go-TOODEOLDO/blob/master/api/swagger.yaml),if anyone else has development requirements for TOODEOLDO API ,they could directly use this API。
Recently,I have become increasingly disappointed with the stability of Toodeldo -the task management service I’ve been using all along。 They[changed their new service provider](Https:// blog .tooleldo .com/hello-from-the-new-Tooleldo-team/), and there are more frequent occurrences of unstable services while new team’s product capabilities also raise some questions. Therefore,I decided to start looking for better task management services。 Because Kindle stopping its services in China event makes me doubt about cloud services’ reliability。 Therefore,I started looking for somelocalized solutionsthat could be serviced through network disks(such as iCloud), so as to better protect my data privacy.
Obsidian is a personal knowledge management (KMS) and note-taking application based on local files, which helps users organize, link and analyze their notes and ideas. It is a purely local software that can be synchronized across multiple devices using methods such as network disks/NAS.
Obsidian Tasks is a powerful Obsidian plugin that allows you to easily manage tasks and to-do items in Obsidian. This plugin can add tasks to your notes through simple syntax, automatically mark the status of tasks according to their completion status. You can use Obsidian Tasks to track personal tasks, work tasks, study tasks, etc. In addition, it also has some other features such as custom task styles, creating filters to find tasks, and automatically moving tasks based on their status. Obsidian + Obsidian Tasks are a perfect combination that can help you better manage your to-do list and improve your productivity.
Image from https://github.com/obsidian-tasks-group/obsidia
My personal KMS has been fully migrated from [Notion] (http://notion.com/)to Obisidan with low migration costs using Obisidan Tasks. After learning about Obsidain Task for some time,I have smoothly transferred my previous usage patterns of “organizing"and “next task"to Obisidan Task.I will continue introducing how I efficiently use Obisidan Task in these two scenarios.
I organize my pending task in the following ways:
[ ]
, usually small scattered ones..todo.md
for overall project coordination.#todo
, usually big issues where I split out several Tasks
at the top after splitting off the TODO
queue tag.Based on this organizational structure,I easily handle these two scenarios: organizingand next task:
Use several viewsto present the recorded tasks in multiple views, including:
.todo
#todo
, often requiring processing of an entire documentThese views allow you to classify your tasks better and quickly review them.By using these views,you can better manage your pending items,and improve your productivity.
This is my projects / View.todo.md`:
## Today
**WIP**
```tasks
status.type is IN_PROGRESS
```
**High(Planning)**
```tasks
not done
priority is above medium
```
**None repetead due today**
```tasks
((not done) AND (due before in 1 day)) OR (done on today)
is not recurring
sort by priority
```
**Repeated due today**
```tasks
((not done) AND (due before in 1 day)) OR (done on today)
is recurring
sort by priority
```
**Over due before today**
```tasks
not done
due before today
sort by priority
```
**Today complete**
```tasks
done
done on today
```
## Future (no repeat)
**in 1 day**
```tasks
not done
due before in 1 day
is not recurring
sort by priority
sort by due
```
**in 3 day**
```tasks
not done
due after in 1 day
due before in 3 day
is not recurring
sort by priority
sort by due
```
**in 7 day**
```tasks
not done
due after in 3 day
due before in 7 day
is not recurring
sort by priority
sort by due
```
## Singles Tasks
**in 7d(non project x no repeate)**
```tasks
not done
status.type is not CANCELLED
NOT (path includes .index)
NOT (path includes .todo)
NOT (path includes .notodo)
due after in 7 days
sort by priority, due
```
**none project x no due x no repeat**
```tasks
not done
status.type is not CANCELLED
NOT (path includes .index)
NOT (path includes .todo)
NOT (path includes .notodo)
no due date
sort by priority, due
```
## Projects
TODO
## Help
> [Queries Syntax](https://obsidian-tasks-group.github.io/obsidian-tasks/queries/)
In Obsidian’s Daily Note plugin, set the template to _templates/daily
so that you can have a daily to-do list that you can use anytime.
This is my Daily setup, divided into four sections: Today’s due single tasks / Today’s due repeating tasks (transactional and unimportant) / Today’s completed tasks / Today’s new tasks (usually Single).
This is my _template/daily.md
file:
## TODO
**WIP**
```tasks
status.type is IN_PROGRESS
```
**New tasks**
**Tasks view - today**
```tasks
((not done) AND (due before {{date}})) OR ((not done) AND (due on {{date}})) OR (done on {{date}})
is not recurring
sort by priority, due
```
**Tasks view - today(repeated)**
```tasks
((not done) AND (due before {{date}})) OR ((not done) AND (due on {{date}})) OR (done on {{date}})
is recurring
sort by priority, due
```
**Over due date**
```tasks
not done
due before {{date}}
sort by priority, due
```
**Projects**
YOURS
**Tasks view - today done**
```tasks
done
done on {{date}}
```
Firstly, it should be noted that Obsidian Tasks is a plugin for Obsidian. If you have not used Obsidian before, it may take some time to get familiar with it. However, if you are already using Obsidian, the bidirectional linking feature of Obsidian Tasks will provide excellent support for GTD scenarios, which is both an advantage and a disadvantage.
Secondly, at present, Obsidian Tasks does not offer timer functionality. However, personally speaking, this is no longer important to me. With age and over ten years of GTD training under my belt, I no longer need tools to maintain focus. When working I am quite focused and do not get distracted frequently.
Finally, it should be noted that there are issues with Obsidian itself. As a tool based on local files only without online tools (although there are other cloud-based solutions), if you want an online service accessible from both your computer and phone then perhaps Obsidian Tasks may not be suitable for you. For me personally though I use iCloud storage as part of Apple’s ecosystem so all my data is stored in iCloud Drive; I use the macOS application and iOS version of Obsidian which allows me to easily use the same Vault (i.e., knowledge base) between my MacBook and iPhone. Before launching either app though I make sure that syncing has completed first.
Time management is an unavoidable topic for modern workers but also one we often struggle with too. The methodology behind GTD provides us with an easy yet efficient way to manage our workloads allowing us to quickly end procrastination or distraction statesand fully immerse ourselves in our work.
Tools are important but they cannot solve all problems alone; when managing time effectively what matters most is being clear about what your goals actually are. When you have a clear mission in mind it will constantly float around in your head meaning even without GTD tools managing things becomes easier.
I hope this article introducing the GTD method can help bring benefits into your work life allowing you to complete tasks more efficiently while still enjoying life’s beauty too! Remember: time is limited so cherish every minute; seize every opportunity making your own life more wonderful!
]]>In order to be well prepared, how can I successfully join an electronics factory after graduating at the age of 35?
Learn from fixing keyboards(it’s a joke)
The keyboard I using is the ErgoDox, an ergonomically split keyboard. More details about ErgoDox can be found in my previous answer.
(former keycap color scheme + hand rest).
After seven or eight years of work, it has been into the water, into the coffee, but also into the soy milk, now finally a few keys are not flexible, pressed a sticky feeling, can not provide a smooth coding feel.
After a few months of using a back up Filco, I finally made up my mind to get the ErgoDox fixed.
For those without soldering experience, you can learn how to soldering.
I have that feeling of typing smoothly again ~
Youthfulness is back~
Finally, I’d like to share my ErgoDox Layout configuration:
If you get something out of it, give it a like.
]]>First of all, I’d like to thank my in-laws for taking the baby for summer vacation, so I can have a weekend to develop this system.
Also thanks to my wife, I did my own thing for two days during the weekend and didn’t criticize me.
Lastly, I would like to thank the company for the team building, providing a hotel to stay and spending a night to fix the Chinese handwriting.
Note: This is the open source version of Excalidraw, thanks to the community
Introduce Excalidraw, product features
What is Excalidraw, it’s (probably) the most powerful online collaborative drawing tool, you can try it by visiting Excalidraw.
If you did not want get a blank diagram, then you can also visit this public panel Excalidraw to join in the creation.
Excalidraw works very well, I summarize a few points.
After this product was promoted internally by our internal partners, we quickly fell in love with it and put a lot of diagrams on Excalidraw, but it also led to a new problem.
How Excalidraw works and how self hosted works:
So what is the core difficulty of self hosted.
Solving the excalidraw-storage data storage problem, aka. replacing the Google Cloud Platform’s firebase service.
If things don’t work out, ask yourself
- Mencius
It is better to seek from Github than to seek from yourself.
- alswl
Let’s start by looking at Excalidraw’s storage system. Firebase is Google’s Serverless service, formerly an independent company (and quite popular), but later acquired by GCP.
I started with the idea of replacing Firebase and found a community service Supabase (found a free SaaS service, amazing)
However, after careful research, I found that Supabase’s API is not compatible with firebase and cannot be simply replaced with .
Excalidraw also relies on Excalidraw+ to provide a paid service. Is there a simple and viable solution from the community? I found a bit of a solution within the community:
They gave a way (although it was later proven that there was still a crooked way).
So I started reviewing their solution, opened the code and saw that I had the right idea (replacing several interfaces of firebase), using my own KV storage (Redis / MySQL / Mongo) to replace it.
Attempted deployment and immediately ran into problems with:
What a coincidence that I’m Frontend Intern // Community Patching Specialist // Kubernetes Cleaner // YAML Senior Specialist , specializing in these few things.
Excalidraw is a free version of the SaaS service, Excalidraw+ is a paid version , what’s the difference?
To sum up.
Or, just buy Excalidraw enterprise version!!! Less works, more enjoy.
One problem is that Chinese fonts are not handwritten , which is very incongruous. Let’s see which fonts work first.
Chinese handwriting fonts supported by macOS:
cursive family:
Windows Chinese support for handwriting is poor, you have to install Office to have more options: 华文行楷; 方正舒体 (release in Office)
If not, only the system’s own italic KaiTi will work.
Finally, relying on my poor front-end level, I made a PoC and submitted a PR to the official: [feat: simple impl of multi font support, for chinese font by alswl - Pull Request #5604 - excalidraw/excalidraw] (https://github.com/excalidraw/excalidraw/pull/5604)
Here is the final result.
A few users came to me to ask how to deploy it. So based on the above solution, I provided a set of one-click pull-up services based on Docker Compose: with collaboration, Chinese font support. See alswl/excalidraw-collaboration.
]]>Static typing is becoming more and more popular. Several languages born after 2010, Go, Rust, TypeScript, etc., have gone the static typing route. Some of the previously popular dynamic languages (Python, PHP, JavaScript) are also actively introducing new language features (Type Hint, TypeScript) to enhance static typing.
Having worked on larger projects in Python, I have experienced the difficulties that dynamic languages pose when projects get large: The cost of code regression during the refactoring phase was unusually high, and much of the historical code was afraid to move. Then the technology stack shifted to Java, and being embraced by the type system made people feel safe.
In the last year, I have been working on a stability-oriented operation and maintenance system. Python was used at the beginning of the system selection. I pushed Python 3.7 in the project and used Python’s type system on a large scale to mitigate potential risks.
Going back to my roots, I spent some time learning about Python’s design and implementation of the type system. This article is a PEP proposal that describes how far Python has come with the type system.
Before we talk about type systems, we need to clarify two concepts, dynamic languages and dynamic types.
A dynamic programming language is a program that can change its structure at runtime. This structure can include functions, objects, variable types, and program structures. Dynamic types are one type of Type System, i.e., programs that can modify variable types during runtime. The other type is the static type: the variable type is determined at compile time and is not allowed to change at runtime. Another division of the type system is strong and weak types, where strong types are instructions that forbid type mismatches and weak types vice versa.
The two concepts, dynamic languages and dynamic types, have different entry points. Python is a dynamic language and a dynamically typed language, or a strongly typed dynamic type. This article will focus on the type system of the Python language and will not cover dynamic language features.
There is an ongoing debate within the industry about which is more powerful: dynamic or static types. Proponents of static types see advantages in three areas: performance, error discovery, and efficient refactoring. Static types can significantly improve runtime efficiency by determining the concrete type at compile time; The ability to detect errors at compile time, especially when projects get progressively larger; The type system can help the IDE to prompt for efficient refactoring. Proponents of dynamic typing argue that analyzing code is simpler, less error-prone, and faster to write.
Python developers have not failed to see this pain point. A series of PEP proposals were created. The type system was introduced to Python through syntax and feature enhancements while retaining the benefits of Python’s dynamic type system.
Python proposed PEP 484 in 2014, followed by a refined version of PEP 483 (The Theory of Type Hints). Its project implementation, the typing module, was released in 3.5. After several iterations of PEP 484, PEP 526, PEP 544, PEP 586, PEP 589, and PEP 591, Python’s type system has become very rich. It even includes relatively rare features like Structural Subtyping and Literal Typing.
Released in December 2014, PEP 483 This is a concise version of the core concepts Guido wrote, making clear the direction, boundaries, do’s and don’ts of building a type system for Python.
PEP 483 does not talk about concrete engineering implementations, but rather outlines how the Python type system is presented to the public.
Define the Type/Class difference, the former being a syntactic analysis concept and the latter a runtime concept.
Under this definition, a Class is a Type, but a Type is not necessarily a Class.
For example, Union[str, int]
is a Type but not a Class.
PEP 483 also introduces built-in base types: Any
/ Unison
/ Optional
/ Tuple
/ Callable
, which support rich upstream variations.
The biggest criticism of the static type system is that it is not flexible enough, and the Go language does not implement generics yet. PEP 483 introduces the use of Python Generic types in a generic The form is as follows:
S = TypeVar('S', str, bytes)
def longest(first: S, second: S) -> S:
return first if len(first) >= len(second) else second
Finally, PEP 483 also mentions some important minor features:
The implementation of PEP 483 relies heavily on PEP 3107 – Function Annotations.
This proposal. PEP 3107 introduces the use of function annotations. For example, func(a: a1, b: b1) -> r1
This code, thewhere the descriptors after the colon are recorded in the __annotations__
variable of func.
PEP 3107 shows the effect as follows, where you can clearly see that the function variables are stored:
def add(x: int, y: int) -> int:
return x + y
add.__annotations__
# {'x': int, 'y': int, 'return': int}
PS: Python now has Decorator decorators / Annotation annotations, where Annotation is also designed with the same name as Java’s Annotation, a potpourri.
PEP 484 – Type Hints A complete description of how the Python type system is designed, how it is used, and what the details are (typing modules), building on PEP 483
This proposal begins by stating:
Python will remain a dynamically typed language, and the authors have no desire to ever make type hints mandatory, even by convention.
In one sentence, the proposal eliminates the possibility of Python evolving to a static system at the language level.
In addition to the features already explained in PEP 483, the proposal also appeals to me in the following ways:
.pyi
file corresponding to the Python file.
This scheme is similar to TS’s @types
file.@overload
is a long time coming, and Python can actually (in a sense) support overloading.Sized
/ Iterable
base interfaces.
Personally, I think this is actually quite a lot of work, and is a way to sort out the dependencies of existing classes.@typehints(foo=str, returns=str)
), comments, Stub files, DocstringPEP 526 - Syntax for Variable Annotations The core proposal is to add Type Hints support to variables.
Similar to function annotation
, it is also stored by annotation.
The difference is that instead of adding a __annotations__
member to the instance, the annotations information for the variable is stored in the context variable __annotations__
.
This is actually quite understandable: when a variable type is defined, the variable is not yet initialized.
I’ll write a demo to demonstrate this:
from typing import List
users: List[int]
# print(__annotations__)
# {'users': typing.List[int]}
As you can see, the above demo has the effect of creating a users
in the context variable, but this users
doesn’t actually exist, it just defines the type.
If you run print(users)
it will throw NameError: name 'users' is not defined
.
It is clearer to look at the bytecode:
L. 1 0 SETUP_ANNOTATIONS
L. 1 2 LOAD_CONST 0
4 LOAD_CONST ('List',)
6 IMPORT_NAME typing
8 IMPORT_FROM List
10 STORE_NAME List
12 POP_TOP
L. 3 14 LOAD_NAME List
16 LOAD_NAME int
18 BINARY_SUBSCR
20 LOAD_NAME __annotations__
22 LOAD_STR 'users'
24 STORE_SUBSCR
26 LOAD_CONST None
28 RETURN_VALUE
As you can clearly see, instead of creating a variable named users, the __annotations__
variable is used.
Note: Python stores variables using the opcode STORE_NAME
.
PS: There are a number of rejected proposals in this proposal, which is quite interesting, and the community has come up with a lot of strange and clever ideas. You can see the caution of the community decision, the difficulty of upgrading the stock system.
The type system discussed in PEP 484 is Nominal Subtyping. This PEP 544 – Protocols: Structural subtyping (static duck typing) This PEP 544 – Protocols: Structural subtyping (static duck typing) is proposed for Structural subtyping.If I had to translate it, I think it could be called named subtypes / isomorphic subtypes. Note that some people also refer to Structural Subtyping as Duck Typing, but the two are not the same, see Duck typing / Comparison with other type systems.
Nominal Subtyping refers to matching types literally, while Structural Subtyping matches by structure (behavior). For example, Go’s Type is a Structural Subtyping implementation.
Here is a simple demo to demonstrate the latter:
from typing import Iterator, Iterable
class Bucket:
...
def __len__(self) -> int: ...
def __iter__(self) -> Iterator[int]: ...
def collect(items: Iterable[int]) -> int: ...
result: int = collect(Bucket()) # Passes type check
The code defines the type Bucket and provides two class members. These two class members happen to be the definition of Interator.Then in practice, you can use Bucket instead of Iterable.
PEP 586 – Literal Types
The Python 3.8 implementation supports the use of literals as types.
For example Literal[4]
, and for a more semantic example Literal['GREEN']
.
My first thought was that this is very similar to Symbol in Scala, which is written as Symbol("GREEN")
.
This feature is quite collegial and easy to write in the DSL.Scala officials have said that the Symbol feature may be removed in the future, so it is recommended to use constants instead.
PEP 589 – TypedDict: Type Hints for Dictionaries with a Fixed Set of Keys
Add a Type of key to Dict, inheriting from TypedDict
.
PEP 591 – Adding a final qualifier to typing
Add the concepts of final
/ Final
, the former is a decorator, the latter is an annotation that marks the class/function/variable as unmodifiable.
At this point, Python 3.8 has the type system features we need every day (not at runtime 😂).
Unfortunately, the typing
module is clearly marked in the documentation as:
The Python runtime does not enforce function and variable type annotations. They can be used by third party tools such as type checkers, IDEs, linters, etc.
Namely, the Python runtime (Intercepter / Code Evaluator) does not support type decorators for functions and variables. These decorators can only be checked by third-party tools, such as type checkers, IDEs, statics, and Linter.
This information illustrates the limitations of Python’s attempts at type safety. All the restrictions, constraints, and limitations don’t happen at runtime. The only way to harvest engineering-above-board value from the type system is to use third-party tools.
It’s true that the Python community is trying to move closer to a type system, but how far can this non-language level Runtime support go? Python lacks a golden father, and its godfather Red Hat has limited resources to invest. Even the community hasn’t finished switching from Python 2 to Python 3 yet, so why? The input/output ratio is too low, the new features are not attractive enough, and there are too many alternatives.
On the other hand, look at the competition: Dynamic languages are moving closer to static languages, and static languages are absorbing features from dynamic languages. For example, the REPL (Read-Eval-Print-Loop) in Java 14. Kotlin / Scala and other languages Type Inference. Perhaps this evolution is more acceptable to users.