Updates to the API surface (and a re-commitment to semver!)

[julian]

Hello from Banff, Alberta! I'm taking a short working vacation here for the week while @baris is holding down the fort — thanks @baris!

Today I wanted to publish an update on what we consider our public API surface, as part of our move to semver. Prior to v2.0.0 of NodeBB, we were operating on a "semver-like" system, where breaking changes were reserved for minor releases (e.g. v1.7.5 to v1.8.0). All other changes were made to patch version, including new functionality. The major version never changed.

We made the change to adopt semver once v2.0.0 landed:

• Patch releases
  • will not be constrained to a specific day, and can occur at any time
  • will occur much more frequently than before
  • will contain only bug fixes
  • will contain fewer (if any) upgrade side-effects
• Minor releases
  • will continue to be restricted to Wednesdays
  • will contain new features and backwards-compatible changes to existing features
  • will contain deprecations to existing features (for removal in the next major version)
• Major releases
  • will be restricted to Wednesdays
  • will contain breaking changes and removal of features (that were deprecated in an earlier minor version release)

One of the core tenets of semver is the declaration of a public API. Essentially, what parts of the application can we commit to remaining consistent, with an implicit promise to not change until a major version?

Since v2, that has always been the Read API and Write API. We have committed to this by integrating our test suite to ensure that the maintained OpenAPI spec matches both APIs.

You may have noticed that by considering only the Read and Write APIs part of the public API surface, that we are only catering to users integrating with NodeBB externally (e.g. via API call), and you'd be right! There was no such guarantee that the API would not change internally (e.g. for plugin developers).

Therefore, as of v3, we are re-confirming our commitment to semver by expanding our public API surface to also include all methods contained in the src/api folder.

What is the src/api folder?

Originally, the methods in this folder were a common place for logic to reside when called by both the Write API and socket.io. We implemented this structure so that we wouldn't repeat ourselves, but also to reduce the risk that the write API would drift apart from socket.io over time.

As of this refactor, and in conjunction with the removal of deprecated socket.io methods in v3, the methods in this folder could be re-purposed.

Methods contained in this folder mirror the write API endpoint-for-endpoint. Therefore, any functionality available via the Write API would also be available to plugin developers via an internal interface.

What does this mean for plugin developers?

Essentially, it means that our recommendation is to use methods in src/api if you wish to interface with NodeBB with a guarantee that your plugin won't break until the next major version. You are always welcome to use other internal methods in src/ directly, or even interface with the database library directly, but those do not come with any implicit guarantees.

For example, you can update a flag by calling the flags library directly:

const flags = require.main.require('./src/flags');

await flags.update(flagId, uid, { state: 'wip' });

Using this library method, there would be no guarantee that we wouldn't later update this internal method to remove the second parameter (uid), and integrate it into the third parameter.

However, when using the flags methods...

const api = require.main.require('./src/api');

await api.flags.update(caller, {
    flagId,
    state: 'wip',
});

... you can rely on the fact that neither the function signature (nor the resulting output) will change until at least the next major version of NodeBB.

Happy hacking!

Quick FAQ

What of the methods in src/controllers/write?

By and large, the function of the methods in src/controllers/write are now to act as an external API surface. It should only consume API parameters and request bodies, and send back the resulting data in a standardized format (an object containing status and response), including a proper HTTP status code.

What is caller?

caller is one of either req (provided by express inside of a route controller), or the socket instance (provided by NodeBB when acting as a socket.io listener method). If you have neither, you can fake it by passing in { uid }, where uid is the calling user's uid. Occasionally, some API methods will check other properties inside of caller, but the only one really required is uid.

[brazzerstop]

@julian said in Updates to the API surface (and a re-commitment to semver!):

Patch releases
will not be constrained to a specific day, and can occur at any time
will occur much more frequently than before
will contain only bug fixes
will contain fewer (if any) upgrade side-effects
Minor releases
will continue to be restricted to Wednesdays
will contain new features and backwards-compatible changes to existing features
will contain deprecations to existing features (for removal in the next major version)

I correctly understand that these changes simplify the life of the developers and now plugins will work longer, without crash if forum upgraded to the releases Patch and Minor?

[julian]

That is correct. A plugin should work with no breakages as long as they do not upgrade to the next major version level.

NodeBB forum admins are still advised to use the recommended plugin version in the ACP, as there is no guarantee that plugins will follow semver.