NodeBB Installation as Module(s)
-
Wouldn't it make sense for NodeBB to be installed as one or many node modules? This would allow for:
- plugins to require('nodebb/src/...') instead of require.main.require('./src/...') which in turn would open way to much easier unit testing for plugins,
- run multiple instances without copying the entire source code directory,
- be more in line with how Node apps are installed generally.
Opinions?
-
NodeBB is an application that connects to a database and serves a website on a port. It's not a module.
We have been wanting to make API stuff easier to work with though.
-
@pitaj Right but there is a certain way of installing applications in the Node ecosystem. For example, the binaries end up in node_modules/.bin from where they can be launched. Typically, a configuration file passed to the binary would also contain a home directory for the given instance of the application. This is how this should be done for the sake of convention, as well as flexibility.
-
@sadaszewski simple CLI programs tend to install that way, yes. Can you give an example of a large application that works that way?
-
@pitaj all applications in Un*x-like systems work that way - separate binaries / immutable resources / working areas. the idea to organize web apps as a mixed directory of everything stems from PHP times. back in the 90s, the HTTP requests, comically, used to map to PHP files in that directory structure. For a modern project in a technology that has nothing to do with the silly idea above, it is just awkward to do the same thing. Another example that comes to mind is perhaps the apps in Windows that come prepackaged with everything since they do not compose nicely with the rest of the packages in the OS. I do not think those are examples worth following especially that Node/NPM provide all the needed tools to keep it modular. Ironically, NodeBB makes massive use of that approach, pulling in thousands of packages of which ALL follow the convention, apart from NodeBB
Are there any good reasons it is beneficial to keep it the way it is now?
-
I meant node applications that are distributed in the way you're describing.
Ironically, NodeBB makes massive use of that approach, pulling in thousands of packages of which ALL follow the convention, apart from NodeBB
All of which are libraries, not applications that connect to a database or serve web content.
One reason that installing via npm is incompatible with NodeBB is that we support a plugin system that installs via npm. This requires modifying the package.json in nodebb, adding whatever plugins are installed to the dependencies list there. That package.json would be overwritten any time someone upgrades via npm, removing any installed plugins.
-
@pitaj said in NodeBB Installation as Module(s):
One reason that installing via npm is incompatible with NodeBB is that we support a plugin system that installs via npm. This requires modifying the package.json in nodebb, adding whatever plugins are installed to the dependencies list there.
This is true, however two solutions come to mind:
- npm (node_modules) is hierarchical if I am not mistaken. One could have NodeBB with its dependencies installed in one directory and all the plugins on top of that installed in a subdirectory. Installing plugins would modify the package.json only in the subdirectory.
- Keep plugins management outside of NodeBB, let npm do it. This would simplify NodeBB's code base and somehow feels more admin-y to my taste.
That package.json would be overwritten any time someone upgrades via npm, removing any installed plugins.
I am not sure what you mean by that but I am pretty sure that the solutions above would not suffer from this effect.
-
A subdirectory wouldn't work, it would have to be in a separate directory set by the user. Otherwise it would be overwritten any time the user upgraded nodebb via npm.
Regardless, I don't see much benefit in entirely rearchitecting that portion of the system to support installing this way.
-
@pitaj said in NodeBB Installation as Module(s):
A subdirectory wouldn't work, it would have to be in a separate directory set by the user. Otherwise it would be overwritten any time the user upgraded nodebb via npm.
Regardless, I don't see much benefit in entirely rearchitecting that portion of the system to support installing this way.
I am not entirely convinced about the whole overwriting story and even if that was the case, what would prevent simply reinstalling the plugins as well?
Sorry to point out the benefits again but I mentioned these in the OP:
This would allow for:
- plugins to require('nodebb/src/...') instead of require.main.require('./src/...') which in turn would open way to proper unit testing for plugins,
1a. Either the way plugins include NodeBB modules is a smell or the way NodeBB is installed. both cannot be right simply because it is very clumsy to write unit tests for plugins - this cannot be OK. - run multiple instances without copying the entire source code directory,
- be more in line with how Node apps are installed generally
- plugins to require('nodebb/src/...') instead of require.main.require('./src/...') which in turn would open way to proper unit testing for plugins,
-
I am not entirely convinced about the whole overwriting story
npm deletes the old version of the module before installing the new module version. So any changes you make inside a module directory that is managed by npm will be deleted.
even if that was the case, what would prevent simply reinstalling the plugins as well?
The package.json defining installed plugins would be deleted.
plugins to require('nodebb/src/...') instead of require.main.require('./src/...') which in turn would open way to proper unit testing for plugins
Unit tests shouldn't need to connect to NodeBB, as units are small pieces of code that can be isolated from the rest, including NodeBB. You can set up your unit tests however you want for your plugins.
As for higher level tests:
Again, NodeBB requires a connection to a database, even while testing. Your plugin test harness would still need to initialize the database as they do now.
run multiple instances without copying the entire source code directory
That would be a nice result of rearchitecting, except then every NodeBB instance must be at the exact same version.
be more in line with how Node apps are installed generally
Again, I'm unaware of any node applications like NodeBB, that must connect to a database to function, that work the way you describe. Please provide an example.
-
@pitaj said in NodeBB Installation as Module(s):
I am not entirely convinced about the whole overwriting story
npm deletes the old version of the module before installing the new module version. So any changes you make inside a module directory that is managed by npm will be deleted.
Ok but why not let npm do the management and add the entries in such a place that it is managed correctly?
even if that was the case, what would prevent simply reinstalling the plugins as well?
The package.json defining installed plugins would be deleted.
Ok I see your point but this is precisely the case because NodeBB insists to run in its source code directory. Indeed, NodeBB should look for package.json in the parent directory and further up the directory tree and add the plugin entries there prior to running npm install.
plugins to require('nodebb/src/...') instead of require.main.require('./src/...') which in turn would open way to proper unit testing for plugins
Unit tests shouldn't need to connect to NodeBB, as units are small pieces of code that can be isolated from the rest, including NodeBB. You can set up your unit tests however you want for your plugins.
That is true, I just mean that plugins notoriously use NodeBB internals by means of require.main.require('./src/database') etc. it would be cleaner if NodeBB actually came as a set of modules and one could simply require('nodebb-database');
As for higher level tests:
Again, NodeBB requires a connection to a database, even while testing. Your plugin test harness would still need to initialize the database as they do now.
Well ok that is true, I guess the main points are the ones before.
run multiple instances without copying the entire source code directory
That would be a nice result of rearchitecting, except then every NodeBB instance must be at the exact same version.
True but it is usually fine to just run the most recent version.
be more in line with how Node apps are installed generally
Again, I'm unaware of any node applications like NodeBB, that must connect to a database to function, that work the way you describe. Please provide an example.
I don't think that connecting to a database somehow "excuses" applications writing inside of their own source code directories. It just feels very much not right.
Among bigger applications I guess ESlint still nicely provides a binary. But it would be great if NodeBB was a pioneer and the first one to do it "right" among the Web apps
