Wondering what is the best way to store semantic metadata in posts. This would include things like:
One way is to embed some JSON, like:
{ x: 12, y: 'b' }
Another way is to use a markdown macro:
[[x=12 y="b"]]
Some or all of these data fields can be cached in the database when a post is updated.
They can also be analyzed to create widgets for viewing and editing. For example, if
[[length value="12" units="km"]]
can create a text input box when the post is edited.
We can connect this to the tagging system to support schemas/ontologies such as:
@seh Yes please 
I feel like storing stringified JSON would be simpler for you to work with. Just from the plugin's point of view, no idea about the software that actually reads the data in the end
@psychobunny we can read it back in on the server-side using a filter, and store in the database for fast access and indexing. but i'm wondering if this is something that should be a global feature, allowing any plugin to read and write arbitrary metadata (keys/values) per post. i think there was something in the NodeBB server code about 'Post Fields' - is this similar somehow?
@psychobunny i'm not sure yet if what i'm planning is antithetical to NodeBB architecture or not. if you have some time i can guide you through Netention's features so that you might suggest how you would implement them in NodeBB. http://geekery.biz is a live demo (anonymous login works) but it may not be obvious how to reach all the features.
also, what's the best way to specify a GUID for a topic/post (or equivalent type of field) so that we can later replace it with a new copy? for example, in RSS, each post already has a GUID. i'd like to re-use that as the topic ID so that when the RSS is reloaded, i want to be able to replace the content most easily and efficiently, with minimal # of DB queries. this will also support some of the other interfaces I want, such as P2P synchronization between servers.
thanks!
Have a look at this thread too, there was a discussion similar to your above question
i think there was something in the NodeBB server code about 'Post Fields' - is this similar somehow
Not really, I think you're talking about filter:post.getFields right? That's more for embedding data at the bottom of your post (where the up/down vote buttons are in this current theme).
also, what's the best way to specify a GUID for a topic/post (or equivalent type of field) so that we can later replace it with a new copy?
A post ID (aka PID) is available in the topic api call, if that's what you're looking for?
@psychobunny said:
Have a look at this thread too, there was a discussion similar to your above question
right. i like the JSON option best
also, what's the best way to specify a GUID for a topic/post (or equivalent type of field) so that we can later replace it with a new copy?
A post ID (aka PID) is available in the topic api call, if that's what you're looking for?
for example, in:
https://github.com/automenta/nodebb-plugin-webin/blob/master/index.js#L152
i would like to specify a topic TID (coming from the RSS item's GUID) so that when i encounter it again on the next RSS reload, i can replace it in one database query. right now a topic's new TID is auto-incremented, not specified by a parameter:
https://github.com/NodeBB/NodeBB/blob/master/src/topics/create.js#L23
maybe there's some other way to do this without changing the current TID design. i could store a mapping of GUIDs to TIDs in the RSS plugin but that doesn't seem like a good idea.
i see there's a few ways to find topics (by TID, by UID, etc..) but not by some other piece of metadata.
https://github.com/NodeBB/NodeBB/blob/master/src/topics.js#L171
btw, here is the netention web server code:
https://github.com/automenta/netjs/blob/master/server/web.js
sorry it's a bit of a mess combining DB (mongodb), express, socket.io setup, and all the API calls in one file. here's a list of the key functions:
loads (at startup) and saves (periodically) system metadata in the database
Wikidata
function loadState,
function saveState
for the server processes new data, coming from client or another server; usually results in add to database
function notice(o, whenFinished, fromSocket) {
function noticeAll(objList) -
delete functions
function deleteObject(objectID, whenFinished, contentAddedToDeletedObject, byClientID) {
function deleteObjects(objs, whenFinished) {
function deleteObjectsWithTag(tag, callback) {
query
function getObjectByID(uri, whenFinished) {
function getObjectsByAuthor(a, withObjects) {
function getObjectsByTag(t, withObject, whenFinished) {
function getLatestObjects(n, withObjects, withError) {
function getExpiredObjects(withObjects) {
users can have multiple profiles/personas:
function addClientSelf(req, uid) {
function getCurrentClientID(req) {
function getClientSelves(req) {
socket.on('become', function(target, _onResult) { when client first login or switches identity
socket.on('disconnect', function() { client disconnect
multi-level scope system for deciding who can access what object: public, trusted, private
function objCanSendTo(o, cid) { implements the
sends new or updated object to all clients and plugins
function broadcast(socket, o, whenFinished) {
when clients publish via websocket
socket.on('pub', function(message, err, success) {
when client attempts to delete an object (by ID)
socket.on('delete', function(objectID, whenFinished) {
current ontology for a client to load on startup
express.get('/ontology/:format', function(req, res) {
wikipedia proxy for selecting tags on live wikipages
express.get('/wiki/search/:query', compression, function(req, rres) {
express.get('/wiki/:tag/html', compression, function(req, rres) {
express.get('/wiki/:tag1/:tag2/html', compression, function(req, rres) {
webrtc roster
function updateUserConnection(oldID, nextID, socket) {
function setUserWebRTC(userID, webrtcID, enabled) {
socket.on('webRTC', function(id, enabled) {
server chat post in specific channel
socket.on('channelSend', function(channel, message) {
@seh said:
global feature, allowing any plugin to read and write arbitrary metadata (keys/values) per post
I also think this would be ideal - we could also use to develop our own plugins for nodebb this way.
@Giorgio-Chiodi thanks
For now @psychobunny suggested to store JSON data in the post content. This will work if the post data is filtered for client display, and read for updates when sent to the server. This can be used to maintain a separate index of objects described in posts. For example, geolocations can be indexed in its own database for optimized queries. A post might have multiple associated geolocations, or several GeoJSON "shapes".
{ hash: { x: 'y', z: 3}, array: [0.5,"b"], number: 0.2 }
Can Categories be considered a special kind of Tag? The main difference I see is that Topics can only appear in one Category though.
