@Giorgio-Chiodi

@Mauricio (slightly off-topic) but this system might be relevant to YoReparo. it's called Curiosume and it is a learn/teach matching system that utilizes Wikipedia to classify skills. es possible se puede reparar cualquier cosa en realidad con Curiosume. more information about it here:

http://www.ingenesist.com/general-info/curiosume-integrating-social-innovation.html
https://www.youtube.com/watch?v=6wCK2jEMKsg

i think there is a lot of optimization still possible to make NodeBB run faster. there is still redundancy in the client/server protocol but this is easily improved.

i am interested in how to scale node web services in a completely distributed network, aka P2P or mesh network. to do this, servers must communicate and synchronize data (but not necessarily all data).

@Giorgio-Chiodi said:

@seh @julian Hello guys, just stumbled into this - could it come in handy for something?

http://cartodb.com/

not really, that's the same functionality i'm hoping the maps + the other plugins will be able to make possible in NodeBB. without involving hosted service

@Giorgio-Chiodi said:

@seh A problem I see with the webin plugin is that it cannot currently refresh more often than 1 minute. I understand this is a limitation of chron, but we should already start looking at a way to push content in real time or with little delay.

refresh sooner than once a minute? that could be done with an ordinary setInterval( ). i suppose the advantage of cron is that the job can be seen / managed from beyond the plugin.

for realtime data sharing, RSS is not ideal because it is based on polling. better is something like pubsubhub which pushes data to you when it updates.

but RSS is rather ubiquitous on the web, and it's better than nothing.

@fyhao said:

I wanna have a plugin hook for post editor, to add new custom button.

i did something like that in this in-progress plugin, but i'm not sure if it's recommended:

https://github.com/automenta/nodebb-plugin-map/blob/master/public/main.js#L61

@Giorgio-Chiodi thanks

For now @psychobunny suggested to store JSON data in the post content. This will work if the post data is filtered for client display, and read for updates when sent to the server. This can be used to maintain a separate index of objects described in posts. For example, geolocations can be indexed in its own database for optimized queries. A post might have multiple associated geolocations, or several GeoJSON "shapes".

{ hash: { x: 'y', z: 3}, array: [0.5,"b"], number: 0.2 }

Can Categories be considered a special kind of Tag? The main difference I see is that Topics can only appear in one Category though.

btw, here is the netention web server code:

https://github.com/automenta/netjs/blob/master/server/web.js

sorry it's a bit of a mess combining DB (mongodb), express, socket.io setup, and all the API calls in one file. here's a list of the key functions:

---

loads (at startup) and saves (periodically) system metadata in the database

function loadState,
function saveState

for the server processes new data, coming from client or another server; usually results in add to database

function notice(o, whenFinished, fromSocket) {
function noticeAll(objList) - 

delete functions

function deleteObject(objectID, whenFinished, contentAddedToDeletedObject, byClientID) {
function deleteObjects(objs, whenFinished) {
function deleteObjectsWithTag(tag, callback) {

query

function getObjectByID(uri, whenFinished) {
function getObjectsByAuthor(a, withObjects) {
function getObjectsByTag(t, withObject, whenFinished) {
function getLatestObjects(n, withObjects, withError) {
function getExpiredObjects(withObjects) {

users can have multiple profiles/personas:

function addClientSelf(req, uid) {

function getCurrentClientID(req) {

function getClientSelves(req) {

        socket.on('become', function(target, _onResult) {  when client first login or switches identity

        socket.on('disconnect', function() { client disconnect

multi-level scope system for deciding who can access what object: public, trusted, private

function objCanSendTo(o, cid) { implements the 

sends new or updated object to all clients and plugins

function broadcast(socket, o, whenFinished) { 

when clients publish via websocket

        socket.on('pub', function(message, err, success) {  

when client attempts to delete an object (by ID)

        socket.on('delete', function(objectID, whenFinished) { 

current ontology for a client to load on startup

express.get('/ontology/:format', function(req, res) {

wikipedia proxy for selecting tags on live wikipages

express.get('/wiki/search/:query', compression, function(req, rres) {
express.get('/wiki/:tag/html', compression, function(req, rres) {
express.get('/wiki/:tag1/:tag2/html', compression, function(req, rres) {

webrtc roster

function updateUserConnection(oldID, nextID, socket) {

function setUserWebRTC(userID, webrtcID, enabled) {

        socket.on('webRTC', function(id, enabled) {

server chat post in specific channel

        socket.on('channelSend', function(channel, message) {

@psychobunny said:

Have a look at this thread too, there was a discussion similar to your above question

right. i like the JSON option best

also, what's the best way to specify a GUID for a topic/post (or equivalent type of field) so that we can later replace it with a new copy?

A post ID (aka PID) is available in the topic api call, if that's what you're looking for?

for example, in:

https://github.com/automenta/nodebb-plugin-webin/blob/master/index.js#L152

i would like to specify a topic TID (coming from the RSS item's GUID) so that when i encounter it again on the next RSS reload, i can replace it in one database query. right now a topic's new TID is auto-incremented, not specified by a parameter:

https://github.com/NodeBB/NodeBB/blob/master/src/topics/create.js#L23

maybe there's some other way to do this without changing the current TID design. i could store a mapping of GUIDs to TIDs in the RSS plugin but that doesn't seem like a good idea.

i see there's a few ways to find topics (by TID, by UID, etc..) but not by some other piece of metadata.

https://github.com/NodeBB/NodeBB/blob/master/src/topics.js#L171

@psychobunny the map stuff is mostly jquery rewriting the client dynamically. not sure if this is a good way to do it - the theme's HTML structure might change at some point and it will not know what to manipulate. anyway it works for now

@Giorgio-Chiodi said:

@rezn8d I really like this idea.

Really really impressive what you made - can we test it?????

the map stuff isn't anywhere near finished. i might be able to get back to it today. but if you want to see what it does so far, can you install the module from git?

this might work, from your nodeBB directory:

npm i automenta/nodebb-plugin-map

and

npm i automenta/nodebb-plugin-webin

let me know if you run into any trouble.

@psychobunny we can read it back in on the server-side using a filter, and store in the database for fast access and indexing. but i'm wondering if this is something that should be a global feature, allowing any plugin to read and write arbitrary metadata (keys/values) per post. i think there was something in the NodeBB server code about 'Post Fields' - is this similar somehow?

@psychobunny i'm not sure yet if what i'm planning is antithetical to NodeBB architecture or not. if you have some time i can guide you through Netention's features so that you might suggest how you would implement them in NodeBB. http://geekery.biz is a live demo (anonymous login works) but it may not be obvious how to reach all the features.

also, what's the best way to specify a GUID for a topic/post (or equivalent type of field) so that we can later replace it with a new copy? for example, in RSS, each post already has a GUID. i'd like to re-use that as the topic ID so that when the RSS is reloaded, i want to be able to replace the content most easily and efficiently, with minimal # of DB queries. this will also support some of the other interfaces I want, such as P2P synchronization between servers.

thanks!

button for adding geo metadata.

updates here:
https://github.com/automenta/nodebb-plugin-map

will continue tomorrow.

for displaying an inline map:
so far it detects an embedded JSON code in the post and if it exists, creates a leaflet map with the coordinates. this reminds me of wordpress shortcodes.

in the screenshot, the post was created by the WebIn plugin from a USGS GeoRSS feed.

@nik said:

I'm totally impressed by this endeavour as it's something not far from what I've been contemplating for a while. Let me know when you get started and will try to contribute

I'm new to writing NodeBB plugins. I started today and experimenting here:

https://github.com/automenta/nodebb-plugin-webin
https://github.com/automenta/nodebb-plugin-map

Now that it loads leaflet.js I'm trying to find the best way to embed and retrieve metadata (including geolocations or GeoJSON data) in posts. We want to display individual maps for posts and also combined maps for collections of posts: all posts in a topic, all posts in a category, etc...

@nik if you have any NodeBB development experience, it would be helpful to know what you can suggest.

Semantic Metadata discussion:

https://community.nodebb.org/topic/1858/embedded-metadata-for-posts-and-other-objects

Wondering what is the best way to store semantic metadata in posts. This would include things like:

• key=value pairs for different types of values: numbers, strings, boolean, etc
• space locations (geographic latitude,longitude)
• time locations (unixtime integer is sufficient)
• etc..

One way is to embed some JSON, like:

{ x: 12, y: 'b' }

Another way is to use a markdown macro:
[[x=12 y="b"]]

Some or all of these data fields can be cached in the database when a post is updated.

They can also be analyzed to create widgets for viewing and editing. For example, if

[[length value="12" units="km"]]

can create a text input box when the post is edited.

We can connect this to the tagging system to support schemas/ontologies such as:
http://schema.org
http://dbpedia.org
http://www.wikidata.org
https://github.com/automenta/netjs/blob/master/server/general.js
https://github.com/automenta/netjs/tree/master/plugin/schema_org

This is the beginning of a plugin for a map view for posts that have geo tags:

A few questions before proceeding:

How should post geographic data be stored? I'm considering a markdown tag like

[[where <lat>,<lon>,<altitude>,<planet>]]

or embedded JSON like:

{where: [ 28.00, -23,44 ] }
{where: { lat: 28.00, lon: -23,44 } }

Embedded metadata like this can be processed by a server-side filter to add to a separate geoindex (for proximity searching), or it can be added directly to the post object when it's inserted into the database.

---

This map is currently displayed as a popup (and could also be a widget) which can change its contents according to what categories/topics/posts are being visited. What is the best way to access the currently displayed objects to scan them for any geographic location metadata in order to plot them on a map?

This set of objects might include users, other NodeBB servers, etc..

---

My reason for making this plugin is to replicate the features in the software shown below, but using the NodeBB framework.

to learn more about nodeBB plugins, i forked @barisusakli 's RSS plugin and am adding some extra features (iCal, geoRSS, web page snapshots) in this "Web Input" plugin:

https://github.com/automenta/nodebb-plugin-webin

seems to be working, but needs some more improvements before it's ready. then next maybe i'll add iCal and GeoRSS.

i got nodeBB running on my PC - easy setup, thanks! starting to see how to make the plugins. will update reports in this thread.

some more screenshots:

Hi! I'm considering how to implement a new version of Netention ( http://github.com/automenta/netjs ) as a set of plugins for NodeBB. Netention is a Node.JS/HTML5 webapp that uses some technologies in common with NodeBB, such as websockets. Bringing the missing features into NodeBB's design would provide a much more solid foundation and larger developer and user community.

Here's the set of features which could be added as NodeBB plugins:

• Geographic Mapping - displaying maps of geolocated objects (posts), and showing contextual embedded maps. I wasn't able to find any NodeBB mapping plugins. Netention started with using OpenStreetMaps but later switched to Leaflet.js which was more than equally capable and much lighter-weight. https://github.com/automenta/netjs/blob/master/client/view.map.js We also have a partial implementation of a 3D WebGL map using Cesium.js https://github.com/automenta/netjs/blob/master/client/view.map.cesium.js

• Semantic Objects - viewing and editing of semantic metadata, using a common ontology, that can be used to describe quantities and qualties of real and imaginary things. I think this can be implemented as a filter for post editor and viewer that adds the extra widgets and parses the embedded metadata for processing on the server-side. The benefit of using semantic tags is matching and coordinating activities. https://github.com/automenta/netjs/blob/master/client/util.js#L797 https://github.com/automenta/netjs/blob/master/client/widget.object.js

• Taggers - ways of selecting semantic tags. includes several kinds of ontology browser for quickly choosing tags. https://github.com/automenta/netjs/blob/master/client/widget.tag.js also includes a wikitagger for choosing any wikipedia page as a tag. https://github.com/automenta/netjs/blob/master/client/widget.wiki.js this was developed for the Curiosume project which is explained here http://www.ingenesist.com/general-info/curiosume-integrating-social-innovation.html

• Graph view - for visualizing and editing relationships between objects. Uses D3 and can include many more features than what currently exists. https://github.com/automenta/netjs/blob/master/client/view.graph.js

• Slides view - uses reveal.js https://github.com/automenta/netjs/blob/master/client/view.browse.js#L129

• WebRTC for peer-to-peer chat, audio, and video. using peer.js, all client-side https://github.com/automenta/netjs/blob/master/client/webrtc.js we also use WebRTC in the code borrowed from Meatspac.es to capture a webcam clip, generate a .GIF, and upload to the server as a new object.

• P2P inter-server network - This might be redundant if a Jabber network exists but this protocol may have some advantages. Needs experimenting. https://github.com/automenta/telepathine https://github.com/automenta/netjs/blob/master/plugin/p2p/netention.js

• Contextual actions (client-side) https://github.com/automenta/netjs/blob/master/client/action.js determines which of a set of actions applies to a given object (post)

• Trust & Value Network uses a graph to calculate flows of "Trust" and "Value" between objects (including users). the Trust level can be used to determine sharing access to data shared with the "trusted" visibility level. In addition there is 'private' (only myself), 'public' everyone on the server, 'anonymous' public without author information, and 'published' which is designated for broadcast push on the p2p network. https://github.com/automenta/netjs/blob/master/server/web.js#L1087

• Ripple.com Plugin this connects to the ripple payment network using the node.js ripple-lib module. it gets public ledger data including balances and trust (credit) lines for accounts that have a ripple wallet associated. the trust lines can also be interpreted as trust between two users who both use ripple. a similar system can be implemented for bitcoin, ethereum, maidsafe, and other cryptocoin networks. https://github.com/automenta/netjs/blob/master/plugin/ripple.js

• Other plugins - https://github.com/automenta/netjs/tree/master/plugin many of these are just empty except for some notes of how they should work.

Thanks! if i think of more features i'll reply. I'll be experimenting with NodeBB soon to see how feasible this is. If anyone would like to help, please feel free to go ahead and start translating any of the Netention code into NodeBB plugins. feedback from experienced NodeBB developers will be greatly appreciated.

Netention is SEMANTIC GROUPWARE that automatically organizes a community of peoples' realities.

It can then interlink them with automatically discovered opportunities that are mutually inter-satisfying - essentially suggesting to its participants how they could realize the desired futures they have described.

Netention is a tool for describing one's current reality (“is”), and potential future realities (“should be”) – as linked data objects. A semantic description of a life (human, animal, plant, or otherwise) can be considered to consist of a set of data declarations representing aspects about which one is concerned or interested. Now we can link the current and desired states of the real and imaginary concepts in our minds - including people, environments, objects, processes, knowledge, or time.

Netention realizes one’s goals based on their desires and translates them into tangible outcomes in one’s own growable network.

It can be used to organize a community, a business, a family, or a government in similar ways because it is designed to consider how the mind expresses thought (especially language) at a neural level.

Netention aims to solve, in general, all resource management and planning issues that occur among communities of participants. A complete system could conceivably eliminate the balkanization of various separate online services that presently serve relatively narrow subsets of the more general problem.

Netention is under development and is experimental. Use at your own risk