Importing very custom IPB

[Nicolas]

After some research I have found that default ipb-importer doesn't provide options to import custom user fields, etc.
I will fork IPB Import
My main concern is data structure, for example in IPB, most addons just altering users table by adding new fields. As for NodeBB It seems It's better to create documents with _key like someCustomField:uid and don't alter User document.
What can you recommend? Main idea is to have some data in database that could be utilized by future plugins.

• User's custom fields. Document for every custom field or group them in one document? Ex: pet:uid, collar:uid or customs:uid
• Custom entities (Example: Awards) - award:1, award:2 ?
• Is there an easy way to integrate custom data import in current Import script?

[julian]

Directly altering the user hash is not recommended, but you can namespace it under the plugin name itself (e.g. An object named user:1:plugin:custom_user_fields) perhaps?

[bentael]

@Nicolas

The Import plugin already alters some the records hashes and adds custom field for later use, they're all using the _imported_ namespace.

{
    "tid": 29897,
    "uid": 8,
    "cid": 2,
    "mainPid": 29897,
    "title": "A quaint old forum tradition: The Introduction",
    "slug": "29897/a-quaint-old-forum-tradition-the-introduction",
    "timestamp": 1414978528000,
    "lastposttime": 1415421136000,
    "postcount": 41,
    "viewcount": 9,
    "locked": false,
    "deleted": false,
    "pinned": false,
    "_imported_tid": 29887,
    "_imported_uid": 16165,
    "_imported_cid": 1,
    "_imported_slug": "",
    "_imported_path": "/Forum/207593/message/1414978528/A+quaint+old+forum+tradition-+The+Introduction%26gt%3B%26gt%3B%26gt%3B",
    "_imported_title": "A quaint old forum tradition: The Introduction>>>",
    "_imported_content": "",
    "_imported_guest": "",
    "_imported_ip": "96.232.167.174",
    "_imported_user_slug": "",
    "_imported_user_path": "",
    "_imported_category_path": "",
    "_imported_category_slug": "",
    "relativeTime": "2014-11-03T01:35:28.000Z",
    ...

check this thread to find out why.
nodebb-plugin-import-network54/issues/2 and how you can delete them if you choose to, the importer does not force you to delete them.

If you want to add more fields, I am more than happy to accept a pull request or to add them my self, as long as they make sense and they're not redundant.

let me know what you have in mind

[Nicolas]

@bentael After some pause, I think I'm ready to work on custom importer. After some investigation I have found, that Importer works with classic/basic entities. Even If I will create custom importer, for example nodebb-plugin-import-ipboard-very-custom, import plugin will use only designed methods (I will be able to import additional fields) like: setup, getPaginatedUsers, getPaginatedCategories, getPaginatedTopics, getPaginatedPosts, and several utility methods like logging, testRun, etc.

My question: as author of importer, what do you recommend to do if I want to import additional entities? (Example: awards)

What I want to accomplish after import:

• Import of standard data: topics, categories, posts, users - It's simple and clear
• Import custom user fields without altering User document for future plugin use
• Import additional entity - Awards for future plugin use
• Import additional entity - Points(like reputation, but isn't part of user table) for future plugin use