How to resolve indexing issue? Users data are indexed
-
Hey
Some of the topics are not indexing on google
But unnecessary things like registered users profile, their following, followers details are indexed on google
Is there any way to make it correct?Or is that indexing by Crawl bot?
Sitemap have only topics, posts. -
I've been facing the same problem.
When I check my website in google using site:https://mydomain.com
Then I only see user groups .....People who follow .....
Other than those URLs....i don't see any posts being indexed.
Don't know the logic.
Even those users are banned ...so why google indexes them ..? -
@srinik @Varun-Ganesh-D We do not control how or what pages Google indexes. We provide them with hints via
sitemap.xml, and you should check to make sure that those pages contain the topics you want to have indexed.We currently do not include user data (following/followed/posts), since we feel it is superfluous data, but when Googlebot lands on a page, it will explore the page and find other links, including links to user profiles and so on and so forth.
Typical reasons posts and topics are not indexed is because the category itself is hidden from spiders or inaccessible to all but registered users, which the Googlebot is not.
-
@srinik Think about it: Where/how does google make its money? Those big data relationship maps are more valuable than your post content. And that is why google bot finds and indexes even against robots.txt.
Yeah, I know googbot does not respect robots.txt and you need to create an account, use their tools, etc. to in efforts to potty train their bot. And therein... they can track even more big data. And googbot still indexes your profiles as it seems it pretty much has a mind of its own and carte blanche to do whatever since we all are so desperate for page rankings.
So, now that we've thought about it for five to ten seconds, do you still find this behavior surprising?
Heh, wife is an accountant: When confused: Follow the Money!!
