kflorian
XMod Pro
Posts: 50
www.corgenius.com
 |
| 9/21/2008 9:11 PM |
|
What is required to make XMOD-stored data visible to search engine indexers? Is it even possible?
I know this is an insanely novice question but I am only now getting to the point where it matters to me whether the data on my several dnn portals is accesible to search engines.
|
|
Ken
Corgenius.com
|
|
|
angusbeare
XMod All-Star
Posts: 120
|
| 9/26/2008 5:53 AM |
|
Good question. I'd like to know the answer too!
My experience is that anything that's not displayed on the page won't get indexed.
My xmod lists seem to get indexed pretty well but I guess as soon as there is data not shown on the page and only visible after search it won't turn up on google.
If your data sets are small then just make sure everything is listed. But if you have thousands of records I don't know what the answer is.
We're not the only ones struggling.
There are hundreds of e-commerce sites out there with products that don't show up in google.
And others that put their entire catalogue in one huge flash object. Just look at Vetus.com and try and find their products in google. Pitiful really.. |
|
|
|
|
kflorian
XMod Pro
Posts: 50
www.corgenius.com
|
| 9/26/2008 9:44 AM |
|
( thinking....thinking...)
I think that XMOD data that presents directly on an aspx page (which is visible by bots) can get indexed. What is not indexed in anything in a so-called "detail" record. That stuff is in the xmod database until requested by a user. So, good user interface demands sometimes concealing data until summoned by the user but, until summoned, that data is in the dark web.
One could create a XMOD list view that exposed all the information for a given XMOD record, hide that page from normal view, tell googlebot that it is there through a reference in robots.txt. But then what? that content is indexed (I think) but still won't be accessible to a human viewer.
Depending on the data one is storing, this is can be no trivial question. |
|
Ken
Corgenius.com
|
|
|
angusbeare
XMod All-Star
Posts: 120
|
| 9/27/2008 2:24 PM |
|
yes, that's what I was saying. The invisible web.
I did a bit of research on this and we are talking about what Google calls the
'deep web' or the 'hidden web'.
Apparently they are working on methods to search online databases for the huge resource of data that's hidden from normal search engine indexing.
Read the stuff on this site, it's encouraging
http://googlesystem.blogspot.com/2008/04/google-starts-to-index-invisible-web.html
Angus |
|
|
|
|
kflorian
XMod Pro
Posts: 50
www.corgenius.com
|
| 10/07/2008 3:23 PM |
|
Surely there has got to be a creative way to solve this. The human interface power of showing summary information with drill-down to the details is absolutely necessary but so is the ability to have important detail content indexed by indexers outside of the DNN environment.
So....my current experiment is to:
1. Create my XMOD data in list / detail structure.
2. Expose to the human user the list information with the normal xmod click-for-details interface.
3. Create another page using the repeater template to display the detail information. Put this on a page that is "visible" to a bot but not in the public menu structure of DNN and place a reference to that page in robots.txt to ensure it gets indexed.
Input and other ideas most welcome.
Ken
|
|
Ken
Corgenius.com
|
|
|
gbrown
XMod All-Star
Posts: 508
Ohio
|
| 10/07/2008 11:39 PM |
|
Check the DNN forums - there was just a thread about a free site map module that will craw dynamic data.
Greg |
|
Xmod - The adjustable wrench for your DNN toolbox |
|
|
kflorian
XMod Pro
Posts: 50
www.corgenius.com
|
| 10/09/2008 9:58 PM |
|
Greg,
Thanks for the response. I searched the forums but I'm not finding the thread you mentioned. Do you have another phrase I can use to try to locate it?
Ken
|
|
Ken
Corgenius.com
|
|
|
gbrown
XMod All-Star
Posts: 508
Ohio
|
| 10/09/2008 11:22 PM |
|
The thread is pinned to the top of Configure It! here:
http://www.dotnetnuke.com/Community/Forums/tabid/795/forumid/108/threadid/207139/scope/posts/Default.aspx
The solutions mentioned here may not do exactly what you are looking for but it may give you some added info.
Greg |
|
Xmod - The adjustable wrench for your DNN toolbox |
|
|
bennyj
XMod Pro
Posts: 62
http://www.luv4meonline.com
|
| 10/13/2008 7:02 PM |
|
This may be what you are looking for.
http://www.ifinity.com.au/Products/Google_Sitemap_For_DNN/ |
|
Free Online Dating |
|
|
fatgeorge
XMod All-Star
Posts: 676
www.fatgeorge.co.uk
UK
|
| 11/03/2008 12:03 PM |
|
I have been doing some digging take a look at this:
http://www.fatgeorge.co.uk/Home/articles/tabid/373/articleType/ArticleView/articleId/44/Trawling-the-Deep-Web-The-XMod-Way.aspx
|
|
.png) we don't make XMod, we make it better! |
|
|
kflorian
XMod Pro
Posts: 50
www.corgenius.com
|
| 11/03/2008 10:16 PM |
|
fatgeorge,
Excellent stuff. I'm glad somebody of your calibre is looking into this. As I said above, this is no small matter and an excellent answer has been wanting. I haven't thought a lot about what your current experiment but it sure looks promising! Of course, it is not only xmod but any data structure / presentation tool that wants to provide summary -> detail structure. For my purposes, having an excellent XMOD-based solution will be superior!
Ken
|
|
Ken
Corgenius.com
|
|
|