Re: Index only one path - Karamasoft Support Forums

Home

Member List

Faq

UltimateSearch

Re: Index only one path

Thread Starter: wazahuis Started: 06-16-2009 7:55 PM Replies: 5

Karamasoft Support Forums » General Discussions » UltimateSearch » Re: Index only one path

Printable Version

« Previous Thread Next Thread »

16 Jun 2009, 7:55 PM

wazahuis is not online. Last active: 6/22/2009 10:54:48 PM

wazahuis

Top 10 Posts

Joined on 06-17-2009
Posts 8

Index only one path

Hi,

I only want to index one part of my website, eg.

www.mysite.com/music/

However the page that points to all the links is on a page like:

www.mysite.com/listall.aspx

scanUrlList element in the .config file obviously doesn't work in this case. I basically need an option that says only include pages in the index with in this pattern www.mysite.com/music/*

Is there any easy way to do this?

16 Jun 2009, 8:46 PM

Karamasoft is not online. Last active: 9/2/2010 12:02:18 PM

Karamasoft

Top 10 Posts

Joined on 09-05-2004
Posts 5,325

Re: Index only one path

You can either copy that page to the music folder and set scanUrlList to
www.mysite.com/music/listall.aspx or use scanDirectoryList and mapPathList together in order to index everything under the music folder.

16 Jun 2009, 10:56 PM

wazahuis

Top 10 Posts

Joined on 06-17-2009
Posts 8

Re: Index only one path

Thanks, but the /music folder is not actually a physical folder (it is a url-rewrite for a dynamic url page - ie. /music/anartist.aspx --> /displayitem.aspx?ID=123), so this won't work - correct?

I did try moving the listall into the /music folder but this still allowed pages from other parts of the sites (sometimes, not always).

17 Jun 2009, 7:04 AM

Karamasoft

Top 10 Posts

Joined on 09-05-2004
Posts 5,325

Re: Index only one path

If you set scanUrlList to www.mysite.com/music/listall.aspx it should have allowed everything under www.mysite.com/music and underneath. We don't know whether it has anything to do with url rewriting, but it shouldn't have gone up the top level directory of music.

17 Jun 2009, 2:36 PM

wazahuis

Top 10 Posts

Joined on 06-17-2009
Posts 8

Re: Index only one path

yeah I did try that but it failed. The listall.aspx lists about 8000 hyperlinks on the page and seems to then sometimes fail (regardless of the maxPageCount or maxPageLength items). If I reduce the listall hyperlinks in size it works. And for some reason it includes pages from the top level. I tried this several times. So I abandoned this way.

The only way I have managed to get it to work is to set to useRobotsMeta true. And in my code look I query UltimateSearch in the useragent and dynmatically write NOINDEX meta for the pages I don't want to include. In doing so, I think I found a bug here because it looks as though in this case the maxPageCount is counting the pages that it follows rather than the pages that it includes in the index (ie. it is also counting NOINDEX pages).

This seems like a really painful way to get around this for a simple task. Other search controls i have been testing have a urlpattern in that I can just say start at this urlpattern (mysite.com/default/aspx) but only include these patterns (eg. mysite.com/music/) which works really well. Feature suggestion? :-)

17 Jun 2009, 2:48 PM

Karamasoft

Top 10 Posts

Joined on 09-05-2004
Posts 5,325

Re: Index only one path

Thank you for your feedback. We will add your request in our future enhancements list to add an include pattern. We will look into increasing maxPageCount for only indexed pages. Meanwhile, you may consider increasing the maxPageCount setting to cover your needs.

Page 1 of 1 (6 items)

Karamasoft Support Forums » General Discussions » UltimateSearch » Re: Index only one path

You can add attachments
You can post new topics
You can reply to topics
You can delete your posts
You can edit your posts
You can create polls
You can vote in polls
Forum statistics are enabled
Forum is unmoderated