Web Development Thread, How Do I Spider Myself in Coding and Web Development; I've made changes to my robots.txt file in the root of my website, but it had me thinking. How can ...
23rd March 2010, 06:09 PM #1
How Do I Spider Myself
I've made changes to my robots.txt file in the root of my website, but it had me thinking. How can I spider myself and see what results I get? I tried my google foo, however with such a grand word as 'spider' it didnt quite get what I was looking for.
So how exactly can I Spider myself?
IDG Tech News
23rd March 2010, 11:42 PM #2
Try Google Webmaster Tools.
You can fetch the page as googlebot and view your rankings for different keywords etc...
1st April 2010, 10:18 AM #3
There's google webmaster tools, which is useful however i noticed a chrome extension called SEO, and when you go onto your site, you click the SEO button and it gives you stats on how the net see's your site and what search engines have it logged.
SEO for Chrome
Obviously you need chrome for it to install. It helped me with my site.
12th November 2010, 12:03 PM #4
- Rep Power
there are lots of free tools available where you can check out.. like Search Engine Spider Test Tool ..there are lots more just google search it..
By RabbieBurns in forum General Chat
Last Post: 22nd March 2010, 05:19 PM
By mattx in forum Jokes/Interweb Things
Last Post: 12th December 2009, 05:56 PM
By MrPstv in forum AV and Multimedia Related
Last Post: 11th May 2009, 02:31 PM
By ICT_GUY in forum General Chat
Last Post: 10th February 2009, 07:39 PM
By FN-GM in forum General Chat
Last Post: 30th January 2008, 06:45 PM
Users Browsing this Thread
There are currently 1 users browsing this thread. (0 members and 1 guests)