Using Sitemaps with Google
As mentioned in the last post, I launched my finance and stock analysis website last month. While setting it up for Google Analytics and Webmaster Tools I once again noticed the warning on the front page of the webmaster tools telling me that i hadn’t submitted any sitemaps and that doing so could help google index pages from my site. Considering I built my site from scratch in php, without using an app like wordpress or joomla which have built in functions for creating sitemaps, I figured i was at a disadvantage in not having one…
After doing a quick bit of research about the format for sitemaps used by google, i put together some code to traverse my site and create a sitemap. I took the time to approximate the update schedule and importance tags which are part of the xml sitemap specification. The final result had just over 1700 URLs…
Here’s a summary of the site’s index status in google in the 3 weeks or so since publishing the site and submitting the sitemap:
- at the time of submission, the website had only 1 URL (the domain root) in the index which had been there for about 2 years.
- About a week later, google webmaster tools seemed to indicate that it had crawled a large amount of URLs in the sitemap. It was reporting a couple of broken links and the crawl statistics said it had crawled over 800 pages. At this point google was still only indexing the front page…
- In the next couple of days there were 27 URLs indexed in google, these seemed to pretty well with the be the URLs with higher importance assigned to them in the sitemap.
- About 2 weeks later i checked in again and saw that there were 533 URLs Indexed by google. This uptake of URLs in google was a lot faster than that observed with other sites that didn’t use a sitemap.
- I also checked in with Webmaster tools and it reported 10 pages had duplicate Description META TAGS. The reality was that every single page in the site had the same description! but google only reported 10 pages, even though it was indexing over 500…
I guess the takeaways are that the update schedule for google’s various data are not in synch at all (not really a surprise considering how much info and services they host). The other major point here is that using a sitemap has been a huge advantage for getting a new site listed. Although, i think a lot of this success was probably due to the trust built up with my domain, the so-called sandbox effect, after having it indexed for about 2 years already.
Anyway, as i also mentioned in the last post, i didn’t want to create external links to the site until Google had taken a good look at it on its own. So now that it has: Low PEG Stocks.com is the name of this stock ratings and analysis website i’m talking about. The site currently has no inbound links according to both Google search and Webmaster tools (these numbers can differ).
Resurrecting this blog…
After about a year of inactivity I’ve resolved to start posting SEO stuff here again…
I’ve recently made some big changes to the way content is managed and generated on the herbal remedies db site, i think the last time i posted some usage stats on the site it was getting around 3 hits per day - it’s now up to around 30 hits a day. I think most of this improvement has come from the ageing factor as well as some external links (aside from the ones on this blog)… More about that site to come…
I’ve also just completed the a Beta version of another project. This one is a bit more serious and kind of less experiment than the Herbal Remedies site. It was over 2 years ago that i registered the domain name of this site and since then have been developing it in my spare time. A lot of hours hae gone into it and i’m glad to say i’ve finally reached the light at the other end of the tunnel.
I want to wait until the googlebot has given it a complete working over before i post a link to it. Essentially the site is a Stock Analysis System which gives ratings on stocks, i post observations on the market on there and i run some model portfolios… Definitely more on this to come shortly…
I have other ideas for blog aggregator type sites which, time permitting, i’ll be putting together over the next couple of months, i’ve seen some great examples of this type of site which seem to achieve great pageranks and readerships, serving a real purpose, while i’ve seen plenty that are just trying to generate thousands of keyword dense pages for search engines to find…
I also have about 4 draft posts from a long time ago which i really should get to finishing.
I guess we’ll see.
Generating advertising income on a low traffic blog or site…
As it stand right now the HRdb site now seems to be generating about 3-4 unique visitors per day. As you might expect the vast majority are from the United States, Canada, U.K. and Australia (mainly due to me). Right now i’d say the basic laws of economics will prevent me from making any kind of income from such a small amount of traffic. I’m hesitant to begin any kind of advertising program for the fear of getting banned or blacklisted as a kind if ‘junk site’. I’ve read that this happened with many adsense publishers a couple of years ago when click fraud became a major issue for google and other search engines. Aside from that, the options are quite limited for sites with limited traffic:
- Casale Media require 10,000 unique visitor/month.
- Chitika required the same at 10,000 visitors/month
- Tribal Fusion require 2,000 per day! (60,000/month)
- Burst Media require 5,000 per month
Many other programs don’t mentions specific minimums but obviously take traffic and visibility into account as part of their approval process…
More to com on this topic, for i have content to build…
Penalty for bad links
Just the other day, my HRdb site dropped several positions on google when searching for its title. The latest update and visit from the googlebot picked up one bad link as revealed in my google webmaster tools. Given that essentially nothing else changed, I have to assume that bad links cause a definite and sustained penalty for a site in pagerank. The page in question was a political article about the banning of ephedra. I had a space before an underscore… We’ll see how long the penalty lasts.
Now that the pagerank experiement has been up and running for a few months, I’ll be redefining the way i keep statistics about record progress. Basically the way i have been doing it is time consuming and has been more or less obsolete by the reports that are generated in google’s Analytics and Webmaster tools.
In the next week i’ll be adding more content to the site and trying to target some of the more popular keywords for the topic. Google is yet to recognise some of the links that have been added from other blogs, but after a quick intermission they are recognising the links from this blog again… More content in the site should draw in more visitors from varying keywords etc…
Some good and bad news on the SEO front…
My Herbal Remedies Database site is now getting about 3 hits a day! This surge in traffic (from 9 as reported in my last post) is partly due to an increase in recent visitors but also from me misreading my Google Analytics numbers…
Moving past that embarrassing gaff, this blog has slipped a spot on google when search for “capricious mind”. Previously I occupied the #2 ranking and was consistently beaten only by Islam watch’s “complete guide to Allah” (I’m serious, search for yourself). However, out of nowhere, a new competitor has snatched the top spot from me and my far eastern compare. A Flickr site titled (in full): “Flickr: Photos from ~*~Capricious Mind~*~”.
Plans for this blog…
As things have progressed, I’m getting a clear idea that this blog is more or less only going to be used for SEO related posts. I’ll be adjusting categories, meta tags etc. accordingly…
After reviewing the traffic stats and search positions of this blog versus the HRdb site, i’m beginning to think that this site will need an improved promotional strategy. Essentially, this page has no inbound links and the topic of SEO is probably one of the most hotly contested in search rankings. A capricious mind does rank well for more obscure keywords (such as the title) but basically remains at the bottom of the heap for the more important search engine optimization related keywords. As a result, traffic is very poor at this stage…
Hopefully i’ll be able to build some external links as well as optimize for some relevant keywords which might not be quite as competitive as say “SEO” or something similar. These things should help to bring more traffic over the next few months. If not, i’m not concerned. The site still has a purpose in being the web log of my web development projects (there are many ideas in the works) which will continue to get more diverse and interesting…
SEO experiment update…
Organic visitors have started to pick up. Although obviously still minute, this is an encouraging sign. Almost all visitors have come from google with only one exception coming from Answers.com. MSN still doesn’t rank any of the site in the first 5 pages of results, while Yahoo is now showing the front page at #13.
Google Analytics have found the site using 8 different each phrases (this excludes my own testing and visiting).
Google webmaster tools now reports that the site ranks in the top 20 for 12 different search terms, 10 of these show up on the first page.
That capricious googlebot…
Site: Herbal Remedies db
http://herbalremediesdb.com
17 (13) pages in the site.
As reported by google (known to me):
- 0 inbound links (1)
- 3 organic visitors (same)
Search rank for the following (prior rank):
- #4 (#1) - herbal remedies db
- #NA (#NA) - herbal remedies
- #12 (#10) - db ginseng
- #NA (#NA) - herbal remedies ginseng
===============================
So as you can see from above, my ranking have slipped slightly after being indexed for the 3rd time on June 23rd. Read the rest of this entry »
Google still seems unstoppable
I noticed this article (seems like i’ve read the same thing at least 20 times) while reading through some financial news sites…
Google performed more than 4 billion searches in a single month for the first time. Its share of the market jumped by 1% over the prior month to 56.3%, increasing the number of searches it performed by 260 million.
Note above it says its market share increased by 1%, the number of searches performed increased by 6.5%, month over month!
Yahoo! performed 1.5 billion searches, for 21.5% of the market, vs. a 21.9% share in April. The number of searches it performed increased somewhat.
Microsoft, which recently announced it will buy online advertising business aQuantive to better compete against Google, saw its share of the search market drop to 8.4%, from 9% the prior month. The number of searches it performed, 605.4 million, increased slightly.
AOL lost 0.1 percentage points for a market share of 5.3%, performing 382 million searches.
Reading this article has only strengthened my resolve that one day (soon) google will virtually own the Internet, basically taking a monopoly over many businesses using Internet based business models. The only real threats to this outcome at the moment are the lack of market share in China and other rapidly emerging foreign markets, plus a few other robust best of breed Internet players like amazon for example. I don’t even think a merger between Yahoo and Microsoft would stop them.
SEO experiment - MSN live and Yahoo!
One thing i forgot to mention in the previous post was that despite being crawled by the bots from MSN and Yahoo, i was unable to find herbal remedies db using their search…
In light of all this i’l be posting shortly about the so called Google sandbox or age factor in ranking…