Frequently
Asked Questions
About Google Sitemaps
- What is Google Sitemaps?
- Who can use Google Sitemaps?
- How much does it cost?
- Why is Google doing this?
- How do I get started?
- Do I need to sign up for a Google
Account?
- Will participating in this program
change my pages' ranking in Google search results?
Sitemaps
- What is the Sitemap Protocol?
- How do I create a Sitemap?
- Will Google crawl and index all of the
URLs in my Sitemap?
- How do I submit my Sitemap to Google?
- How long will it take for my URLs to be
crawled after I generate and submit a Sitemap?
- I don't want my Sitemap available for the
world to see. Can I make it visible only to Google?
- I have no easy way to extract URLs from my
database of dynamic URLs. How can I generate a Sitemap?
- What other formats can I use for my
sitemaps?
- What is the simplest sitemap I can submit?
- Can I check my Sitemap for compliance
before submitting it?
Sitemap Generator
- What is Sitemap Generator?
- What are the requirements for Sitemap
Generator?
- How do I run Sitemap Generator?
- Where can I download Sitemap Generator?
- What if my URLs can't be extracted by one
of the mechanisms supported by Sitemap Generator?
1.
What is Google Sitemaps?
Google
Sitemaps is an experiment in web crawling. Using Sitemaps to inform and
direct our crawlers, we hope to
expand our coverage of the web and improve the time to inclusion in our
index. By placing a Sitemap-formatted file
on your webserver, you enable our crawlers to find out what pages are
present and which have recently changed,
and to crawl your site accordingly.
Basically, the
two steps to participating in Google Sitemaps are:
- Generate a Sitemap in the
correct format using Sitemap Generator.
- Update your Sitemap when you
make changes to your site.
2.
Who can use Google Sitemaps?
Google
Sitemaps is intended for all web site owners, from those with a single
web page to companies with
millions of ever-changing pages. If either of the following are true,
then you may be especially interested in Google Sitemaps:
- You want Google to crawl
more of your web pages.
- You want to be able to tell
Google when content on your site changes.
3.
How much does it cost?
Absolutely
nothing. Google has never charged for placement in our search results,
and we don't have any plans to do so.
4.
Why is Google doing this?
In alignment
with Google's mission to organize the world's information and make it
universally accessible,
this collaborative crawling system will allow our crawlers to optimize
the usefulness of Google's index for
users by improving its coverage and freshness.
5.
How do I get started?
Read 'How do I create a Sitemap'
below to learn about the format for Google Sitemaps. We also have
detailed
documentation on the Sitemap
Protocol and the Sitemap Generator
if you'd like to skip straight to the technical details.
6.
Do I need to sign up for a Google Account?
You don't need
an account to generate and submit a Sitemap. However, we encourage you
to
sign up for an account so that
you can track the status of your Sitemaps and view diagnostic
information for your submissions. Having an account will not
affect your site's ranking within our results. If you already use
Gmail, Groups, My Search History, Alerts, or Froogle
Shopping List, you already have a Google Account and can sign in with
your existing account to use Google Sitemaps.
7.
Will participating in this program change my pages' ranking in Google
search results?
No. Using
Google Sitemaps will not influence your PageRank; there will be no
change in how we calculate the ranking of your pages.
1.What
is the Sitemap Protocol?
The Sitemap
Protocol is a dialect of XML for summarizing sitemap information that
is relevant to web crawlers.
For each URL, you can include crawl "hints" like the last modified date
and approximate change frequency.
You can read more about the Sitemap Protocol here.
2.
How do I create a Sitemap?
There are a
number of methods you can use to create a Sitemap. We are proud to
offer TM Sitemap
Generator,
downloadable from the same place - it's
a simple script that
generates Sitemaps for basic use cases.
You can read more about the Sitemap Generator below. If the Sitemap
Generator will not work for your site structure,
we encourage you to write your own script for generating Sitemaps and
share it with others.
3.
Will Google crawl and index all of the URLs in my Sitemap?
We don't
guarantee that we'll crawl or index all of your URLs. However, we use
the data in your Sitemap to
learn about your site's structure, which will allow us to improve our
crawler schedule and do a better job
crawling your site in the future. In most cases, webmasters will
benefit from Sitemap submission, and in no case
will you be penalized for it.
4.
How do I submit my Sitemap to Google?
There are a
number of ways to submit your Sitemap for inclusion in Google Sitemaps.
The Sitemap Generator script
can build and submit your Sitemap automatically. If you don't use the
Sitemap Generator, you may also submit your
Sitemap through your Google Sitemaps account; or, you may use an HTTP
request for submission.
To submit your
Sitemap using an HTTP request, issue your request to the following URL:
www.google.com/webmasters/sitemaps/ping?sitemap=sitemap_url
For example,
if your Sitemap is located at http://www.example.com/sitemap.gz, your
URL will become:
www.google.com/webmasters/sitemaps/ping?sitemap=http://www.example.com/sitemap.gz
URL
encode everything after the
/ping?q=:
www.google.com/webmasters/sitemaps/ping?sitemap=http%3A%2F%2Fwww.yoursite.com%2Fsitemap.gz
You can issue
the HTTP request using wget, curl, or another mechanism of your
choosing. A successful request will
return an HTTP 200 response code; if you receive a different response,
you should resubmit your request.
Note: If you are providing a Sitemap index file, you only need to issue
one HTTP request that includes the location
of the Sitemap index file; you do not need to issue individual requests
for each Sitemap listed in the index.
5.
How long will it take for my URLs to be crawled after I generate and
submit a Sitemap?
During this
beta period, we cannot make any predictions or guarantees about when or
if your URLs will be
crawled or added to our index. Over time, we expect both coverage and
time-to-index to improve as we refine
our processes and better understand webmasters' needs.
6. I
don't want my Sitemap available for the world to see. Can I make it
visible only to Google?
Your Sitemap should be placed in a
location
on your webserver that is accessible to our crawlers. If you'd like to
prevent other robots from accessing your Sitemap file, you may use a
robots.txt file that complies with the
Robots
Exclusion Standard.
If you maintain web pages that you do not
wish to include in your Sitemap, you can remove them from the final
Sitemap
file that is posted to your site.
7. I
have no easy way to extract URLs from my database of dynamic URLs. How
can I generate a Sitemap?
You can use
any reasonably large access log (i.e., an Apache log) to submit your
URLs. The
Sitemap
Generator allows
you to generate a Sitemap from a list of URLs, from your access logs,
or by pointing to a directory path hosting
static files corresponding to URLs.
8.
What other formats can I use for my sitemaps?
We also
support the Open
Archives Initiative (OAI) protocol for metadata harvesting,
a popular protocol in the library world.
If your sitemaps are already available in OAI-PMH version 2.0 format,
you are welcome to submit these.
We also accept RSS 2.0 and Atom 0.3 syndication feeds, using the
link/lastMod fields.
Finally, if
you simply want to give us a list of your URLs, read "What is the
simplest sitemap I can submit?" below.
9.
What is the simplest sitemap I can submit?
We strongly
recommend that you use an XML format such as Sitemap or OAI for your
sitemaps, since they allow you to
associate additional information with each URL. However, we can also
accept sitemaps in the form of a text file containing
a simple list of URLs. The simple sitemap format consists of a list of
URLs with one URL per line. For example:
http://www.example.com/catalog?item=1 http://www.example.com/catalog?item=11 ...
Notes about
this format:
- Your URLs must not include
embedded newlines.
- You must fully specify URLs
because Google tries to crawl the URLs exactly as you provide them.
- Your sitemap files must use
UTF-8 encoding.
10.
Can I check my Sitemap for compliance before submitting it?
Google uses an
XML schema to define the elements and attributes that can appear in
your Sitemap file.
You can download this schema from the links below:
For Sitemaps:
http://www.google.com/schemas/sitemap/0.84/sitemap.xsd
For Sitemap index files:
http://www.google.com/schemas/sitemap/0.84/siteindex.xsd
There are a
number of tools available to help you validate the structure of your
Sitemap based on this schema.
You can find a list of XML-related tools at each of the following
locations:
http://www.w3.org/XML/Schema#Tools
http://www.xml.com/pub/a/2000/12/13/schematools.html
In order to validate your Sitemap or Sitemap index file against a
schema, the XML file will need additional headers.
If you're using the Sitemap Generator, these headers are already
included. If you are using a different tool for
creating your sitemaps, the header in the XML file should look like the
examples below.
Sitemap:
<?xml version='1.0' encoding='UTF-8'?> <urlset xmlns="http://www.google.com/schemas/sitemap/0.84" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.google.com/schemas/sitemap/0.84 http://www.google.com/schemas/sitemap/0.84/sitemap.xsd">
<url> ... </url> </urlset>
Sitemap index
file:
<?xml version='1.0' encoding='UTF-8'?> <sitemapindex xmlns="http://www.google.com/schemas/sitemap/0.84" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.google.com/schemas/sitemap/0.84 http://http://www.google.com/schemas/sitemap/0.84/siteindex.xsd">
<sitemap> ... </sitemap>
</sitemapindex>
1.
What is Sitemap Generator?
TM Sitemap
Generator is a simple Perl script that can be configured to
automatically
create Sitemaps and submit it into Google.
Sitemap Generator can create these Sitemaps from a directory path
hosting static files
corresponding to URLs. You can read more about Sitemap Generator here.
2.
What are the requirements for Sitemap Generator?
Sitemap
Generator requires Perl scripts interpretor.
3.
How do I run Sitemap Generator?
We've placed installation and execution tips for TM
Google Sitemap Generator here.
4.
Where can I download Sitemap Generator?
Archived Sitemap
Generator source code placed here here.
|