Please help. Come here if you have a good heart will give rep!

Hi. I have over 60,000+ pages on my site. All the free Google map generators I goto only can do up to 500 links OR directory such as:


mysite.com/1
mysite.com/2

and not

mysite.com/1/page1.htm
mysite.com/1/page2.htm
etc,...

Does anyone know of a server side Script or free service that crawls/searches/indexes/does ALL of my URLS and saves it as a Google sitemap? Joomla has something like that but im not using a cms on this site, it is static HTML pages that I have been adding to for a while.

Please help.
mysite.com/1/page2.htm

EDIT: I have looked at the Google one, but is there one that is easier?


EDIT 2: Can I combine Google Sitemaps? Let's say that I have a site map for mysite.com/1 can I copy the sitemap from mysite/2 and paste it into teh bottom of the sitemap for mysite/1 to make one sitemap?

 

my site is alrge as well and has static pages and dynamic
after repeated attempts with different scripts for sitemap generation I ended up with the best solution so far - the "Free Google Sitemap generator phpSitemapNG"
from
http://enarion.net/google/phpsitemapng/

it's a PHP script you install and have to configure a LITTLE ( exclude files and folders )

then it's doing an excellent job.
I recommend you use the last stable version NOT the current devel version.
further
if you have static and dynamic pages
make 2 configurations - 2 runs
else script run might hang
single run for all site went wrong for me
when I split the sitemap into dynamic and static
static is done on server - folder listings = hence any number of URL's in seconds
dynamic is done of course by crawling
that takes a while

i needed to include the time out feature in the config - it appeared that my host shuts down PHP scripts running too long

try it's minutes to install and easy to use and reuse may even be done by cron if all config once is perfect

support of the script also is excellent - just post in the forum there.
btw.
phpSitemapNG is one of the several 3rd party sitemap generators listed on Google's sitemap pages

 

Oh thank you so much.

 

You can also use the Python script Google references in the Help pages. Personally, I hate it because I have so many sites that installing a script and configuring it for each domain isn't a solution in my book.

Large sites may require you to go the Python route - but with a site that large, the effort is worth it.

Cabo

 

the google python script on dynamic pages needs log-files to run on and i have tried that and it SUCKS resources so much that my host never can love that script

in addition sitemap creation from log also includes many wrong or mis spelled URLs that need to be filtered carefully after visual inspection of ... 10-thousands of lines ...?

I have dropped G's python long ago, retried once and dumped it for good.

I love the phpsitemapng PHP script and the new devel version looks even more promising for large sites or multi sites

 

I also have very large sites. And the 500 page limit make the Generator useless.

I ran across this one http://gsitecrawler.com/ that does large sites. It takes a while, but the results are very good.

Hope this helps,

Jim Catanich