Try our SEO Web Spider — a new professional SEO crawler for up to 1-million URLs, completely free for life.
London Web Factory

Blog

Practical notes on web design, performance, SEO, PPC and digital marketing.

Best Practices for Creating XML Sitemaps

An XML sitemap helps search engines discover, crawl and index your website. But a good sitemap should do more than simply list every URL found. Images, videos, redirects, duplicate and non-indexable pages must be handled correctly, while larger sites need careful sitemap structure. This guide explains best practices for creating accurate, search-engine-friendly XML sitemaps.

Learn more

Upgrading

Use this guide when installing a newer release over an older copy of LWF SEO Spider.

Learn more

Troubleshooting

This guide covers common installation, browser, crawl, result and registration problems in LWF SEO Spider.

Learn more

Responsible Crawling

LWF SEO Spider is designed for SEO, accessibility, content, quality-assurance and technical analysis. Use it only where you have a lawful and reasonable basis to send crawl requests.

Learn more