This is a tutorial on how to crawl a website using python. The source code of the crawler is also provided in case you are feeling lazy to code.
Read this post on teckguide.blogspot.com