|
⇤ ← Revision 1 as of 2015-02-03 14:41:35
Size: 107
Comment:
|
Size: 298
Comment: add info about how to
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 2: | Line 2: |
== Theory == The basic method to check if link is dead is sending a HEAD request. Advanced stuff requires crawling links that are alive, and skipping those that are already visited. |
Python is often used to crawl the internet. One of the useful application for that is finding dead links.
Theory
The basic method to check if link is dead is sending a HEAD request.
Advanced stuff requires crawling links that are alive, and skipping those that are already visited.