如果经常下载网站的访问日志查看，通过查看日志来发现那家公司的蜘蛛在爬我的网站。今天查看日志的时候，发现以下几行日志（其实有很多这样的日志，我只列出有代表的几行。），以前也发现了这些东西，只是没有注意，通过仔细观察，有了新的发现。
日志列表：
61.178.184.107 - - [27/Oct/2006:05:11:44 -0700] "GET /qq.txt HTTP/1.1" 404 579 "-" "Mozilla/3.0 (compatible; Indy Library)"
219.140.166.246 - - [27/Oct/2006:05:19:01 -0700] "GET /s8upfile_photo.asp HTTP/1.1" 404 591 "-" "InetURL:/1.0"
219.140.166.246 - - [27/Oct/2006:05:19:02 -0700] "GET /upfile_photo.asp HTTP/1.1" 404 589 "-" "InetURL:/1.0"
219.139.140.189 - - [27/Oct/2006:05:55:52 -0700] "GET /bbs/data/s8dvbbs6.mdb HTTP/1.1" 404 594 "-" "InetURL:/1.0"
219.139.140.189 - - [27/Oct/2006:05:55:52 -0700] "GET /bbs/data/dvbbs6.mdb HTTP/1.1" 404 592 "-" "InetURL:/1.0"
219.139.140.189 - - [27/Oct/2006:05:55:55 -0700] "GET /data/s8dvbbs6.mdb HTTP/1.1" 404 590 "-" "InetURL:/1.0"
219.139.140.189 - - [27/Oct/2006:05:55:55 -0700] "GET /data/dvbbs6.mdb HTTP/1.1" 404 588 "-" "InetURL:/1.0"
219.139.140.189 - - [27/Oct/2006:05:55:56 -0700] "GET /bbs/data/s8dvbbs7.mdb HTTP/1.1" 404 594 "-" "InetURL:/1.0"
219.139.140.189 - - [27/Oct/2006:05:55:58 -0700] "GET /bbs/data/dvbbs7.mdb HTTP/1.1" 404 592 "-" "InetURL:/1.0"
222.88.224.244 - - [27/Oct/2006:05:59:19 -0700] "GET /servu.php HTTP/1.1" 404 582 "-" "Mozilla/3.0 (compatible; Indy Library)"
222.88.224.244 - - [27/Oct/2006:05:59:19 -0700] "GET /bbs/s8servu.php HTTP/1.1" 404 588 "-" "InetURL:/1.0"
222.88.224.244 - - [27/Oct/2006:05:59:20 -0700] "GET /bbs/servu.php HTTP/1.1" 404 586 "-" "Mozilla/3.0 (compatible; Indy Library)"
222.170.97.246 - - [27/Oct/2006:06:45:18 -0700] "GET /ip.txt HTTP/1.1" 404 579 "-" "Mozilla/3.0 (compatible; Indy Library)"
222.170.97.246 - - [27/Oct/2006:06:45:19 -0700] "GET /s8qq.txt HTTP/1.1" 404 581 "-" "InetURL:/1.0"
222.170.97.246 - - [27/Oct/2006:06:45:25 -0700] "GET /qq.txt HTTP/1.1" 404 579 "-" "Mozilla/3.0 (compatible; Indy Library)"
124.114.76.28 - - [27/Oct/2006:06:47:43 -0700] "GET /s8password.txt HTTP/1.1" 404 587 "-" "InetURL:/1.0"
124.114.76.28 - - [27/Oct/2006:06:47:43 -0700] "GET /password.txt HTTP/1.1" 404 585 "-" "InetURL:/1.0"
124.114.76.28 - - [27/Oct/2006:06:47:45 -0700] "GET /pp.txt HTTP/1.1" 404 579 "-" "InetURL:/1.0"
124.114.76.28 - - [27/Oct/2006:06:47:46 -0700] "GET /s8touqq.txt HTTP/1.1" 404 584 "-" "InetURL:/1.0"
124.114.76.28 - - [27/Oct/2006:06:47:47 -0700] "GET /touqq.txt HTTP/1.1" 404 582 "-" "InetURL:/1.0"
219.140.166.246 - - [27/Oct/2006:05:19:01 -0700] "GET /s8upfile_photo.asp HTTP/1.1" 404 591 "-" "InetURL:/1.0"
219.140.166.246 - - [27/Oct/2006:05:19:02 -0700] "GET /upfile_photo.asp HTTP/1.1" 404 589 "-" "InetURL:/1.0"
219.139.140.189 - - [27/Oct/2006:05:55:52 -0700] "GET /bbs/data/s8dvbbs6.mdb HTTP/1.1" 404 594 "-" "InetURL:/1.0"
219.139.140.189 - - [27/Oct/2006:05:55:52 -0700] "GET /bbs/data/dvbbs6.mdb HTTP/1.1" 404 592 "-" "InetURL:/1.0"
219.139.140.189 - - [27/Oct/2006:05:55:55 -0700] "GET /data/s8dvbbs6.mdb HTTP/1.1" 404 590 "-" "InetURL:/1.0"
219.139.140.189 - - [27/Oct/2006:05:55:55 -0700] "GET /data/dvbbs6.mdb HTTP/1.1" 404 588 "-" "InetURL:/1.0"
219.139.140.189 - - [27/Oct/2006:05:55:56 -0700] "GET /bbs/data/s8dvbbs7.mdb HTTP/1.1" 404 594 "-" "InetURL:/1.0"
219.139.140.189 - - [27/Oct/2006:05:55:58 -0700] "GET /bbs/data/dvbbs7.mdb HTTP/1.1" 404 592 "-" "InetURL:/1.0"
222.88.224.244 - - [27/Oct/2006:05:59:19 -0700] "GET /servu.php HTTP/1.1" 404 582 "-" "Mozilla/3.0 (compatible; Indy Library)"
222.88.224.244 - - [27/Oct/2006:05:59:19 -0700] "GET /bbs/s8servu.php HTTP/1.1" 404 588 "-" "InetURL:/1.0"
222.88.224.244 - - [27/Oct/2006:05:59:20 -0700] "GET /bbs/servu.php HTTP/1.1" 404 586 "-" "Mozilla/3.0 (compatible; Indy Library)"
222.170.97.246 - - [27/Oct/2006:06:45:18 -0700] "GET /ip.txt HTTP/1.1" 404 579 "-" "Mozilla/3.0 (compatible; Indy Library)"
222.170.97.246 - - [27/Oct/2006:06:45:19 -0700] "GET /s8qq.txt HTTP/1.1" 404 581 "-" "InetURL:/1.0"
222.170.97.246 - - [27/Oct/2006:06:45:25 -0700] "GET /qq.txt HTTP/1.1" 404 579 "-" "Mozilla/3.0 (compatible; Indy Library)"
124.114.76.28 - - [27/Oct/2006:06:47:43 -0700] "GET /s8password.txt HTTP/1.1" 404 587 "-" "InetURL:/1.0"
124.114.76.28 - - [27/Oct/2006:06:47:43 -0700] "GET /password.txt HTTP/1.1" 404 585 "-" "InetURL:/1.0"
124.114.76.28 - - [27/Oct/2006:06:47:45 -0700] "GET /pp.txt HTTP/1.1" 404 579 "-" "InetURL:/1.0"
124.114.76.28 - - [27/Oct/2006:06:47:46 -0700] "GET /s8touqq.txt HTTP/1.1" 404 584 "-" "InetURL:/1.0"
124.114.76.28 - - [27/Oct/2006:06:47:47 -0700] "GET /touqq.txt HTTP/1.1" 404 582 "-" "InetURL:/1.0"
另一个网站的日志发现：
218.28.132.170 - - [27/Oct/2006:15:59:07 -0700] "GET /qq.txt HTTP/1.1" 404 579 "-" "Mozilla/3.0 (compatible; Indy Library)"
218.28.132.170 - - [27/Oct/2006:15:59:08 -0700] "GET /qqhao.txt HTTP/1.1" 404 582 "-" "InetURL:/1.0"
218.28.132.170 - - [27/Oct/2006:15:59:08 -0700] "GET /123.txt HTTP/1.1" 404 580 "-" "InetURL:/1.0"
218.28.132.170 - - [27/Oct/2006:15:59:08 -0700] "GET /qq123.txt HTTP/1.1" 404 582 "-" "InetURL:/1.0"
218.28.132.170 - - [27/Oct/2006:15:59:08 -0700] "GET /qqmima.txt HTTP/1.1" 404 583 "-" "InetURL:/1.0"
218.28.132.170 - - [27/Oct/2006:15:59:09 -0700] "GET /mima.txt HTTP/1.1" 404 581 "-" "InetURL:/1.0"
218.28.132.170 - - [27/Oct/2006:15:59:09 -0700] "GET /pwd.txt HTTP/1.1" 404 580 "-" "InetURL:/1.0"
218.28.132.170 - - [27/Oct/2006:15:59:09 -0700] "GET /mail.txt HTTP/1.1" 404 581 "-" "InetURL:/1.0"
218.28.132.170 - - [27/Oct/2006:15:59:08 -0700] "GET /qqhao.txt HTTP/1.1" 404 582 "-" "InetURL:/1.0"
218.28.132.170 - - [27/Oct/2006:15:59:08 -0700] "GET /123.txt HTTP/1.1" 404 580 "-" "InetURL:/1.0"
218.28.132.170 - - [27/Oct/2006:15:59:08 -0700] "GET /qq123.txt HTTP/1.1" 404 582 "-" "InetURL:/1.0"
218.28.132.170 - - [27/Oct/2006:15:59:08 -0700] "GET /qqmima.txt HTTP/1.1" 404 583 "-" "InetURL:/1.0"
218.28.132.170 - - [27/Oct/2006:15:59:09 -0700] "GET /mima.txt HTTP/1.1" 404 581 "-" "InetURL:/1.0"
218.28.132.170 - - [27/Oct/2006:15:59:09 -0700] "GET /pwd.txt HTTP/1.1" 404 580 "-" "InetURL:/1.0"
218.28.132.170 - - [27/Oct/2006:15:59:09 -0700] "GET /mail.txt HTTP/1.1" 404 581 "-" "InetURL:/1.0"
通过查看其他网站，同样发现了类似的访问日志。
通过分析日志，我们可以得出以下结论：
有某些人派出蜘蛛（严格来说，这不是蜘蛛。）每天在爬您的网站，试图寻找漏洞或者其他用有信息。（应该是某种扫描软件）
这些日志有下面几类：
1。想获取网站漏洞重要管理文件，如：
219.140.166.246 - - [27/Oct/2006:05:19:02 -0700] "GET /upfile_photo.asp HTTP/1.1" 404 589 "-" "InetURL:/1.0"
222.88.224.244 - - [27/Oct/2006:05:59:19 -0700] "GET /servu.php HTTP/1.1" 404 582 "-" "Mozilla/3.0 (compatible; Indy Library)"
222.88.224.244 - - [27/Oct/2006:05:59:19 -0700] "GET /bbs/s8servu.php HTTP/1.1" 404 588 "-" "InetURL:/1.0"
222.88.224.244 - - [27/Oct/2006:05:59:20 -0700] "GET /bbs/servu.php HTTP/1.1" 404 586 "-" "Mozilla/3.0 (compatible; Indy Library)"
222.88.224.244 - - [27/Oct/2006:05:59:19 -0700] "GET /servu.php HTTP/1.1" 404 582 "-" "Mozilla/3.0 (compatible; Indy Library)"
222.88.224.244 - - [27/Oct/2006:05:59:19 -0700] "GET /bbs/s8servu.php HTTP/1.1" 404 588 "-" "InetURL:/1.0"
222.88.224.244 - - [27/Oct/2006:05:59:20 -0700] "GET /bbs/servu.php HTTP/1.1" 404 586 "-" "Mozilla/3.0 (compatible; Indy Library)"
2。 想获取数据库，如：
219.139.140.189 - - [27/Oct/2006:05:55:55 -0700] "GET /data/dvbbs6.mdb HTTP/1.1" 404 588 "-" "InetURL:/1.0"
219.139.140.189 - - [27/Oct/2006:05:55:56 -0700] "GET /bbs/data/s8dvbbs7.mdb HTTP/1.1" 404 594 "-" "InetURL:/1.0"
219.139.140.189 - - [27/Oct/2006:05:55:58 -0700] "GET /bbs/data/dvbbs7.mdb HTTP/1.1" 404 592 "-" "InetURL:/1.0"
219.139.140.189 - - [27/Oct/2006:05:55:56 -0700] "GET /bbs/data/s8dvbbs7.mdb HTTP/1.1" 404 594 "-" "InetURL:/1.0"
219.139.140.189 - - [27/Oct/2006:05:55:58 -0700] "GET /bbs/data/dvbbs7.mdb HTTP/1.1" 404 592 "-" "InetURL:/1.0"
3。 想获取qq、mail、服务器信息等，如：
218.28.132.170 - - [27/Oct/2006:15:59:08 -0700] "GET /qqmima.txt HTTP/1.1" 404 583 "-" "InetURL:/1.0"
218.28.132.170 - - [27/Oct/2006:15:59:09 -0700] "GET /mima.txt HTTP/1.1" 404 581 "-" "InetURL:/1.0"
218.28.132.170 - - [27/Oct/2006:15:59:09 -0700] "GET /pwd.txt HTTP/1.1" 404 580 "-" "InetURL:/1.0"
218.28.132.170 - - [27/Oct/2006:15:59:09 -0700] "GET /mail.txt HTTP/1.1" 404 581 "-" "InetURL:/1.0"
218.28.132.170 - - [27/Oct/2006:15:59:09 -0700] "GET /mima.txt HTTP/1.1" 404 581 "-" "InetURL:/1.0"
218.28.132.170 - - [27/Oct/2006:15:59:09 -0700] "GET /pwd.txt HTTP/1.1" 404 580 "-" "InetURL:/1.0"
218.28.132.170 - - [27/Oct/2006:15:59:09 -0700] "GET /mail.txt HTTP/1.1" 404 581 "-" "InetURL:/1.0"
通过写 robots.txt 文件应该是没有用的，因为这不循序通过写 robots.txt 的规则。
解决的办法是：
1。在命名关键文件的时候，尽量使用前缀，比如hualan888_database.asp，hualan4321_qq.txt。这样可以避免被别人猜测到文件名，防止被下载或者执行文件。
2。重要的文件不要放到网站上，或者过期的时候要删除。比如qq密码，服务器ip和用户密码等，不要贪图方便。
3。修改一些重要文件的路径，防止被猜测到。如这条日志：
219.139.140.189 - - [27/Oct/2006:05:55:55 -0700] "GET /data/dvbbs6.mdb HTTP/1.1" 404 588 "-" "InetURL:/1.0"
显然，219.139.140.189 想获取“ /data/dvbbs6.mdb ” 文件，也就是想获取动网数据库。 修改路径后，就可以数据库被盗。
4。在每个目录下都放一个空白的index.html文件，在一定程序上也可以防止别查看、扫描到目录。pw，dz的论坛也是这样做的。