URL编码
可以使用HttpApp.pas中的HTTPEncode
对百度,直接使用HTTPEncode()编码就好了,但对Google,有点特别,需要做如下的编码
HTTPEncode(UTF8Encode(url))
即先做utf8编码再做url编码
IdHTTP的设置
IdHTTPWorker.AllowCookies := False;
IdHTTPWorker.HandleRedirects := False;
IdHTTPWorker.Request.Accept := 'image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/x-shockwave-flash, */*';
//IdHTTPWorker.Request.AcceptLanguage := 'zh-cn';
IdHTTPWorker.Request.AcceptEncoding := 'gzip, deflate';
IdHTTPWorker.Request.Host := 'www.google.cn';
IdHTTPWorker.Request.Connection := 'Keep-Alive';
IdHTTPWorker.Request.UserAgent := 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) ; .NET CLR 1.1.4322; .NET CLR 2.0.50727)';
IdHTTPWorker.Request.Referer := 'http://www.google.cn/';
代理
IdHTTPWorker.ProxyParams.ProxyServer := '127.0.0.1';
IdHTTPWorker.ProxyParams.ProxyPort := 8888;
抓取网页源代码时,如果网页时utf-8编码,就需要做如下解码先:
Utf8ToAnsi(IdHTTPWorking.Post(kaixinIndexUrl, Parameters));
但针对gb2316就不行了
针对gbk的编码,如果不设置
IdHTTPWorker.Request.AcceptLanguage := 'zh-cn';
则连Utf8ToAnsi()都不需要调用,直接拿Post()的返回即可(哪怕含有中文字符也没问题,不会是乱码)