htmlparser实现从网页上抓取数据
package parser;
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.FileWriter;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.MalformedURLException;
import java.net.URL;
/**
* 基本能实现网页抓取，不过要手动输入URL 将整个html内容保存到指定文件
*
*@author chenguoyong
*
*/
public class ScrubSelectedWeb {
privatefinal static String CRLF = System.getProperty("line.separator");
/**
* @param args
*/
publicstatic void main(String[] args) {
try{
URLur = newURL("http://10.249.187.199:8083/injs100/");
InputStreaminstr = ur.openStream();
Strings, str;
BufferedReaderin = new BufferedReader(new InputStreamReader(instr));
StringBuffersb = new StringBuffer();
BufferedWriterout = new BufferedWriter(new FileWriter(
"D:/outPut.txt"));
while((s = in.readLine()) != null) {
sb.append(s+ CRLF);
}
System.out.println(sb);
str= new String(sb);
out.write(str);
out.close();
in.close();
}catch (MalformedURLException e) {
e.printStackTrace();
}catch (IOException e) {
e.printStackTrace();
}
}
}
基本能实现网页抓取，不过要手动输入URL，此外没有重构。只是一个简单的思路。
http://c.tieba.baidu.com/p/3347869190
http://c.tieba.baidu.com/p/3347869658
http://bbs.tianya.cn/post-763-693512-1.shtml
http://bbs.tianya.cn/post-763-693513-1.shtml
http://bbs.tianya.cn/post-763-693514-1.shtml
http://bbs.tianya.cn/post-763-693515-1.shtml
http://bbs.tianya.cn/post-763-693516-1.shtml
http://bbs.tianya.cn/post-763-693517-1.shtml
http://bbs.tianya.cn/post-763-693518-1.shtml
http://bbs.tianya.cn/post-763-693519-1.shtml
http://bbs.tianya.cn/post-763-693521-1.shtml
http://bbs.tianya.cn/post-763-693522-1.shtml
http://bbs.tianya.cn/post-763-693523-1.shtml
http://bbs.tianya.cn/post-763-693524-1.shtml
http://bbs.tianya.cn/post-763-693525-1.shtml
http://bbs.tianya.cn/post-763-693527-1.shtml
http://bbs.tianya.cn/post-763-693526-1.shtml
http://bbs.tianya.cn/post-763-693528-1.shtml
http://bbs.tianya.cn/post-763-693529-1.shtml
http://bbs.tianya.cn/post-763-693530-1.shtml
http://bbs.tianya.cn/post-763-693531-1.shtml
http://bbs.tianya.cn/post-763-693532-1.shtml
http://bbs.tianya.cn/post-763-693533-1.shtml
http://bbs.tianya.cn/post-763-693534-1.shtml
http://bbs.tianya.cn/post-763-693535-1.shtml
http://bbs.tianya.cn/post-763-693536-1.shtml
http://bbs.tianya.cn/post-763-693537-1.shtml
http://bbs.tianya.cn/post-763-693538-1.shtml
http://bbs.tianya.cn/post-763-693539-1.shtml
http://bbs.tianya.cn/post-763-693540-1.shtml
http://bbs.tianya.cn/post-763-693541-1.shtml
http://bbs.tianya.cn/post-763-693542-1.shtml
http://bbs.tianya.cn/post-763-693543-1.shtml
http://bbs.tianya.cn/post-763-693544-1.shtml
http://bbs.tianya.cn/post-763-693545-1.shtml
http://bbs.tianya.cn/post-763-693546-1.shtml
http://bbs.tianya.cn/post-763-693547-1.shtml
http://bbs.tianya.cn/post-763-693548-1.shtml
http://bbs.tianya.cn/post-763-693549-1.shtml
http://bbs.tianya.cn/post-763-693550-1.shtml
http://bbs.tianya.cn/post-763-693551-1.shtml
http://bbs.tianya.cn/post-763-693552-1.shtml
http://bbs.tianya.cn/post-763-693553-1.shtml
http://bbs.tianya.cn/post-763-693554-1.shtml
http://bbs.tianya.cn/post-763-693555-1.shtml
http://bbs.tianya.cn/post-763-693556-1.shtml
http://bbs.tianya.cn/post-763-693557-1.shtml
http://bbs.tianya.cn/post-763-693557-1.shtml
http://bbs.tianya.cn/post-763-693558-1.shtml
http://bbs.tianya.cn/post-763-693559-1.shtml
http://bbs.tianya.cn/post-763-693560-1.shtml
http://bbs.tianya.cn/post-763-693561-1.shtml
http://bbs.tianya.cn/post-763-693562-1.shtml
http://bbs.tianya.cn/post-763-693563-1.shtml
http://bbs.tianya.cn/post-763-693564-1.shtml
http://bbs.tianya.cn/post-763-693565-1.shtml
http://bbs.tianya.cn/post-763-693566-1.shtml
http://bbs.tianya.cn/post-763-693567-1.shtml
http://bbs.tianya.cn/post-763-693568-1.shtml
http://bbs.tianya.cn/post-763-693569-1.shtml
http://bbs.tianya.cn/post-763-693570-1.shtml
http://bbs.tianya.cn/post-763-693571-1.shtml
http://bbs.tianya.cn/post-763-693572-1.shtml
http://bbs.tianya.cn/post-763-693573-1.shtml
http://bbs.tianya.cn/post-763-693573-1.shtml
http://bbs.tianya.cn/post-763-693574-1.shtml
http://bbs.tianya.cn/post-763-693575-1.shtml
http://bbs.tianya.cn/post-763-693576-1.shtml
http://bbs.tianya.cn/post-763-693576-1.shtml
http://bbs.tianya.cn/post-763-693577-1.shtml
http://bbs.tianya.cn/post-763-693578-1.shtml
http://bbs.tianya.cn/post-763-693579-1.shtml
http://bbs.tianya.cn/post-763-693580-1.shtml
http://bbs.tianya.cn/post-763-693581-1.shtml
http://bbs.tianya.cn/post-763-693582-1.shtml
http://bbs.tianya.cn/post-763-693583-1.shtml
http://bbs.tianya.cn/post-763-693584-1.shtml
http://bbs.tianya.cn/post-763-693585-1.shtml
http://bbs.tianya.cn/post-763-693586-1.shtml
http://bbs.tianya.cn/post-763-693587-1.shtml
http://bbs.tianya.cn/post-763-693588-1.shtml
http://bbs.tianya.cn/post-763-693589-1.shtml
http://bbs.tianya.cn/post-763-693590-1.shtml
http://bbs.tianya.cn/post-763-693591-1.shtml
http://bbs.tianya.cn/post-763-693483-1.shtml
http://bbs.tianya.cn/post-763-693484-1.shtml
http://bbs.tianya.cn/post-763-693486-1.shtml
http://bbs.tianya.cn/post-763-693487-1.shtml
http://bbs.tianya.cn/post-763-693488-1.shtml
http://bbs.tianya.cn/post-763-693489-1.shtml
http://bbs.tianya.cn/post-763-693490-1.shtml
http://bbs.tianya.cn/post-763-693491-1.shtml
http://bbs.tianya.cn/post-763-693492-1.shtml
http://bbs.tianya.cn/post-763-693493-1.shtml
http://bbs.tianya.cn/post-763-693494-1.shtml
http://bbs.tianya.cn/post-763-693495-1.shtml
http://bbs.tianya.cn/post-763-693496-1.shtml
http://bbs.tianya.cn/post-763-693497-1.shtml
http://bbs.tianya.cn/post-763-693498-1.shtml
http://bbs.tianya.cn/post-763-693499-1.shtml
http://bbs.tianya.cn/post-763-693500-1.shtml
http://bbs.tianya.cn/post-763-693501-1.shtml
http://bbs.tianya.cn/post-763-693502-1.shtml
http://bbs.tianya.cn/post-763-693503-1.shtml
http://bbs.tianya.cn/post-763-693504-1.shtml
http://bbs.tianya.cn/post-763-693505-1.shtml
http://bbs.tianya.cn/post-763-693506-1.shtml
http://bbs.tianya.cn/post-763-693507-1.shtml
http://bbs.tianya.cn/post-763-693508-1.shtml
http://bbs.tianya.cn/post-763-693509-1.shtml
http://bbs.tianya.cn/post-763-693510-1.shtml
http://bbs.tianya.cn/post-763-693511-1.shtml
郑重声明：本站内容如果来自互联网及其他传播媒体，其版权均属原媒体及文章作者所有。转载目的在于传递更多信息及用于网络分享，并不代表本站赞同其观点和对其真实性负责，也不构成任何其他建议。
