public void filterLogins() throws IOException, SQLException{
for(int i = 0; i<200; ++i){
BufferedReader bufferedReader = new BufferedReader(new FileReader(folder + String.format("\\data\\part-%05d", i)));
long prev_id = 0;
boolean contains = false;
while(bufferedReader.ready()){ //very big file
String line = bufferedReader.readLine();
Login login = new Login(line);
if ( login.userId == prev_id && !contains )
continue;
if ( samples.contains(login.userId) ){
mysql.execute("INSERT INTO ..."); // i think it doesn't matter in this case
contains = true;
}else{
contains = false;
}
prev_id = login.userId;
}
bufferedReader.close();
System.out.println((double)i/2.0);
}
}
此功能可以长时间工作,因为数据是更大的文件。2小时前,这是OutOfMemory异常的美眉
mysql是的实例
public class MySQLHandler {
private Connection connection = null;
MySQLHandler() throws ClassNotFoundException, SQLException{
try{
Class.forName("com.mysql.jdbc.Driver");
connection = DriverManager.getConnection("jdbc:mysql://localhost:3306/database", "root", "");
}catch (ClassNotFoundException e){
System.out.println("Failed jdbc driver load.");
throw e;
}
}
public void close() throws SQLException{
if ( connection != null)
connection.close();
}
public boolean execute(String sql) throws SQLException{
Statement statement = connection.createStatement();
return statement.execute(sql);
}
}
登录它只是带有数据的类。(id,名称,值等)。
似乎您是在每次创建Statement,而不是在循环中关闭该语句,这会导致内存泄漏。执行完成后,关闭语句。
public boolean execute(String sql) throws SQLException{
Statement statement = connection.createStatement();
return statement.execute(sql);
}
喜欢
public boolean execute(String sql) throws SQLException{
Statement statement = null;
try {
statement = connection.createStatement();
return statement.execute(sql);
}finaly{
if (statement != null) statement.close();
}
UPDATE
作为@Holger在评论中提到,如果您使用的是JDK 7或更高,那么你可以使用的try-与资源喜欢以下
public boolean execute(String sql) throws SQLException{
try(Statement s=connection.createStatement()) {
return s.execute(sql);
}
}
本文收集自互联网,转载请注明来源。
如有侵权,请联系 [email protected] 删除。
我来说两句