來(lái)源:https://www.cnblogs.com/barrywxx/p/10700283.html 使用POI或JXLS導(dǎo)出大數(shù)據(jù)量(百萬(wàn)級(jí))Excel報(bào)表常常面臨兩個(gè)問(wèn)題: 1. 服務(wù)器內(nèi)存溢出; 2. 一次從數(shù)據(jù)庫(kù)查詢出這么大數(shù)據(jù),查詢緩慢。 當(dāng)然也可以分頁(yè)查詢出數(shù)據(jù),分別生成多個(gè)Excel打包下載,但這種生成還是很緩慢。 大數(shù)據(jù)量導(dǎo)入請(qǐng)參考:Java實(shí)現(xiàn)大批量數(shù)據(jù)導(dǎo)入導(dǎo)出(100W以上) -(一)導(dǎo)入 那么如何解決呢? 我們可以借助XML格式利用模板替換,分頁(yè)查詢出數(shù)據(jù)從磁盤(pán)寫(xiě)入XML,最終會(huì)以Excel多sheet形式生成。親測(cè)2400萬(wàn)行數(shù)據(jù),生成Excel文件4.5G,總耗時(shí)1.5分鐘。 https://www.cnblogs.com/barrywxx/p/10700283.html 我利用StringTemplate模板解析技術(shù)對(duì)XML模板進(jìn)行填充。當(dāng)然也可以使用FreeMarker, Velocity等Java模板技術(shù)實(shí)現(xiàn)。 首先引入StringTemplate所需Jar包: 使用技術(shù)為 stringTemplate pom.xml: antlr antlr 2.7.7 org.antlr stringtemplate 3.2.1 首先準(zhǔn)備導(dǎo)出Excel模板,然后打開(kāi)-》另存為-》選擇格式為XML,然后用文本打開(kāi)XML,提取XML頭模板(head.st可通用),數(shù)據(jù)體模板(boday.st): head.st可通用: <workbook xmlns="urn:schemas-microsoft-com:office:spreadsheet" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns:ss="urn:schemas-microsoft-com:office:spreadsheet" xmlns:html="http://www./TR/REC-html40"> 1996-12-17T01:32:42Z 2013-08-02T09:21:24Z 11.9999 4530 8505 480 120 False False 生成大數(shù)據(jù)量Excel類(lèi): ExcelGenerator: package test.exportexcel; import org.antlr.stringtemplate.StringTemplate; import org.antlr.stringtemplate.StringTemplateGroup; import test.exportexcel.bean.Row; import test.exportexcel.bean.Worksheet; import java.io.*; import java.util.ArrayList; import java.util.List; import java.util.Random; /** * 類(lèi)功能描述:generator big data Excel * * @author WangXueXing create at 19-4-13 下午10:23 * @version 1.0.0 */ public class ExcelGenerator { public static void main(String[] args) throws FileNotFoundException{ ExcelGenerator template = new ExcelGenerator(); template.output2(); } /** * 生成數(shù)據(jù)量大的時(shí)候,該方法會(huì)出現(xiàn)內(nèi)存溢出 * @throws FileNotFoundException */ public void output1() throws FileNotFoundException{ StringTemplateGroup stGroup = new StringTemplateGroup("stringTemplate"); StringTemplate st4 = stGroup.getInstanceOf("test/exportexcel/template/test"); Listworksheets = new ArrayList<>(); File file = new File("/home/barry/data/output.xls"); PrintWriter writer = new PrintWriter(new BufferedOutputStream(new FileOutputStream(file))); for(int i=0;i<30;i++){ Worksheet worksheet = new Worksheet(); worksheet.setSheet("第"+(i+1)+"頁(yè)"); Listrows = new ArrayList<>(); for(int j=0;j<6000;j++){ Row row = new Row(); row.setName1("zhangzehao"); row.setName2(""+j); row.setName3(i+" "+j); rows.add(row); } worksheet.setRows(rows); worksheets.add(worksheet); } st4.setAttribute("worksheets", worksheets); writer.write(st4.toString()); writer.flush(); writer.close(); System.out.println("生成excel完成"); } /** * 該方法不管生成多大的數(shù)據(jù)量,都不會(huì)出現(xiàn)內(nèi)存溢出,只是時(shí)間的長(zhǎng)短 * 經(jīng)測(cè)試,生成2400萬(wàn)數(shù)據(jù),2分鐘內(nèi),4.5G大的文件,打開(kāi)大文件就看內(nèi)存是否足夠大了 * 數(shù)據(jù)量小的時(shí)候,推薦用JXLS的模板技術(shù)生成excel文件,誰(shuí)用誰(shuí)知道,大數(shù)據(jù)量可以結(jié)合該方法使用 * @throws FileNotFoundException */ public void output2() throws FileNotFoundException{ long startTimne = System.currentTimeMillis(); StringTemplateGroup stGroup = new StringTemplateGroup("stringTemplate"); //寫(xiě)入excel文件頭部信息 StringTemplate head = stGroup.getInstanceOf("test/exportexcel/template/head"); File file = new File("/home/barry/data/output.xls"); PrintWriter writer = new PrintWriter(new BufferedOutputStream(new FileOutputStream(file))); writer.print(head.toString()); writer.flush(); int sheets = 400; //excel單表最大行數(shù)是65535 int maxRowNum = 60000; //寫(xiě)入excel文件數(shù)據(jù)信息 for(int i=0;i<sheets;i++){< p=""> StringTemplate body = stGroup.getInstanceOf("test/exportexcel/template/body"); Worksheet worksheet = new Worksheet(); worksheet.setSheet(" "+(i+1)+" "); worksheet.setColumnNum(3); worksheet.setRowNum(maxRowNum); Listrows = new ArrayList<>(); for(int j=0;j<maxrownum;j++){< p=""> Row row = new Row(); row.setName1(""+new Random().nextInt(100000)); row.setName2(""+j); row.setName3(i+""+j); rows.add(row); } worksheet.setRows(rows); body.setAttribute("worksheet", worksheet); writer.print(body.toString()); writer.flush(); rows.clear(); rows = null; worksheet = null; body = null; Runtime.getRuntime().gc(); System.out.println("正在生成excel文件的 sheet"+(i+1)); } //寫(xiě)入excel文件尾部 writer.print(""); writer.flush(); writer.close(); System.out.println("生成excel文件完成"); long endTime = System.currentTimeMillis(); System.out.println("用時(shí)="+((endTime-startTimne)/1000)+"秒"); } } 定義JavaBean: WorkSheet.java: package test.exportexcel.bean; import java.util.List; /** * 類(lèi)功能描述:Excel sheet Bean * * @author WangXueXing create at 19-4-13 下午10:21 * @version 1.0.0 */ public class Worksheet { private String sheet; private int columnNum; private int rowNum; private Listrows; public String getSheet() { return sheet; } public void setSheet(String sheet) { this.sheet = sheet; } public ListgetRows() { return rows; } public void setRows(Listrows) { this.rows = rows; } public int getColumnNum() { return columnNum; } public void setColumnNum(int columnNum) { this.columnNum = columnNum; } public int getRowNum() { return rowNum; } public void setRowNum(int rowNum) { this.rowNum = rowNum; } } Row.java: package test.exportexcel.bean; /** * 類(lèi)功能描述:Excel row bean * * @author WangXueXing create at 19-4-13 下午10:22 * @version 1.0.0 */ public class Row { private String name1; private String name2; private String name3; public String getName1() { return name1; } public void setName1(String name1) { this.name1 = name1; } public String getName2() { return name2; } public void setName2(String name2) { this.name2 = name2; } public String getName3() { return name3; } public void setName3(String name3) { this.name3 = name3; } } 另附實(shí)現(xiàn)源碼: https://files.cnblogs.com/files/barrywxx/exportexcel.zip ☆ 往期精彩 ☆ |
|