mongodb批量处理

mongodb支持批量插入。

1.使用Java mongodb api

查看源码com.mongodb.MongoCollectionImpl，有两个方法

@Override public void insertMany(final List<? extends TDocument> documents) { insertMany(documents, new InsertManyOptions()); } @Override public void insertMany(final List<? extends TDocument> documents, final InsertManyOptions options) { notNull("documents", documents); List<InsertRequest> requests = new ArrayList<InsertRequest>(documents.size()); for (TDocument document : documents) { if (document == null) { throw new IllegalArgumentException("documents can not contain a null value"); } if (getCodec() instanceof CollectibleCodec) { document = ((CollectibleCodec<TDocument>) getCodec()).generateIdIfAbsentFromDocument(document); } requests.add(new InsertRequest(documentToBsonDocument(document))); } executor.execute(new MixedBulkWriteOperation(namespace, requests, options.isOrdered(), writeConcern) .bypassDocumentValidation(options.getBypassDocumentValidation())); }

insertMany(final List<? extends TDocument> documents) 默认使用 private boolean ordered = true;即有序插入。

insertMany(final List<? extends TDocument> documents, final InsertManyOptions options)调用者本身设置。

ordered属性有什么用？

/** * Gets whether the documents should be inserted in the order provided, stopping on the first failed insertion. The default is true. * If false, the server will attempt to insert all the documents regardless of an failures. * * @return whether the the documents should be inserted in order */
    public boolean isOrdered() { return ordered; }

大概意思是：

true：按提供的顺序插入文档，并在首次插入失败时中止。 （按顺序插入文档，遇到失败后中止。以前已经插入成功的不返回，失败及失败以后的不插入。）

false: 服务器将尝试插入全部文档，而不考虑失败。（只要能插入成功的，都存储进数据库）

 

2.spring-data-mongodb

org.springframework.data.mongodb.core.MongoTemplate

如下是该方法的源码和使用案例：

/* * (non-Javadoc) * @see org.springframework.data.mongodb.core.ExecutableInsertOperation#bulkOps(org.springframework.data.mongodb.core.BulkMode, java.lang.String) */
    public BulkOperations bulkOps(BulkMode bulkMode, String collectionName) { return bulkOps(bulkMode, null, collectionName); }

public int batchInsertStudents() {

 BulkWriteResult result = null;

try { 　　List<Student> documents = new ArrayList<>();
      　　String collectionName = "myTestCol"; 　　BulkOperations bulkOp = this.mongoTemplate.bulkOps(BulkMode.UNORDERED, collectionName);
      　　for(int i = 502610; i< 2000000; i++) { 　　Student student = new Student(); 　　student.setId(String.valueOf(i)); 　　student.setAge(i); 　　student.setGender("男"); 　　student.setName("李三"+ i); 　　documents.add(student); 　　} 　　bulkOp.insert(documents); result = bulkOp.execute();
 }catch (DuplicateKeyException e) { System.out.println("**********" + e.getMessage()); 　} return result; }

有两种模式：

/** * Mode for bulk operation. **/
    enum BulkMode { /** Perform bulk operations in sequence. The first error will cancel processing. */ ORDERED, /** Perform bulk operations in parallel. Processing will continue on errors. */ UNORDERED };

与以前的order（true|false）相对应。