其他 - 使用分区的Spring Batch多线程

我的问题陈述是-我必须将多个文件传递给spring批处理读取器，并且读取器以parellel运行。如果我们使用grid-size = 100，则将有100个不合逻辑的线程。解决此问题的方法是什么，即处理线程数量有限的许多文件。

@Bean
    public Step orderStep1() throws IOException {
        return stepBuilderFactory.get("orderStep1")
                .partitioner("slaveStep", partitioner())
                .step(slaveStep())
                .gridSize(100)
                .taskExecutor(taskExecutor())
                 .build();
    }

Task executor will be
@Bean
    public TaskExecutor taskExecutor() {
        SimpleAsyncTaskExecutor taskExecutor = new SimpleAsyncTaskExecutor();
        return taskExecutor;
    }

partitoner will be
public Map<String, ExecutionContext> partition(int gridSize) {

      Map<String, ExecutionContext> partitionData = new HashMap<String, ExecutionContext>();  
      for (int i = 0; i < gridSize; i++) {
            ExecutionContext executionContext = new ExecutionContext();
            executionContext.putString("file", fileList.get(i)); //passing filelist
            executionContext.putString("name", "Thread" + i);
           partitionData.put("partition: " + i, executionContext);
        }

        return partitionData;
    }

并使用stepExecutionContext动态传递文件

查看更多