如何在Spring Batch中使用ClassifierCompositeItemProcessor并将数据写入同一表以进行插入和Upsert?

Pra_A

我通过链接-https: //github.com/spring-projects/spring-batch/blob/master/spring-batch-infrastructure/src/test/java/org/springframework/batch/item/support/ClassifierCompositeItemProcessorTests。 java,但并没有从中脱颖而出。

我正在尝试将ETL Informatica映射逻辑替换为Batch。我期待以分离出Status=IStatus=U成单独的(个人)处理器,然后进一步进行查找和按摩的数据,然后直接写入这些记录到表中为状态= I和用于状态= U,则执行另一个复杂的逻辑(如查找,按摩以及匹配和合并逻辑),然后将这些记录再次向上插入同一张表中。

我试图做POC,我想在其中分离记录中的记录

CustomerClassifier.java

public class CustomerClassifier implements Classifier<Customer, ItemProcessor<Customer, Customer>> {

    private ItemProcessor<Customer, Customer> insertCustomerProcessor;
    private ItemProcessor<Customer, Customer> updateCustomerProcessor;
    
    public CustomerClassifier(ItemProcessor<Customer, Customer> evenCustomerProcessor, ItemProcessor<Customer, Customer> oddCustomerProcessor) {
        this.insertCustomerProcessor= insertCustomerProcessor;
        this.updateCustomerProcessor= updateCustomerProcessor;
    }
    
    @Override
    public ItemProcessor<Customer, Customer> classify(Customer customer) {
        return customer.getStatus().equals("I") ? insertCustomerProcessor : updateCustomerProcessor;
    }
}

OddCustomerProcessor.java

public class OddCustomerProcessor implements ItemProcessor<Customer, Customer> {

    @Override
    public Customer process(Customer item) throws Exception {
        Customer customer = new Customer();
        // Perform some msaaging and lookups here
        customer.setId(item.getId());
        customer.setFirstName(item.getFirstName());
        customer.setLastName(item.getLastName());
        customer.setBirthdate(item.getBirthdate());
        customer.setStatus(item.getStatus());
        return customer;
    }
}

EvenCustomerProcessor.java

public class EvenCustomerProcessor implements ItemProcessor<Customer, Customer> {

    @Override
    public Customer process(Customer item) throws Exception {
        Customer customer = new Customer();
        // Perform some msaaging and lookups here
        customer.setId(item.getId());
        customer.setFirstName(item.getFirstName());
        customer.setLastName(item.getLastName());
        customer.setBirthdate(item.getBirthdate());
        customer.setStatus(item.getStatus());
        return customer;
    }
}

CustomLineAggregator.java

public class CustomLineAggregator implements LineAggregator<Customer> {
    private ObjectMapper objectMapper = new ObjectMapper();

    @Override
    public String aggregate(Customer item) {
        try {
            return objectMapper.writeValueAsString(item);
        } catch (Exception e) {
            throw new RuntimeException("Unable to serialize Customer", e);
        }
    }
}

客户.java

@Data
@AllArgsConstructor
@Builder
@NoArgsConstructor
public class Customer {
    private Long id;
    private String firstName;
    private String lastName;
    private String birthdate;
    private String status;
}

错误-

ClassifierCompositeItemProcessor <Customer,Customer>类型的方法setClassifier(Classifier <?super Customer,ItemProcessor <?,?Extended Customer >>)不适用于参数(CustomerClassifier)

组态

@Configuration
public class JobConfiguration {
    @Autowired
    private JobBuilderFactory jobBuilderFactory;
    
    @Autowired
    private StepBuilderFactory stepBuilderFactory;
    
    @Autowired
    private DataSource dataSource;
    
    @Bean
    public JdbcPagingItemReader<Customer> customerPagingItemReader(){
        // reading database records using JDBC in a paging fashion
        JdbcPagingItemReader<Customer> reader = new JdbcPagingItemReader<>();
        reader.setDataSource(this.dataSource);
        reader.setFetchSize(1000);
        reader.setRowMapper(new CustomerRowMapper());
        
        // Sort Keys
        Map<String, Order> sortKeys = new HashMap<>();
        sortKeys.put("id", Order.ASCENDING);
        
        // MySQL implementation of a PagingQueryProvider using database specific features.
        MySqlPagingQueryProvider queryProvider = new MySqlPagingQueryProvider();
        queryProvider.setSelectClause("id, firstName, lastName, birthdate");
        queryProvider.setFromClause("from customer");
        queryProvider.setSortKeys(sortKeys);
        
        reader.setQueryProvider(queryProvider);
        
        return reader;
    }
    
    @Bean
    public EvenCustomerProcessor evenCustomerProcessor() {
        return new EvenCustomerProcessor();
    }
    
    @Bean
    public OddCustomerProcessor oddCustomerProcessor() {
        return new OddCustomerProcessor();
    }
    
    @Bean 
    public JdbcBatchItemWriter<Customer> customerItemWriter(){
    JdbcBatchItemWriter<Customer> batchItemWriter = new JdbcBatchItemWriter<>();
    batchItemWriter.setDataSource(dataSource);
    batchItemWriter.setSql(""); // Query Goes here
    return batchItemWriter;
}
    
    @Bean
    public ClassifierCompositeItemProcessor<Customer, Customer> classifierCustomerCompositeItemProcessor() throws Exception{
        ClassifierCompositeItemProcessor<Customer, Customer> itemProcessor = new ClassifierCompositeItemProcessor<>();
        itemProcessor.setClassifier(new CustomerClassifier(evenCustomerProcessor(), oddCustomerProcessor()));
    }
    
    @Bean
    public Step step1() throws Exception {
        return stepBuilderFactory.get("step1")
                .<Customer, Customer> chunk(10)
                .reader(customerPagingItemReader())
                .processor(classifierCustomerCompositeItemProcessor())
                .writer(customerItemWriter())
                .build();
    }
    
    @Bean
    public Job job() throws Exception {
        return jobBuilderFactory.get("job")
                .start(step1())
                .build();
    }
}
Mahmoud Ben Hassine

您可以删除CustomerClassifier和定义复合物料处理器,如下所示:

@Bean
public ClassifierCompositeItemProcessor<Customer, Customer> classifierCustomerCompositeItemProcessor(
        EvenCustomerProcessor evenCustomerProcessor,
        OddCustomerProcessor oddCustomerProcessor
) {
    ClassifierCompositeItemProcessor<Customer, Customer> itemProcessor = new ClassifierCompositeItemProcessor<>();
    itemProcessor.setClassifier(new Classifier<Customer, ItemProcessor<?, ? extends Customer>>() {
        @Override
        public ItemProcessor<?, ? extends Customer> classify(Customer customer) {
            return customer.getStatus().equals("I") ? evenCustomerProcessor : oddCustomerProcessor;
        }
    });
    return itemProcessor;
}

然后,如下更新步骤定义:

@Bean
public Step step1() throws Exception {
    return stepBuilderFactory.get("step1")
            .<Customer, Customer> chunk(10)
            .reader(customerPagingItemReader())
            .processor(classifierCustomerCompositeItemProcessor(evenCustomerProcessor(), oddCustomerProcessor()))
            .writer(customerItemWriter())
            .build();
}

本文收集自互联网,转载请注明来源。

如有侵权,请联系 [email protected] 删除。

编辑于
0

我来说两句

0 条评论
登录 后参与评论

相关文章