spring-projects / spring-projects/spring-batch

Spring batch fails to restart

Open
#3,771 10 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

status: waiting-for-reporter type: bug
Dominant language
Java
Stars
3k
Forks
2.5k
Avg merge
6d 53m
Merged PRs (30d)
3

Description

Using spring batch 2.3.1.RELEASE.
Spring batch fails to restart with error

java.lang.ClassCastException: Value for key=[SimpleStepExecutionSplitter.GRID_SIZE] is not of type: [class java.lang.Long], it is [(class java.lang.Integer)6]
at org.springframework.batch.item.ExecutionContext.readAndValidate(ExecutionContext.java:290) ~[spring-batch-infrastructure-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.batch.item.ExecutionContext.getLong(ExecutionContext.java:189) ~[spring-batch-infrastructure-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.batch.item.ExecutionContext.getLong(ExecutionContext.java:206) ~[spring-batch-infrastructure-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.batch.core.partition.support.SimpleStepExecutionSplitter.getContexts(SimpleStepExecutionSplitter.java:211) ~[spring-batch-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.batch.core.partition.support.SimpleStepExecutionSplitter.split(SimpleStepExecutionSplitter.java:178) ~[spring-batch-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.batch.core.partition.support.AbstractPartitionHandler.handle(AbstractPartitionHandler.java:59) ~[spring-batch-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.batch.core.partition.support.PartitionStep.doExecute(PartitionStep.java:106) ~[spring-batch-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:208) ~[spring-batch-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148) ~[spring-batch-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:68) ~[spring-batch-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:68) ~[spring-batch-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:169) ~[spring-batch-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:144) ~[spring-batch-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.java:137) ~[spring-batch-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:319) ~[spring-batch-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:147) ~[spring-batch-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) ~[spring-core-5.2.7.RELEASE.jar:5.2.7.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:140) ~[spring-batch-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) ~[spring-aop-5.2.7.RELEASE.jar:5.2.7.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.2.7.RELEASE.jar:5.2.7.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.2.7.RELEASE.jar:5.2.7.RELEASE]
at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127) ~[spring-batch-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.2.7.RELEASE.jar:5.2.7.RELEASE]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.2.7.RELEASE.jar:5.2.7.RELEASE]
at com.sun.proxy.$Proxy46.run(Unknown Source) ~[na:na]
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.execute(JobLauncherApplicationRunner.java:199) ~[spring-boot-autoconfigure-2.3.1.RELEASE.jar:2.3.1.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.executeLocalJobs(JobLauncherApplicationRunner.java:173) ~[spring-boot-autoconfigure-2.3.1.RELEASE.jar:2.3.1.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.launchJobFromProperties(JobLauncherApplicationRunner.java:160) ~[spring-boot-autoconfigure-2.3.1.RELEASE.jar:2.3.1.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.run(JobLauncherApplicationRunner.java:155) ~[spring-boot-autoconfigure-2.3.1.RELEASE.jar:2.3.1.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.run(JobLauncherApplicationRunner.java:150) ~[spring-boot-autoconfigure-2.3.1.RELEASE.jar:2.3.1.RELEASE]
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:786) ~[spring-boot-2.3.1.RELEASE.jar:2.3.1.RELEASE]
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:776) ~[spring-boot-2.3.1.RELEASE.jar:2.3.1.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:322) ~[spring-boot-2.3.1.RELEASE.jar:2.3.1.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1237) ~[spring-boot-2.3.1.RELEASE.jar:2.3.1.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1226) ~[spring-boot-2.3.1.RELEASE.jar:2.3.1.RELEASE]
at com.springbatch.example.Application.main(Application.java:29) ~[classes/:na]

Code to reproduce:

@Configuration
@EnableBatchProcessing
@Import(DataSourceConfig.class)
public class JobConfig {


    private final JobBuilderFactory jobBuilderFactory;

    private final StepBuilderFactory stepBuilderFactory;


    public JobConfig(
            final JobBuilderFactory jobBuilderFactory,
            final StepBuilderFactory stepBuilderFactory) {
        this.jobBuilderFactory = jobBuilderFactory;
        this.stepBuilderFactory = stepBuilderFactory;
    }

    @Bean
    public Job job(final Step parse) {
        return jobBuilderFactory.get("test")
                .flow(parse)
                .end()
                .build();
    }

    @Bean
    public Step parse(final Step parserWorker, final TstPartitioner partitioner) {
        return stepBuilderFactory.get("parser")
                .partitioner("parser", partitioner)
                .step(parserWorker)
                .taskExecutor(taskExecutor())
                .build();
    }

    @Bean
    public Step parserWorker(final ItemReader<Integer> reader, LoggingItemWriter loggingItemWriter) {
        return stepBuilderFactory.get("process")
                .<Integer, Integer>chunk(10)
                .reader(reader)
                .writer(loggingItemWriter)
                .build();
    }

    @Bean
    public ListItemReader<Integer> reader() {
        return new ListItemReader<>(Arrays.asList(1, 2, 3, 4, 5, 6));
    }


    @Bean
    public TaskExecutor taskExecutor() {
        return new SimpleAsyncTaskExecutor();
    }

@Component
@Slf4j
public class LoggingItemWriter implements ItemWriter<Integer> {


    @Override
    public void write(List<? extends Integer> list) {

        log.info("Writing : {}", list);
        //throw new RuntimeException(); using this as restart point. Uncomment this for normal run and comment it for restart.
    }
}

In this example Partitioner  doesnt have any importance, but i just wanted to use it because it is easy to reproduce with dummy partioner attached to step
@Component
public class TstPartitioner implements Partitioner {

    @Override
    public Map<String, ExecutionContext> partition(int gridSize) {
        Map<String, ExecutionContext> map = new HashMap<>();
        int i = 1;
        File dir = null;
        try {
            dir = new ClassPathResource("").getFile();
            FileFilter fileFilter = new WildcardFileFilter(Arrays.asList("*.csv"));
            File[] files = dir.listFiles(fileFilter);
            if (files != null)
                for (File file : files) {
                    ExecutionContext context = new ExecutionContext();
                    context.putString("file", file.getAbsolutePath());
                    map.put("worker" + i, context);
                    i++;
                }
            return map;
        } catch (IOException e) {
            e.printStackTrace();
        }
        return map;
    }
}


@Component
public class BatchConfigurer extends DefaultBatchConfigurer {

    private final DataSource dataSource;

    public BatchConfigurer(DataSource dataSource) {
        this.dataSource = dataSource;
    }

    @Override
    @SuppressWarnings("findbugs:NP_NULL_ON_SOME_PATH_FROM_RETURN_VALUE")
    public JobRepository createJobRepository() throws Exception {
        JobRepositoryFactoryBean jobRepositoryFactoryBean = new JobRepositoryFactoryBean();
        jobRepositoryFactoryBean.setDataSource(dataSource);
        jobRepositoryFactoryBean.setTransactionManager(getTransactionManager());
        jobRepositoryFactoryBean.setIsolationLevelForCreate("ISOLATION_DEFAULT");
        jobRepositoryFactoryBean.setValidateTransactionState(true);
        jobRepositoryFactoryBean.setSerializer(serializer());
        jobRepositoryFactoryBean.afterPropertiesSet();
        return jobRepositoryFactoryBean.getObject();
    }

    @Override
    @SuppressWarnings("findbugs:NP_NULL_ON_SOME_PATH_FROM_RETURN_VALUE")
    public JobExplorer createJobExplorer() throws Exception {
        JobExplorerFactoryBean jobExplorerFactoryBean = new JobExplorerFactoryBean();
        jobExplorerFactoryBean.setDataSource(dataSource);
        jobExplorerFactoryBean.setSerializer(serializer());
        jobExplorerFactoryBean.afterPropertiesSet();

        return jobExplorerFactoryBean.getObject();
    }

In my production code, i have many objects stored in execuetion context , which i didnt include in this example
@Bean
    private Jackson2ExecutionContextStringSerializer serializer() {
        Jackson2ExecutionContextStringSerializer serializer = new Jackson2ExecutionContextStringSerializer();
        ObjectMapper objectMapper = new ObjectMapper();
        serializer.setObjectMapper(objectMapper);
        return serializer;
    }
}

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the restart failure with the provided JobConfig, partitioner, BatchConfigurer, and Jackson2ExecutionContextStringSerializer setup. Trace SimpleStepExecutionSplitter.getContexts through ExecutionContext.getLong and readAndValidate, then verify that a failed partitioned job can restart without the reported ClassCastException.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spring, spring-boot
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.