Update documentation

Author: fmbenhassine

spring-batch-docs/modules/ROOT/nav.adoc

@@ -18,7 +18,6 @@
 *** xref:step/chunk-oriented-processing/restart.adoc[]
 *** xref:step/chunk-oriented-processing/configuring-skip.adoc[]
 *** xref:step/chunk-oriented-processing/retry-logic.adoc[]
-*** xref:step/chunk-oriented-processing/controlling-rollback.adoc[]
 *** xref:step/chunk-oriented-processing/transaction-attributes.adoc[]
 *** xref:step/chunk-oriented-processing/registering-item-streams.adoc[]
 *** xref:step/chunk-oriented-processing/intercepting-execution.adoc[]

spring-batch-docs/modules/ROOT/pages/job/advanced-meta-data.adoc

@@ -161,6 +161,36 @@ framework, it sets the status of the current
 `BatchStatus.STOPPED`, saves it, and does the same
 for the `JobExecution` before finishing.
 
+[[graceful-shutdown]]
+== Handling external interruption signals
+
+As of v6.0+, Spring Batch provides a `JobExecutionShutdownHook` that you can attach to the JVM runtime
+in order to intercept external interruption signals and gracefully stop the job execution:
+
+[source, java]
+----
+Thread springBatchHook = new JobExecutionShutdownHook(jobExecution, jobOperator);
+Runtime.getRuntime().addShutdownHook(springBatchHook);
+----
+
+A `JobExecutionShutdownHook` requires the job execution to track as well as a reference to a job operator
+that will be used to stop the execution.
+
+[[recover-job]]
+== Recovering a job
+
+If a graceful shutdown is not performed properly (ie the JVM is shutdown abruptly), Spring Batch will not
+have a chance to update the execution state correctly in order to restart the failed job execution. In this
+case, the job execution will stay in a `STARTED` state which is not restartable. In this case, it is possible
+to recover such a job execution with the `JobOperator` API:
+
+[source, java]
+----
+JobExecution jobExecution = ...; // get the job execution to recover
+jobOperator.recover(jobExecution);
+jobOperator.restart(jobExecution);
+----
+
 [[aborting-a-job]]
 == Aborting a Job
 

spring-batch-docs/modules/ROOT/pages/retry.adoc

@@ -16,7 +16,6 @@ Examples include remote calls to a web service that fails because of a network g
 ====
 As of version 2.2.0, the retry functionality was pulled out of Spring Batch.
 It is now part of a new library, https://github.com/spring-projects/spring-retry[Spring Retry].
-Spring Batch still relies on Spring Retry to automate retry operations within the framework.
-See the reference documentation of Spring Retry for details about
-key APIs and how to use them.
+As of v6.0, Spring Batch does *not* use Spring Retry to automate retry operations within the framework,
+and is now based on the https://docs.spring.io/spring-framework/reference/7.0/core/resilience.html#resilience-annotations-retryable[retry feature] provided by Spring Framework 7.0.
 ====

spring-batch-docs/modules/ROOT/pages/scalability.adoc

@@ -22,8 +22,10 @@ These break down into categories as well, as follows:
 
 * Multi-threaded Step (single-process)
 * Parallel Steps (single-process)
+* Local Chunking of Step (single-process)
 * Remote Chunking of Step (multi-process)
 * Partitioning a Step (single or multi-process)
+* Remote Step (multi-process)
 
 First, we review the single-process options. Then we review the multi-process options.
 
@@ -84,50 +86,7 @@ implementations. The simplest multi-threaded `TaskExecutor` is a
 The result of the preceding configuration is that the `Step` executes by reading, processing,
 and writing each chunk of items (each commit interval) in a separate thread of execution.
 Note that this means there is no fixed order for the items to be processed, and a chunk
-might contain items that are non-consecutive compared to the single-threaded case. In
-addition to any limits placed by the task executor (such as whether it is backed by a
-thread pool), the tasklet configuration has a throttle limit (default: 4).
-You may need to increase this limit to ensure that a thread pool is fully used.
-
-
-[tabs]
-====
-Java::
-+
-When using Java configuration, the builders provide access to the throttle limit, as
-follows:
-+
-.Java Configuration
-[source, java]
-----
-@Bean
-public Step sampleStep(TaskExecutor taskExecutor, JobRepository jobRepository, PlatformTransactionManager transactionManager) {
-	return new StepBuilder("sampleStep", jobRepository)
-				.<String, String>chunk(10).transactionManager(transactionManager)
-				.reader(itemReader())
-				.writer(itemWriter())
-				.taskExecutor(taskExecutor)
-				.throttleLimit(20)
-				.build();
-}
-----
-
-XML::
-+
-For example, you might increase the throttle-limit, as follows:
-+
-[source, xml]
-----
-<step id="loading"> <tasklet
-    task-executor="taskExecutor"
-    throttle-limit="20">...</tasklet>
-</step>
-----
-
-====
-
-
-
+might contain items that are non-consecutive compared to the single-threaded case.
 
 Note also that there may be limits placed on concurrency by any pooled resources used in
 your step, such as a `DataSource`. Be sure to make the pool in those resources at least
@@ -246,6 +205,63 @@ aggregating the exit statuses and transitioning.
 
 See the section on xref:step/controlling-flow.adoc#split-flows[Split Flows] for more detail.
 
+[[localChunking]]
+== Local Chunking
+
+Local chunking is a new feature that allows you to process chunks of items in parallel, locally within the same JVM using multiple threads.
+This is particularly useful when you have a large number of items to process and want to take advantage of multi-core processors.
+With local chunking, you can configure a chunk-oriented step to use multiple threads to process chunks of items concurrently.
+Each thread will read, process and write its own chunk of items independently, while the step will manage the overall execution and commit the results.
+
+This feature is possible by using the `ChunkMessageChannelItemWriter`, which is an item writer that submits chunk
+requests to local workers from a `TaskExecutor`:
+
+[source, java]
+----
+@Bean
+public ChunkTaskExecutorItemWriter<Vet> itemWriter(ChunkProcessor<Vet> chunkProcessor) {
+    ThreadPoolTaskExecutor taskExecutor = new ThreadPoolTaskExecutor();
+    taskExecutor.setCorePoolSize(4);
+    taskExecutor.setThreadNamePrefix("worker-thread-");
+    taskExecutor.setWaitForTasksToCompleteOnShutdown(true);
+    taskExecutor.afterPropertiesSet();
+    return new ChunkTaskExecutorItemWriter<>(chunkProcessor, taskExecutor);
+}
+----
+
+The `ChunkMessageChannelItemWriter` requires a `TaskExecutor` to process chunk concurrently
+as well as a `ChunkProcessor` to define what to do with each chunk. Here is an example of
+a chunk processor that writes each chunk of items to a relational database table:
+
+[source, java]
+----
+@Bean
+public ChunkProcessor<Vet> chunkProcessor(DataSource dataSource, TransactionTemplate transactionTemplate) {
+    String sql = "insert into vets (firstname, lastname) values (?, ?)";
+    JdbcBatchItemWriter<Vet> itemWriter = new JdbcBatchItemWriterBuilder<Vet>().dataSource(dataSource)
+        .sql(sql)
+        .itemPreparedStatementSetter((item, ps) -> {
+            ps.setString(1, item.firstname());
+            ps.setString(2, item.lastname());
+        })
+        .build();
+
+    return (chunk, contribution) -> transactionTemplate.executeWithoutResult(transactionStatus -> {
+        try {
+            itemWriter.write(chunk);
+            contribution.incrementWriteCount(chunk.size());
+            contribution.setExitStatus(ExitStatus.COMPLETED);
+        }
+        catch (Exception e) {
+            transactionStatus.setRollbackOnly();
+            contribution.setExitStatus(ExitStatus.FAILED.addExitDescription(e));
+        }
+    });
+}
+----
+
+You can find an example of this scaling technique in the https://github.com/spring-projects/spring-batch/tree/main/spring-batch-samples/src/main/java/org/springframework/batch/samples/chunking/local[Local Chunking Sample].
+
 [[remoteChunking]]
 == Remote Chunking
 
@@ -543,3 +559,62 @@ The following example shows how to define late binding in XML:
 
 ====
 
+[[remoteStep]]
+== Remote Step execution
+
+As of v6.0, Spring Batch provides support for remote step executions, allowing you to execute steps of a batch job on remote machines or clusters.
+This feature is particularly useful for large-scale batch processing scenarios where you want to distribute the workload across multiple nodes to improve performance and scalability.
+Remote step execution is provided by the `RemoteStep` class, which uses Spring Integration messaging channels to enable communication between the local job execution environment and the remote step executors.
+
+A `RemoteStep` is configured as a regular step by providing the remote step name and a messaging template to send step execution requests to remote workers:
+
+[source, java]
+----
+@Bean
+public Step step(MessagingTemplate messagingTemplate, JobRepository jobRepository) {
+    return new RemoteStep("step", "workerStep", jobRepository, messagingTemplate);
+}
+----
+
+On the worker side, you need to define the remote step to execute (`workerStep` in this example) and configure
+a Spring Integration flow to intercept step execution requests and invoke the `StepExecutionRequestHandler`:
+
+[source, java]
+----
+@Bean
+public Step workerStep(JobRepository jobRepository, JdbcTransactionManager transactionManager) {
+    return new StepBuilder("workerStep", jobRepository)
+        // define step logic
+        .build();
+}
+
+/*
+ * Configure inbound flow (requests coming from the manager)
+ */
+@Bean
+public DirectChannel requests() {
+    return new DirectChannel();
+}
+
+@Bean
+public IntegrationFlow inboundFlow(ActiveMQConnectionFactory connectionFactory, JobRepository jobRepository,
+        StepLocator stepLocator) {
+    StepExecutionRequestHandler stepExecutionRequestHandler = new StepExecutionRequestHandler();
+    stepExecutionRequestHandler.setJobRepository(jobRepository);
+    stepExecutionRequestHandler.setStepLocator(stepLocator);
+    return IntegrationFlow.from(Jms.messageDrivenChannelAdapter(connectionFactory).destination("requests"))
+        .channel(requests())
+        .handle(stepExecutionRequestHandler, "handle")
+        .get();
+}
+
+@Bean
+public StepLocator stepLocator(BeanFactory beanFactory) {
+    BeanFactoryStepLocator beanFactoryStepLocator = new BeanFactoryStepLocator();
+    beanFactoryStepLocator.setBeanFactory(beanFactory);
+    return beanFactoryStepLocator;
+}
+----
+
+You can find a complete example in the https://github.com/spring-projects/spring-batch/tree/main/spring-batch-samples/src/main/java/org/springframework/batch/samples/remotestep[Remote Step Sample].
+

spring-batch-docs/modules/ROOT/pages/step/chunk-oriented-processing/configuring-skip.adoc

@@ -21,18 +21,21 @@ The following Java example shows an example of using a skip limit:
 ----
 @Bean
 public Step step1(JobRepository jobRepository, PlatformTransactionManager transactionManager) {
+    int skipLimit = 10;
+    var skippableExceptions = Set.of(FlatFileParseException.class);
+    SkipPolicy skipPolicy = new LimitCheckingExceptionHierarchySkipPolicy(skippableExceptions, skipLimit);
+
 	return new StepBuilder("step1", jobRepository)
 				.<String, String>chunk(10).transactionManager(transactionManager)
 				.reader(flatFileItemReader())
 				.writer(itemWriter())
 				.faultTolerant()
-				.skipLimit(10)
-				.skip(FlatFileParseException.class)
+				.skipPolicy(skipPolicy)
 				.build();
 }
 ----
 +
-Note: The `skipLimit` can be explicitly set using the `skipLimit()` method. If not specified, the default skip limit is set to 10.
+Note: The `skipLimit` can be explicitly set using the `skipLimit()` method.
 
 XML::
 +
@@ -55,8 +58,6 @@ The following XML example shows an example of using a skip limit:
 
 ====
 
-
-
 In the preceding example, a `FlatFileItemReader` is used. If, at any point, a
 `FlatFileParseException` is thrown, the item is skipped and counted against the total
 skip limit of 10. Exceptions (and their subclasses) that are declared might be thrown
@@ -66,81 +67,8 @@ the step execution, but the limit applies across all skips. Once the skip limit
 reached, the next exception found causes the step to fail. In other words, the eleventh
 skip triggers the exception, not the tenth.
 
-One problem with the preceding example is that any other exception besides a
-`FlatFileParseException` causes the `Job` to fail. In certain scenarios, this may be the
-correct behavior. However, in other scenarios, it may be easier to identify which
-exceptions should cause failure and skip everything else.
-
-[tabs]
+[NOTE]
 ====
-Java::
-+
-The following Java example shows an example excluding a particular exception:
-+
-.Java Configuration
-[source, java]
-----
-@Bean
-public Step step1(JobRepository jobRepository, PlatformTransactionManager transactionManager) {
-	return new StepBuilder("step1", jobRepository)
-				.<String, String>chunk(10).transactionManager(transactionManager)
-				.reader(flatFileItemReader())
-				.writer(itemWriter())
-				.faultTolerant()
-				.skipLimit(10)
-				.skip(Exception.class)
-				.noSkip(FileNotFoundException.class)
-				.build();
-}
-----
-+
-Note: The `skipLimit` can be explicitly set using the `skipLimit()` method. If not specified, the default skip limit is set to 10.
-
-XML::
-+
-The following XML example shows an example excluding a particular exception:
-+
-.XML Configuration
-[source, xml]
-----
-<step id="step1">
-    <tasklet>
-        <chunk reader="flatFileItemReader" writer="itemWriter"
-               commit-interval="10" skip-limit="10">
-            <skippable-exception-classes>
-                <include class="java.lang.Exception"/>
-                <exclude class="java.io.FileNotFoundException"/>
-            </skippable-exception-classes>
-        </chunk>
-    </tasklet>
-</step>
-----
-
-====
-
-
-
-By identifying `java.lang.Exception` as a skippable exception class, the configuration
-indicates that all `Exceptions` are skippable. However, by "`excluding`"
-`java.io.FileNotFoundException`, the configuration refines the list of skippable
-exception classes to be all `Exceptions` __except__ `FileNotFoundException`. Any excluded
-exception class is fatal if encountered (that is, they are not skipped).
-
-For any exception encountered, the skippability is determined by the nearest superclass
-in the class hierarchy. Any unclassified exception is treated as 'fatal'.
-
-
-[tabs]
-====
-Java::
-+
-The order of the `skip` and `noSkip` method calls does not matter.
-
-XML::
-+
-The order of the `<include/>` and `<exclude/>` elements does not matter.
-
+The skip limit applies to all skips (read, process and write).
 ====
 
-
-