Post

(Spring Batch) 병렬 처리 방법 모음

Spring Batch에서 지원하는 배치 병렬 처리 방식

  • AsyncItemProcessor / AsyncItemWriter → 한 step 내에서 processor만 병렬 수행해야 할 때
  • Multi-threaded Step → 한 step 내에서 reader, processor, writer를 chunk 단위로 병렬 수행해야 할 때
  • Parallel Steps → 여러 step들을 병렬로 수행해야 할 때
  • Externalizing Batch Process Execution → 외부 remote 서버에서 병렬 수행 필요할 때. (master-worker 모델)

    • Remote Chunking of Step→ 스텝 내의 Processor, Writer가 무거운 작업이라 외부 remote 서버들에서 병렬로 돌리고 싶을 때
    • Remote Partitioning (Partitioning a Step)**→ Reader 부터가 무거운 작업이라 Reader를 포함한 Step 전체를 쪼개 외부 remote 서버들에서 병렬로 돌리고 싶을 때. *꼭 외부가 아니라 local thread로도 가능

https://docs.spring.io/spring-batch/docs/current/reference/html/scalability.html

AsyncItemProcessor / AsyncItemWriter

Multi-threaded Step

TaskExecutor의 종류

https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/core/task/TaskExecutor.html

- All Known Implementing Classes 부분 참조

https://www.baeldung.com/java-threadpooltaskexecutor-core-vs-max-poolsize

Parallel Steps

Remote Chunking

https://godekdls.github.io/Spring%20Batch/scalingandparallelprocessing/#73-remote-chunking

https://godekdls.github.io/Spring%20Batch/springbatchintegration/#remote-chunking

Remote Partitioning

https://godekdls.github.io/Spring%20Batch/scalingandparallelprocessing/#74-partitioning

https://godekdls.github.io/Spring%20Batch/springbatchintegration/#remote-partitioning

참고

https://backtony.github.io/spring/2022-01-29-spring-batch-11/

https://godekdls.github.io/Spring%20Batch/scalingandparallelprocessing/

https://docs.spring.io/spring-batch/docs/4.2.x/reference/html/index-single.html#scalability

This post is licensed under CC BY 4.0 by the author.