I have a spring batch job in which I set the chunk size to 1000 and the reader in that job is a JpaPagingItemReader
.
In the reader I set the page size to 20
. Does this mean that every chuck of 1000 items read the reader must take out of the db 20 items?
If not what is the difference between them?
With your current configuration, if your every read item makes to writer ( i.e. if it doesn't get filtered out in processor ) then you will need 1000/20 = 50 database reads to reach a chunk size i.e. when you actually call the writer for writing.
Spring Batch holds processed items in memory till you reach a chunk size and holding items costs memory.
Your current configuration is holding data in memory and making unnecessary database calls while we wish to reduce both of those things.
So your configuration needs to be reverse of what you are doing i.e. increase reader page size to a minimum equal to chunk size / commit interval or more so data once read gets processed in small small chunks then you go again and read database .
So as you have noticed till current write up, conceptually these are unrelated concepts - reader page size is to minimize database calls ( and this concept is not a spring batch concept but reader specific - if its not a paging reader, this concept doesn't come into picture ) while chunk size is about committing processed data in small small chunks to reduce memory foot print.
이 기사는 인터넷에서 수집됩니다. 재 인쇄 할 때 출처를 알려주십시오.
침해가 발생한 경우 연락 주시기 바랍니다[email protected] 삭제
몇 마디 만하겠습니다