Hi All,
I am trying to generate reports in my project using spring batch. I have more than 4 million records in my database. Earlier I have set commit-interval as 1 but after reading your response, I have set commit-interval as 100 and page-size as 100 but still, it's taking more than 2 days to pull the reports. It is taking 3 to 4 minutes to get the records and process the records and pasting in my CSV file. I have set commit-interval as 1000 also but it took more than 20 minutes to get the records and process the records and pasting in my CSV file.
Please help me, friends. maybe I am doing something wrong.
Jobs-Context.xml
<step id="slave" xmlns="http://www.springframework.org/schema/batch">
<chunk reader="pagingItemReader" writer="flatFileItemWriter"
processor="itemProcessor" commit-interval="100" />
</tasklet>
</step>
itemreader bean
<bean id="pagingItemReader"
class="org.springframework.batch.item.database.JdbcPagingItemReader"
scope="step">
<property name="dataSource" ref="gemsDataSource" />
<property name="queryProvider">
<bean
class="org.springframework.batch.item.database.support.SqlPagingQueryProviderFactoryBean">
<property name="dataSource" ref="gemsDataSource" />
<property name="selectClause" value="SELECT * " />
<property name="fromClause" value="***QUERY****/>
<property name="whereClause" value="where rn between :fromId and :toId" />
<property name="sortKey" value="rn" />
</bean>
</property>
<!-- Inject via the ExecutionContext in rangePartitioner -->
<property name="parameterValues">
<map>
<entry key="fromId" value="#{stepExecutionContext[fromId]}" />
<entry key="toId" value="#{stepExecutionContext[toId]}" />
</map>
</property>
<property name="pageSize" value="100" />
<property name="rowMapper">
<bean class="com.hello.ItemRowMapper" />
</property>
</bean>
Please let me know if there is any issue with my code.
1
