How can we perform the batch jobs in camunda …as i have multiple records which are waiting for due date say 10k+ records …i want them to process in a batch of say 500 / 15min of time span , how can we do it in camunda and java application.
please note …we cannot use the dashboard batch option to run it because we have to perform at certain level in middle of the flow that is at due dates (as the number of records are waiting at due date are huge for that particular date). how can we do it is there any option we can use here with respect to camunda or database job table or java here please help
Thanks for reply @Niall , we cant use it in Time- > Date section for due date right we might need to create the service task/user task before the timer section right to use the multi instance either parallel or sequential ?
we have more than 15k + records at table called "Camunda.act_ru_job "which are waiting for due to to process them…for same day due date there will be 10k+ records in one go. so we wanted to avoid this records to process them in one go as it causing heap memory n GC overhead.
so we wanted to split it across the different configurable batch size and configurable time like within first 15m we can process about 500 records n remaining from resp next 15 -15 minutes time frame with 500 records each.
so to fetch the records do we use the GET /process-instance this api with process definition key as request param so will i get the all the records of 15k ?
n then i can process them either parallel or sequential i am not sure here what’s best in these two for my use case?
records are lying inside the database we need to retrieve them n process them in batch wise to avoid memory issues
Can you let me know how many instances of your camunda 7 engine you are running in the cluster - and also let me know what the process engine settings are?
And the engine settings? Have you tried to tweak them at all to improve performance. This page might be able to help you understand what you could do to improve the performance.