I have a 3 broker cluster setup. When I tried to increase the partitions from 3 to 25, I also had to increase the memory resources too to startup itself. From the discussions here, I understand that the disk requirements will be considerably affected by number of partitions. But did not quite understand how the runtime memory is proportional to number of partitions for startup?
These are the changes I had to do:
Broker version: 0.25.0
Workers: 1 http worker
Exporters : Kafka and Hazelcast
memory: 3Gi -> increased from 1Gi
memory: 6Gi -> increased from 2Gi
Changed default heap size to:
I am running on a Linux machine with 8 cores.
Could you please help me understand this?