I see this periodically in the broker log for Zeebe 0.23.1 running on a local machine with one broker:
2020-05-18 11:21:14.809 [Broker-0-SnapshotDirector-1] [Broker-0-zb-fs-workers-1] INFO io.zeebe.logstreams.snapshot - Finished taking snapshot, need to wait until last written event position 30064885224 is committed, current commit position is 30064885224. After that snapshot can be marked as valid.
2020-05-18 11:21:14.809 [Broker-0-SnapshotDirector-1] [Broker-0-zb-fs-workers-1] INFO io.zeebe.logstreams.snapshot - Current commit position 30064885224 is greater then 30064885224, snapshot is valid.
2020-05-18 11:36:14.676 [Broker-0-SnapshotDirector-1] [Broker-0-zb-fs-workers-0] WARN io.zeebe.logstreams.snapshot - Failed to obtain a pending snapshot directory for position 30064888960
2020-05-18 11:51:14.729 [Broker-0-SnapshotDirector-1] [Broker-0-zb-fs-workers-0] WARN io.zeebe.logstreams.snapshot - Failed to obtain a pending snapshot directory for position 30064888960
2020-05-18 12:06:14.755 [Broker-0-SnapshotDirector-1] [Broker-0-zb-fs-workers-0] WARN io.zeebe.logstreams.snapshot - Failed to obtain a pending snapshot directory for position 30064888960
2020-05-18 12:21:14.802 [Broker-0-SnapshotDirector-1] [Broker-0-zb-fs-workers-0] WARN io.zeebe.logstreams.snapshot - Failed to obtain a pending snapshot directory for position 30064888960
2020-05-18 12:36:14.822 [Broker-0-SnapshotDirector-1] [Broker-0-zb-fs-workers-1] WARN io.zeebe.logstreams.snapshot - Failed to obtain a pending snapshot directory for position 30064888960
2020-05-18 12:51:14.831 [Broker-0-SnapshotDirector-1] [Broker-0-zb-fs-workers-0] WARN io.zeebe.logstreams.snapshot - Failed to obtain a pending snapshot directory for position 30064888960
2020-05-18 13:06:14.842 [Broker-0-SnapshotDirector-1] [Broker-0-zb-fs-workers-1] WARN io.zeebe.logstreams.snapshot - Failed to obtain a pending snapshot directory for position 30064888960
2020-05-18 13:21:14.922 [Broker-0-SnapshotDirector-1] [Broker-0-zb-fs-workers-1] WARN io.zeebe.logstreams.snapshot - Failed to obtain a pending snapshot directory for position 30064888960
2020-05-18 13:36:14.980 [Broker-0-SnapshotDirector-1] [Broker-0-zb-fs-workers-0] WARN io.zeebe.logstreams.snapshot - Failed to obtain a pending snapshot directory for position 30064888960
2020-05-18 13:51:14.983 [Broker-0-SnapshotDirector-1] [Broker-0-zb-fs-workers-0] WARN io.zeebe.logstreams.snapshot - Failed to obtain a pending snapshot directory for position 30064888960
2020-05-18 14:06:14.995 [Broker-0-SnapshotDirector-1] [Broker-0-zb-fs-workers-0] WARN io.zeebe.logstreams.snapshot - Failed to obtain a pending snapshot directory for position 30064888960
2020-05-18 14:21:15.447 [Broker-0-SnapshotDirector-1] [Broker-0-zb-fs-workers-0] WARN io.zeebe.logstreams.snapshot - Failed to obtain a pending snapshot directory for position 30064888960
2020-05-18 14:36:15.586 [Broker-0-SnapshotDirector-1] [Broker-0-zb-fs-workers-0] WARN io.zeebe.logstreams.snapshot - Failed to obtain a pending snapshot directory for position 30064888960
No, plenty of disk space (> 1 TB left). I download the latest release from Github and run the executable on a local machine. The only change to the yaml config is enabling Elasticsearch exporter. The error sometimes comes back, but restarting broker usually gets rid of it. Sometimes it goes away by itself, followed by:
2020-05-18 23:46:00.725 [Broker-0-SnapshotDirector-1] [Broker-0-zb-fs-workers-1] INFO io.zeebe.logstreams.snapshot - Current commit position 34359811960 is greater then 34359811960, snapshot is valid.
2020-05-20 11:19:59.223 [Broker-0-SnapshotDirector-1] [Broker-0-zb-fs-workers-1] WARN io.zeebe.logstreams.snapshot - Failed to obtain a pending snapshot directory for position 94494478368
2020-05-20 11:34:59.224 [Broker-0-SnapshotDirector-1] [Broker-0-zb-fs-workers-1] WARN io.zeebe.logstreams.snapshot - Failed to obtain a pending snapshot directory for position 94494774432
2020-05-20 11:49:59.242 [Broker-0-SnapshotDirector-1] [Broker-0-zb-fs-workers-0] WARN io.zeebe.logstreams.snapshot - Failed to obtain a pending snapshot directory for position 94495073336
2020-05-20 12:04:59.253 [Broker-0-SnapshotDirector-1] [Broker-0-zb-fs-workers-1] WARN io.zeebe.logstreams.snapshot - Failed to obtain a pending snapshot directory for position 94495369720
2020-05-20 12:19:59.267 [Broker-0-SnapshotDirector-1] [Broker-0-zb-fs-workers-1] WARN io.zeebe.logstreams.snapshot - Failed to obtain a pending snapshot directory for position 94495667488
2020-05-20 12:34:59.282 [Broker-0-SnapshotDirector-1] [Broker-0-zb-fs-workers-1] WARN io.zeebe.logstreams.snapshot - Failed to obtain a pending snapshot directory for position 94495998208
2020-05-20 12:49:59.316 [Broker-0-SnapshotDirector-1] [Broker-0-zb-fs-workers-1] WARN io.zeebe.logstreams.snapshot - Failed to obtain a pending snapshot directory for position 94496329704
2020-05-20 13:04:59.348 [Broker-0-SnapshotDirector-1] [Broker-0-zb-fs-workers-1] WARN io.zeebe.logstreams.snapshot - Failed to obtain a pending snapshot directory for position 94496647088
2020-05-20 13:19:59.367 [Broker-0-SnapshotDirector-1] [Broker-0-zb-fs-workers-1] WARN io.zeebe.logstreams.snapshot - Failed to obtain a pending snapshot directory for position 94496944296
2020-05-20 13:34:59.373 [Broker-0-SnapshotDirector-1] [Broker-0-zb-fs-workers-1] WARN io.zeebe.logstreams.snapshot - Failed to obtain a pending snapshot directory for position 94497251888
2020-05-20 13:49:59.376 [Broker-0-SnapshotDirector-1] [Broker-0-zb-fs-workers-1] WARN io.zeebe.logstreams.snapshot - Failed to obtain a pending snapshot directory for position 94497583000
2020-05-20 14:04:59.405 [Broker-0-SnapshotDirector-1] [Broker-0-zb-fs-workers-1] WARN io.zeebe.logstreams.snapshot - Failed to obtain a pending snapshot directory for position 98784509744
2020-05-20 14:19:59.428 [Broker-0-SnapshotDirector-1] [Broker-0-zb-fs-workers-1] WARN io.zeebe.logstreams.snapshot - Failed to obtain a pending snapshot directory for position 98784808368
2020-05-20 14:34:59.451 [Broker-0-SnapshotDirector-1] [Broker-0-zb-fs-workers-1] WARN io.zeebe.logstreams.snapshot - Failed to obtain a pending snapshot directory for position 98785106032
2020-05-20 14:49:59.479 [Broker-0-SnapshotDirector-1] [Broker-0-zb-fs-workers-1] WARN io.zeebe.logstreams.snapshot - Failed to obtain a pending snapshot directory for position 98785404504
2020-05-20 15:04:59.493 [Broker-0-SnapshotDirector-1] [Broker-0-zb-fs-workers-1] WARN io.zeebe.logstreams.snapshot - Failed to obtain a pending snapshot directory for position 98785712152
2020-05-20 15:19:59.505 [Broker-0-SnapshotDirector-1] [Broker-0-zb-fs-workers-1] WARN io.zeebe.logstreams.snapshot - Failed to obtain a pending snapshot directory for position 98786010752
2020-05-20 15:34:59.532 [Broker-0-SnapshotDirector-1] [Broker-0-zb-fs-workers-1] WARN io.zeebe.logstreams.snapshot - Failed to obtain a pending snapshot directory for position 98786308512
2020-05-20 15:49:59.544 [Broker-0-SnapshotDirector-1] [Broker-0-zb-fs-workers-1] WARN io.zeebe.logstreams.snapshot - Failed to obtain a pending snapshot directory for position 98786602840
2020-05-20 16:04:59.548 [Broker-0-SnapshotDirector-1] [Broker-0-zb-fs-workers-0] WARN io.zeebe.logstreams.snapshot - Failed to obtain a pending snapshot directory for position 98786896008
2020-05-20 16:19:59.566 [Broker-0-SnapshotDirector-1] [Broker-0-zb-fs-workers-1] WARN io.zeebe.logstreams.snapshot - Failed to obtain a pending snapshot directory for position 98787191968
2020-05-20 16:34:59.578 [Broker-0-SnapshotDirector-1] [Broker-0-zb-fs-workers-1] WARN io.zeebe.logstreams.snapshot - Failed to obtain a pending snapshot directory for position 98787489680
2020-05-20 16:49:59.602 [Broker-0-SnapshotDirector-1] [Broker-0-zb-fs-workers-1] WARN io.zeebe.logstreams.snapshot - Failed to obtain a pending snapshot directory for position 98787785328
context:
zeebe 0.23.1
es Exporter
deployed in docker swarm
Quick followup - Can this issue be in any way related to failure to show new workflows deployed, or running instances, for example, in the community monitor?
Eetay