Error while trying to deploy a workflow after upgrading to latest docker-compose

Hi Guys,

I just updated my repo from GitHub - camunda-community-hub/zeebe-docker-compose: Zeebe with Operate Docker Compose configuration to start playing with the latest version of zeebe β†’ 0.23.1

I start the container instances with

docker-compose up

But when trying to deploy a workflow using the following command

zbctl deploy test.bpmn --insecure --address localhost:26500

I get the following error

Error: rpc error: code = NotFound desc = Expected to execute command, but this command refers to an element that doesn’t exist.

This is the log I got about that error

2020-05-14 17:45:37.483 [io.zeebe.gateway.impl.broker.BrokerRequestManager] [Broker-0-zb-actors-0] ERROR io.zeebe.gateway - Error handling gRPC request

io.grpc.StatusRuntimeException: NOT_FOUND: Expected to execute command, but this command refers to an element that doesn't exist.

at io.grpc.Status.asRuntimeException(Status.java:524) ~[grpc-api-1.28.1.jar:1.28.1]

at io.zeebe.gateway.EndpointManager.convertThrowable(EndpointManager.java:397) ~[zeebe-gateway-0.23.1.jar:0.23.1]

at io.zeebe.gateway.EndpointManager.lambda$sendRequest$3(EndpointManager.java:311) ~[zeebe-gateway-0.23.1.jar:0.23.1]

at io.zeebe.gateway.impl.broker.BrokerRequestManager.lambda$sendRequest$3(BrokerRequestManager.java:148) ~[zeebe-gateway-0.23.1.jar:0.23.1]

at io.zeebe.gateway.impl.broker.BrokerRequestManager.sendRequestInternal(BrokerRequestManager.java:174) ~[zeebe-gateway-0.23.1.jar:0.23.1]

at io.zeebe.gateway.impl.broker.BrokerRequestManager.lambda$sendRequest$4(BrokerRequestManager.java:162) ~[zeebe-gateway-0.23.1.jar:0.23.1]

Caused by: io.zeebe.gateway.cmd.PartitionNotFoundException: Expected to execute command, but this command refers to an element that doesn't exist.

at io.zeebe.gateway.impl.broker.BrokerRequestManager.determineBrokerNodeIdProvider(BrokerRequestManager.java:206) ~[zeebe-gateway-0.23.1.jar:0.23.1]

at io.zeebe.gateway.impl.broker.BrokerRequestManager.sendRequestInternal(BrokerRequestManager.java:172) ~[zeebe-gateway-0.23.1.jar:0.23.1]

I also see an error when the broker is starting

2020-05-14 17:45:01.495 [Broker-0-ZeebePartition-1] [Broker-0-zb-actors-0] ERROR io.zeebe.broker.system - Unexpected error on stream processor installation!

java.lang.RuntimeException: Could not deserialize object. Deserialization stuck at offset 1 of length 8

at io.zeebe.msgpack.UnpackedObject.wrap(UnpackedObject.java:35) ~[zeebe-msgpack-value-0.23.1.jar:0.23.1]

at io.zeebe.db.impl.rocksdb.transaction.TransactionalColumnFamily.get(TransactionalColumnFamily.java:66) ~[zeebe-db-0.23.1.jar:0.23.1]

at io.zeebe.db.impl.rocksdb.transaction.TransactionalColumnFamily.get(TransactionalColumnFamily.java:134) ~[zeebe-db-0.23.1.jar:0.23.1]

at io.zeebe.db.impl.rocksdb.transaction.TransactionalColumnFamily.get(TransactionalColumnFamily.java:58) ~[zeebe-db-0.23.1.jar:0.23.1]

at io.zeebe.engine.state.LastProcessedPositionState.getPosition(LastProcessedPositionState.java:33) ~[zeebe-workflow-engine-0.23.1.jar:0.23.1]

at io.zeebe.engine.state.ZeebeState.getLastSuccessfulProcessedRecordPosition(ZeebeState.java:157) ~[zeebe-workflow-engine-0.23.1.jar:0.23.1]

at io.zeebe.engine.processor.StreamProcessor.recoverFromSnapshot(StreamProcessor.java:241) ~[zeebe-workflow-engine-0.23.1.jar:0.23.1]

at io.zeebe.engine.processor.StreamProcessor.onActorStarted(StreamProcessor.java:99) ~[zeebe-workflow-engine-0.23.1.jar:0.23.1]

at io.zeebe.util.sched.ActorJob.invoke(ActorJob.java:73) [zeebe-util-0.23.1.jar:0.23.1]

at io.zeebe.util.sched.ActorJob.execute(ActorJob.java:39) [zeebe-util-0.23.1.jar:0.23.1]

at io.zeebe.util.sched.ActorTask.execute(ActorTask.java:115) [zeebe-util-0.23.1.jar:0.23.1]

at io.zeebe.util.sched.ActorThread.executeCurrentTask(ActorThread.java:107) [zeebe-util-0.23.1.jar:0.23.1]

at io.zeebe.util.sched.ActorThread.doWork(ActorThread.java:91) [zeebe-util-0.23.1.jar:0.23.1]

at io.zeebe.util.sched.ActorThread.run(ActorThread.java:195) [zeebe-util-0.23.1.jar:0.23.1]

Caused by: io.zeebe.msgpack.spec.MsgpackReaderException: Unable to determine map type, found unknown header byte 0x00 at reader offset 0

at io.zeebe.msgpack.spec.MsgPackReader.exceptionOnUnknownHeader(MsgPackReader.java:474) ~[zeebe-msgpack-core-0.23.1.jar:0.23.1]

at io.zeebe.msgpack.spec.MsgPackReader.readMapHeader(MsgPackReader.java:79) ~[zeebe-msgpack-core-0.23.1.jar:0.23.1]

at io.zeebe.msgpack.value.ObjectValue.read(ObjectValue.java:88) ~[zeebe-msgpack-value-0.23.1.jar:0.23.1]

at io.zeebe.msgpack.UnpackedObject.wrap(UnpackedObject.java:31) ~[zeebe-msgpack-value-0.23.1.jar:0.23.1]

... 13 more

2020-05-14 17:45:01.503 [Broker-0-ZeebePartition-1] [Broker-0-zb-actors-0] ERROR io.zeebe.broker.system - Failed to install leader partition 1

2020-05-14 17:45:01.510 [Broker-0-ZeebePartition-1] [Broker-0-zb-actors-0] INFO io.zeebe.broker.system - Unexpected failures occurred when installing leader services, stepping down

2020-05-14 17:45:01.512 [Broker-0-StreamProcessor-1] [Broker-0-zb-actors-1] ERROR io.zeebe.logstreams - Actor Broker-0-StreamProcessor-1 failed in phase FAILED.

Any ideas?

Thanks!

1 Like

I was able to fix this. Apparently, old data was in docker volumes was not compatible with the new version of zeebe.

So I had to remove the docker volumes to be able to successfully migrate to the new version

docker volume rm operate_zeebe_data operate_zeebe_elasticsearch_data

2 Likes

Also, you need to modify or migrate your model: https://zeebe.io/blog/2020/04/zeebe-operate-023-release/

2 Likes