Hi Zeebe Folks
Zeebe software is great and provides us with various desired functionalities. Great work guys !!! I have few comments to make which I saw and due to which I believe some code requires fix. Please feel free to let me know if you think otherwise.
We use Zeebe 0.24.1 and Kafka Exporter 2.0.0 version and we are performing some performing tests on it. Since we are going to host Zeebe in our workplace we won’t have control of what clients provide data to us that are exposed as services. In the past we have come up with our own Json serialization when using Kafka Exporter 1.0.0 due to issues in serializing and deserializing data. Recently we got the latest 2.0.0 Kafka exporter as the new implementation is based on JSON and thought of not using our own Json serialization/deserialization logic. The new exporter works just fine provided we give the data correctly. If it is having a bad entry then whole serialization/deserialization gets messed up and as a result Kafka exporter might exit or continuously spans error log messages. For instance consider the below code where a worker is sending a publish event
final ZeebeFuture<PublishMessageResponse> send = client.newPublishMessageCommand()
.messageName(messageName)
.correlationKey(correlationKey)
.variables(**eventVariables**)
.send();
Here the eventVariables is a map and this might be coming from the client. We won’t be having any control to it. The map itself can be entry and the entry inside might have a proper key but value might be null. When we try to use Kaka exporter we get serialization and deserialization issues as a result this bad record creates exporter problems and I believe it works fine only if the entries are not null.
For another issue which I noticed while I was walking through code for serialization and deserialization I figured out that it uses zeebe immutables and looking at the code it has setter method with some flags. For instance consider the below immutable code in “ImmutableJobRecordValue” class that we have in zeebe-immutables
@JsonProperty("variables")
public void setVariables(Map<String, Object> variables) {
this.variables = variables;
this.variablesIsSet = true;
}
Please note that similar logic exists in multiple places.
In the above we are not checking whether the collection is empty or null, we are just simply setting the variables and turning on the flag to be true. As a result it might cause serialization/deserialization issues where ever the logic is relying on this flag.
For our use case the issue was in the entry of variables map where the key exists and value was null. I apologize if I have missed adding any information. Please feel free to ask follow up questions and I will get back to you. We think this requires fix so that we don’t have to come up with our custom wrapper to perform all these validations before passing it onto Zeebe/Exporter. Please let me know if the above makes sense and whether it requires new changes from your end so that we can won’t be performing any modifications from our end.
Thanks
Vijay
