Number Variables: input != output (depending on size)

When I start a workflow with some variable, where one variable “foo” happens to be of type long, then this “foo” changes to an integer when getting it inside a worker.

In my case the long is still within the integer-range, and since Zeebe is storing vars as JSON I assume the type-information is lost, and hence the client converts the value to a integer because it fits. But when “foo” is actually big enough for an long, I get a long back.

// those variables are added to the workflow during initialization
Map<String, Object> vars = new HashMap<>(3);
vars.put("X", Long.MAX_VALUE);
vars.put("Y", 1000);

// Worker
@Override
public void handle(JobClient client, ActivatedJob job) {
    long x = (long)job.getVariablesAsMap().get("X"); // works because X is big enough, but breaks when X changes to a int
    long y = (long)job.getVariablesAsMap().get("Y") // error because of int
    long y2 = (long)(int)job.getVariablesAsMap().get("Y"); // works as well but breaks the same

    client.newCompleteCommand(job.getKey());
}

This makes it merely impossible to have a type-safe variable-access. Am I missing something?

This issue ocured with the Java-Client, but I expect it to be the same with Go. Only JS is probably not affected.

Hi @lostiniceland,

thank you for raising this up! I see your point.

As you mentioned, the variables are stored as JSON - so the type information is lost. One option would be to deserialize numbers in the (Java) client always as long (or double in case of a floating-point number).

Would this work for you?
Do you have another idea?

Best regards,
Philipp

Hi Philipp,

yes, that would work. Should I open a Github-Issue?
For the Go-Client you probably need something similar ( uint8 , uint16 , uint32 , uint64 , int8 , int16 , int32 and int64)

Out of curiosity (unrelated to this issue): why store in JSON when you already communicate via gRPC? Couldn’t you just store the Protobuf binaries or is JSON easier for compatibility reasons?

BTW: you could also use a POJO to deserialize the variables: https://docs.zeebe.io/java-client-examples/data-pojo.html. This might be also a good way to ensure type-safety.

What do you think?

Regarding:

It’s because we started with something different from gRPC. However, let’s discuss if it would be possible to use Protobuf (which gRPC is based on) instead of JSON (which is encoded as MessagePack internally).

As far as I know, you can’t serialize a generic map/document (e.g. variables) in Protobuf. Protobuf requires a schema. In order to process the variables in the broker (e.g. evaluating an expression), you need to walk through the variables/document. But without knowing the schema, I think it is not possible.

What do you think?
Do you have an idea?

Maybe I can test this later.

EDIT: I can confirm when using POJOs the deserialization works as expected. Due to Bean-Convention the client converts to the correct type.

With regards to Protobuf and generic maps I guess you are right, but I am no expert on Protobuf (just used it a couple of times).

1 Like