Cameron Braid: Since zeebe v1 .x is going to break backwards compatibility, would you consider changing the key generator so that new zeebe installs will generate new unique keys, rather than generating ones that collide with previous installs ?
Cameron Braid: This will greatly ease the migration from pre v1 to v1. For example when using an exporter with either zeebe simple monitor or zeeqs. If such a key change was implemented, then the new zeebe v1 can be installed in parallel with the old version importing the exported records into the same simple monitor/zeeqs instance. Then when workflow instances on the old zeebe install are complete it can be decomissioned
Cameron Braid: I would also imagine this might be a suitable pattern for scaling zeebe with no downtime. For example start off with a single non replicated cluster. When load demands spin up a second zeebe higher capacity cluster in parralel.
Cameron Braid: I realise that this can be achieved a different way - say by introducing a clusterId, Either way, it really needs to be incorporated into zeebe core. SO that a) all exporters can support this value. and b) all tools that import exported data become aware of this new field.
Josh Wulf: The exported data has a different schema. This is part of the breaking change. The engine internals are being changed to an event-sourcing model.
So you won’t be able to use the same ES schema/index with it - and if Simple Monitor or Zeeqs are coupled to parts of the schema that are changing, they will also have a breaking change.
Cameron Braid: ok, fair enough, that prevents the prev1 to v1 use case… but what about the second use case ?
Josh Wulf: A blue/green architecture approach is a good idea. You would probably want to stand up a new version and put 10% of your production traffic on it to test it before going all in.
A clusterId as part of unique key would be needed to use the same data stores. I think it’s worth opening a feature request, if one is not open already.
Note: This post was generated by Slack Archivist from a conversation in the Zeebe Slack, a source of valuable discussions on Zeebe (get an invite). Someone in the Slack thought this was worth sharing!
If this post answered a question for you, hit the Like button - we use that to assess which posts to put into docs.