What's the use of 'maxMessageSize' config? Can I increase gRPC maximum size?

hi:

I am trying to send one large message(about 50mb) using java client to a broker with gateway.
I have change 'maxMessageSize` to 80MB in the config file, below is the stack trace of the error.

Jul 10, 2020 2:57:58 PM io.grpc.netty.NettyServerStream$TransportState deframeFailed
WARNING: Exception processing message
io.grpc.StatusRuntimeException: RESOURCE_EXHAUSTED: gRPC message exceeds maximum size 4194304: 79571781
at io.grpc.Status.asRuntimeException(Status.java:524)
at io.grpc.internal.MessageDeframer.processHeader(MessageDeframer.java:387)
at io.grpc.internal.MessageDeframer.deliver(MessageDeframer.java:267)
at io.grpc.internal.MessageDeframer.request(MessageDeframer.java:161)
at io.grpc.internal.AbstractStream$TransportState.requestMessagesFromDeframer(AbstractStream.java:205)
at io.grpc.netty.NettyServerStream$Sink$1.run(NettyServerStream.java:112)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:834)

client
          .newPublishMessageCommand()
          .messageName("large-message")
          .correlationKey("test-key")
          .messageId("test-id")
          .variables(largeJsonString)
          .send()
          .join();

Any idea? Thank you

Hi @ltxlouis

please don’t use Zeebe as storage system. It is not intended and not recommended to handle such big messages. We already have found issues with the current defaults, see for example this https://github.com/zeebe-io/zeebe/issues/4249

I would suggest that you store the data somewhere else, in a system which is intended to to handel such big data like mongo db or elastic etc. You can then use a key or id to reference the data and use zeebe only for orchestrating your services.

If you still want it, then yes you need to change the maxMessageSize.

Greets
Chris

2 Likes

Thanks for your help! I will try mongodb.