From v5.0 version there is option to provide custom serializer.
Custom serializer instance must meet a few requirements:
Have a encode (data) method, that returns encoded data
Have a decode (data) method, that returns decoded data
Have a protocol string property, that contains a protocol name. This name is concatenated with "wamp.2." string and is then passed as websocket subprotocol http header.
Have a isBinary boolean property, that indicates, is this a binary protocol or not.
Take a look at JsonSerializer.js or MsgpackSerializer.js as examples.