Cmem-plugin-kafka

Added

- support for sending tombstone message from kafka producer plugin
- https://medium.com/damienthomlutz/deleting-records-in-kafka-aka-tombstones-651114655a16

Changed

- Upgrade template to V6.0.1
- upgrades to use cmem-plugin-base 4.3.0 (for CMEM 23.3)
- SASL Password parameter type from StringParameterType to PasswordParameterType

Fixes

- upload of file resources was broken for 23.2

Changed

- upgrades to use cmem-plugin-base 4.1.0 (for CMEM 23.2, python 3.11)

Added

- `Client_Id` parameter has been added to the advanced section of Kafka Consumer.
- When the `Client_Id` field is empty, the plugin defaults to `DNS:TASK ID`.
- `Local Consumer Queue Size` parameter to limit local consumer queue size
- Consume input entities in KafkaProducerPlugin to generate messages
- Generate output entities in KafkaConsumerPlugin from messages
- Consume JSON dataset in KafkaProducerPlugin to generate messages
- Generate JSON dataset in KafkaConsumerPlugin from messages
- `Compression Type` parameter has added to support standard compression
- `message limit` parameter has been added to the advanced section of kafka consumer
- `disable commit` parameter has been added to the advanced section of kafka consumer

Changed

- `Messages Dataset` Parameter moved from the advanced section to Main section (will change again in future)
- Replace `none` with `error` in Auto Offset Reset parameter choice list
- moved kakfa producer/consumer configuration check from __init__ to validate() and calling explictly from execute()
- resolved endless loop caused by improper kafka server configuration
- generate entities with "message as plain value"

Added

- Kafka Consumer (Receive Messages) Plugin

Fixed

- Documentation and help texts

Changed

- upgrade to confluent-kafka 1.9.2

Fixed

- use requests response stream for xml parsing (avoid out of memory)