Ekuiper

Overview

eKuiper 1.3.0 refactors the project to comply to LF edge standard including renaming the module name, project layout and adding copyright statements. We also refactore the build and CI scripts to migrate the docker images and rename the artifacts etc. We also provide a lot of new features for the SQL runtime and the EdgeX support.

Breaking Changes

- Project module, layout and package refactor. The plugins must be revised to import the new module of eKuiper and adapt to the package change.
- Refactor ui meta API

Features

- Refactor alias mechanism and support more usage scenarios
- Add window_start() and window_end() function
- Support message type property for edgeX source to allow connect to message bus directly
- Support more sink data template function
- Support shared instance of source
- Refactor SQL validation to refine aggregate related validation
- Add copyright statement
- Rename docker image

Fixes

- Fix influxdb plugin for multiple instances
- MQTT source: Fix resubscription failure when reconnecting
- Clean expired checkpoint memory and storage
- Clean checkpoint store when deleting rule
- Reduce the docker image size

Docs

- Update to rename kuiper to eKuiper
- Update EdgeX v2 related doc
- Add tutorial for eKuiper and OpenYurt integration

Thanks

- wb.lixinlin provides a fix for the mqtt source

Overview

eKuiper 1.2.1 supports EdgeX Ireland and adapts to EdgeX V2 API. This brings some [breaking changes](https://github.com/lf-edge/ekuiper/blob/1.2.1/docs/en_US/edgex/edgex_rule_engine_tutorial.md#migrate-to-edgex-v2). Especially, the commonly used meta `Device` has now renamed to `DeviceName`, rules that refer to the meta must be updated with the new name. Additionally, EdgeX binary readings are now supported to read into eKuiper stream. And Kuiper byte data can also be sinked to EdgeX binary reading.

Since the project has joined LF Edge and renamed to eKuiper, we also update the documentations to adopt the new name.

Features

- EdgeXSource and EdgeXSink support EdgeX V2 API
- EdgeXSource and EdgeXSink support EdgeX binary readings

Fixes

- Fix parse error for subtraction operator in some condition
- Allow short-circuit for logical operation
- Fix [the join calculation problem](https://github.com/lf-edge/ekuiper/issues/817) when joined streams have multiple tuples
- Fix sink meta API problem when a rule has multiple sinks
- Build
- Refactor plugins and tools to submodules
- Error Message Fixes
- Refine error messages for get rules API
- Do not print nil error when closing rule normally
- Documentation fixes
- Rename Kuiper to eKuiper
- Fix incorrect port in CLI/REST examples
- Fix incorrect translation in the Chinese version windows doc

Thanks

- ccb1900 provides a fix for the documentation

Overview

Kuiper 1.2.1 beta.1 added the support of binary reading and fix some problems of the support of Edgex v2 message. It is a pre-release for integration test.

Features & fixes

- Support binary reading of Edgex v2 message
- Update to the latest Edgex v2 modules

Overview

Kuiper 1.2.1 beta added the support of Edgex v2 message. It is a pre-release for integration test.

Features & fixes

- Support Edgex v2 message

Overview

Kuiper 1.2.0 added external function to map existing REST, gRPC or msgpackrpc services into user defined functions by configuration. It is an easy to use extension method to complement plugin. It is more recommened than plugin extension for all scenarios except performance critical scenario. External function can be defined in a batch by a json file and one or multiple schema files (only support protobuf format now). It is possible to hot reload by CLI or REST API. Addionally, http options are supported in the proto schema to better define the REST services.

A new type of source table was added in this release. It works as a snapshot at the current time point for the source. Users can use it as a lookup dictionary or referenced state by joining it with a stream.

We also added a new SQL expression: case expression to evaluates a list of conditions and returns one of multiple possible result expressions. Several enhanements were also added to the built-in functions. And the wildcard is now supported as the parameter of user defined functions. Finally, we fixed several issues for both the product and document.


Features & fixes

- Support external function to invoke exiting services in SQL by configuration only
- Implement [external function](https://github.com/emqx/kuiper/blob/master/docs/en_US/extension/external_func.md#external-function) configuration parsing and execution of REST services, gRPC services and msgpackrpc services
- Support protobuf as the schema type, support [google http api](https://github.com/emqx/kuiper/blob/master/docs/en_US/extension/external_func.md#http-options) options in the protobuf to better define the REST services
- Support table type to represent the current state of the stream
- Support [table](https://github.com/emqx/kuiper/blob/master/docs/en_US/sqls/tables.md) type, statements and management CLI/REST API
- Add [file source](https://github.com/emqx/kuiper/blob/master/docs/en_US/rules/sources/file.md) which is typically used as a static table
- Allow to use existing source type as a continuous table
- Support [case expression](https://github.com/emqx/kuiper/blob/master/docs/en_US/sqls/query_language_elements.md#case-expression)
- Support * as parameter of custom functions
- Support string parameter for [json functions](https://github.com/emqx/kuiper/blob/master/docs/en_US/sqls/built-in_functions.md#json-functions)
- Allow bigint, float and string as the parameter of format_time function
- Rename "reengine_field" to "kuiper_field" for non-specified fields
- Build
- Fix a permission problem when injecting configuration through mouted volumes by Docker
- Add check for markdown directory
- Add check for deb/rpm
- Update tdengine version
- Fixes
- Avoid printing password in logs
- Let meta fields ignore cases
- Fix problems when joining without stream name in the field ref
- Fix alias for same field names in multiple stream problem
- Fix access subfolder failure problem in absolute mode
- Fix stream describe problem for multi-level embedded structs
- Documentation fixes
- Fix some broken links
- Fix several typos
- Update json expression doc

Thanks

- jrtitus provides several fixes for the documentation
- tmactao provides a fix for the documentation

Overview

Kuiper 1.1.2 added a sample plugin *LabelImage* to demonstrate the integration with machine learning framework TensorFlow Lite. Additionally, function plugin was enhanced to support defining multiple functions in one plugin to organize functions in category and reduce the complexity to create functions. To build the extended plugins, the build tag `--trimpath` is now required in order to improve the compatibility. We also support more built in functions like geohash related functions and cardinality , support more expressions like negative array index and support more rule properties like sendError. Finally, we fix several issues for both the product and document.

Features & fixes

- Integration with TensorFlowLite
- Sample plugin *LabelImage* to leverage pre-trained TensorFlowLite model to label images
- Tutorial about [**Run TensorFlow Lite model with Kuiper function plugin**](https://github.com/emqx/kuiper/blob/master/docs/en_US/plugins/functions/tensorflow_lite_tutorial.md)
- Plugin enhancements
- Support multiple functions in one function plugin
- Support to get Kuiper root path in context `ctx.GetRootPath()`
- Refactor image functions to one plugin
- Add [GeoHash plugin](https://github.com/emqx/kuiper/blob/master/docs/en_US/plugins/functions/functions.md#geohash-plugin) to support several geo hash functions
- Build
- Add `--trimpath` build option for *kuiperd* build. Since this release, this property is required to [build extended plugins](https://github.com/emqx/kuiper/blob/master/docs/en_US/extension/overview.md#setup-the-plugin-developing-environment)
- Run process/container as *kuiper* user to avoid security risk
- Support negative array index such as `array[:-1]`
- Add built in function `cardinality(array)` to get the length of array
- Add rule property *sendError* to specify if runtime errors will send to the sink
- Fixes
- Fix stop rule memory clean up problem
- Restore httpPull dataSource support
- Fixed describe stream issue for bytea type
- Fixed arbitrary session window error at the first input
- Remove unused empty folders inside data
- Documentation fixes
- Fix several links
- Fix an index error for built-in functions doc

Thanks

- beaufrusetta provides a fix to run process/container as *kuiper* user
- noahlaux provides a fix for the documentation