Robot framework library for producing and consuming messages on Kafka 1.0+
APACHE-2.0 License
This is a robot framework wrapper around pykafka, the best python kafka library out there as of this writing, and the only one that supports kafka 1.0, 1.1 and 2.3.
This library provides some robotframework keywords that make working with Kafka topics easier. You can produce and consume from topics, and the framework will manage the producers and consumers for you, behind the scenes, based on the topics and consumer groups you specify. See below for more details.
Only python 3.X is supported.
The older version 0.10 supports python 2.7.
This following dependencies are downloaded when you install robotframework-pykafka:
Robotframework-jsonlibrary is not technically required for the library but the test examples use it, and it is recommended for JSON manipulation.
pip3 install [-U ] [--user] robotframework-pykafka
# For testing you may wish to install these too:
pip3 install -U --user --upgrade robotframework-jsonlibrary robotframework-httplibrary
To import this library into your robot test, use the following command:
Library robotframework_pykafka
1.0.X through 2.3.X.
Only string keys are supported. Currently JSON conversions (from string to dict) will have to be done by clients.
Only string or JSON strings are supported.
The following environment variables are assumed to have been set up in the session where this code is executed. Defaults will be provided if not specified.
Before reading from a Kafka topic, you have to initialize a consumer by telling it how to read from the topic, from the newest/latest message or from the oldest/earliest.
For both the "Kafka Set Consumer Offset" keywords, the consumerGroupName is optional; it defaults to be the same as the topicName. If specified, you will use the consumer that references this specific consumerGroupName, which may be important. For example, if you want to test two consumers of the same topic, set the 2nd consumer to a different consumerGroupName so the offsets are not shared.
Kafka Set Consumer Offset To Latest | topicName | [ consumerGroupName ]
Kafka Set Consumer Offset To Earliest | topicName | [ consumerGroupName ]
Kafka Consume String | topicName | [ consumerGroupName ]
Kafka Consume JSON | topicName | [ consumerGroupName ]
See the tests/ directory for some example tests of a kafka streams application.
They are meant to run against a simple kafka streaming app on github.
First get kafka and this app running: https://github.com/invadergir/kafka-streams-scala-template
cd tests
OUTPUTDIR=testresults
robot -d $OUTPUTDIR ./text-produce-consume.robot ./json-produce-consume.robot
This software is copyright 2018 Zebra Technologies, and is licensed under the Apache 2 software license.