Hello everyone!!
I am implementing a machine learning application that also visualizes the live input data into graphs. However i have not yet comprehended the use of Redis in channels. More specifically, if i were to deploy this app on a java framework i would have used Apache Kafka for the live streaming of data, and i will try to implement it in my app using pykafka or something similar. However this is where redis comes because in every tutorial i watch they use redis but i cant understand the difference between Kafka even though i find that kafka is significantly faster. Can someone explain to me a it or point me to correct info.
Thanx a lot in advance
What is it you’re actually trying to ask here?
Redis is a data store. Kafka is a full message broker service, more similar to ActiveMQ, RabbitMQ, ZeroMQ, etc. They’re really two different things.
1 Like
I’d say the main difference (if you only look at the Redis pub/sub mode, which is what one might compare to Kafka) is how they store messages.
- Kafka is at-least-once: It stores messages and replays them if it thinks the other end didn’t get them, so you get message one or more times (and have to handle duplicate messages as your failure case)
- Redis Pub/Sub is at-most-once: It pushes messages to a client if one is around, but if not, or there’s an error, the message will vanish. You have missing messages as your failure case.
Any queue is going to be one of these two patterns - it just depends how you want it to fail. Generally, Kafka is a much better queue, as Ken says - however, Kafka takes quite a bit of work to set up well, so tutorials are probably using Redis as it’s very easy to get going.
(one note that is Django Channels does use Redis as well, but when it does it uses Redis lists to achieve an at-least-once type queue)
1 Like
Thank you very much you were really helpfull.