Redis + Node.js + Socket.IO – Event-driven, subscription-based broadcasting

Recently, I have been working on building a server for broadcasting messages over socket connections. The basic design I had in mind was something like this-

1. The broadcast channels would basically be Redis Pub/Sub channels, with input coming from outside of the server(either a script or another server).
2. Each socket client would define the channels it wanted to listen to, via subscribe and unsubscribe requests to the server.
3. I did not want to instantiate a new Redis client per socket connection, but rather one Redis client per channel. This way, it could be shared(for listening) among all the socket connections subscribing to it.

Given below is the code for this server. The dependencies you will have to satisfy for running it are-
1. Redis
2. Node.js
3. The following Node.js modules- socket.io, redis.
If you don’t, some Googling will get it done soon enough 😛


//Port config
const PORT = 3000;

//Requires and main server objects
var redis = require('redis');
var socketio = require('socket.io');
var app = require('http').createServer().listen(PORT);
var io = socketio.listen(app);

//This object will contain all the channels being listened to.
var global_channels = {};

//Server Logic goes here
io.on('connection', function(socketconnection){

//All the channels this connection subscribes to
socketconnection.connected_channels = {}

//Subscribe request from client
socketconnection.on('subscribe', function(channel_name){
//Set up Redis Channel
if (global_channels.hasOwnProperty(channel_name)){
//If channel is already present, make this socket connection one of its listeners
global_channels[channel_name].listeners[socketconnection.id] = socketconnection;
}
else{
//Else, initialize new Redis Client as a channel and make it subscribe to channel_name
global_channels[channel_name] = redis.createClient();
global_channels[channel_name].subscribe(channel_name);
global_channels[channel_name].listeners = {};
//Add this connection to the listeners
global_channels[channel_name].listeners[socketconnection.id] = socketconnection;
//Tell this new Redis client to send published messages to all of its listeners
global_channels[channel_name].on('message', function(channel, message){
Object.keys(global_channels[channel_name].listeners).forEach(function(key){
global_channels[channel_name].listeners[key].send(message);
});
});
}

socketconnection.connected_channels[channel_name] = global_channels[channel_name];

});

//Unsubscribe request from client
socketconnection.on('unsubscribe', function(channel_name){
if (socketconnection.connected_channels.hasOwnProperty(channel_name)){
//If this connection is indeed subscribing to channel_name
//Delete this connection from the Redis Channel's listeners
delete global_channels[channel_name].listeners[socketconnection.id];
//Delete channel from this connection's connected_channels
delete socketconnection.connected_channels[channel_name];
}
});

//Disconnect request from client
socketconnection.on('disconnect', function(){
//Remove this connection from listeners' lists of all channels it subscribes to
Object.keys(socketconnection.connected_channels).forEach(function(channel_name){
delete global_channels[channel_name].listeners[socketconnection.id];
});
});

});

Lines 20-43 define the server’s behavior when a new request is made by a connected socket, to subscribe to a certain broadcast channel. A broadcast channel is nothing but a Redis client that subscribes to a given pubsub channel, with the added property of a collection of listeners. Each of these Redis clients is configured to send every message it receives on the pubsub channel, to all of its listeners. The global object global_channels maintains a mapping of channel_name to Redis client instance, so that it can be shared among listeners. Each socket client also maintains its own collection of channels it subscribes to.

On unsubscribing (lines 45-54), the socket client is removed from the Redis client’s listeners, and the Redis client is removed from the socket client’s connected channels. On disconnecting, the socket is removed from the listeners of all Redis clients it subscribes to.

Now, heres a small Node.js client that will connect to the above server, subscribe to somechannel, and unsubscribe as soon as the first message is received. You will need the socket.io-client Node module for this.


var io = require('socket.io-client');
var serverUrl = 'http://localhost:3000';
var conn = io.connect(serverUrl);

conn.emit('subscribe', 'somechannel');
conn.on('message', function (message){
console.log(message);
conn.emit('unsubscribe', 'somechannel');
});

The server code in this post is pretty basic, and its only job is to showcase the functionality I intended to show. For production/serious applications, there are quite a few changes you would want to make, such as-

1. Token/user-password based authentication. This could also ensure that every client can subscribe/unsubscribe only to a given/alloted set of channels.

2. Other security measures, such as disconnecting a client thats not subscribed to any channel for a given timeout.

3. Deleting channels with no listeners.

This is my first time writing proper Javascript/Node.js code, so do let me know if I have made some mistake or there are any optimizations possible to the code above 🙂 .

EDIT 1:

You could also go with one-Redis-client-per-sever (instead of channel), it would just mean storing the relevant mappings and implementing the routing logic in the ‘message’ event for the Redis client. Would provide considerable gains in case of a high number of channels.

Advertisements

5 thoughts on “Redis + Node.js + Socket.IO – Event-driven, subscription-based broadcasting

  1. Excellent description over Redis pub/sub and redis connection with socket.io. I have one confusion regarding using redis that if we can do use socket.io framework for publishing and replying to messages, then why did we need redis here exactly ?

  2. Very helpful and concise! How do you connect to the external process, here?

    global_channels[channel_name] = redis.createClient();

    I’m assuming this is likely to be another socket.io server? Simply provide a URL and port, and then listen for events?

    Thanks again!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s