Redis : Usage Patterns

Redis Databases

  1. Use different Redis databases for different kinds of data. In Redis, databases are identified by an integer index, not by a database name. By default, a client is connected to database 0. With the SELECT command you can switch to a different database:
    • redis> select 3
      OK
  1. Each Redis database has its own keyspace. By using different databases for your ‘staging’ and ‘production’ data, for example, you don’t have to worry about key clashes between the two

 

References:

  1. http://www.rediscookbook.org/multiple_databases.html
  2. http://stackoverflow.com/questions/13386053/how-do-i-change-between-redis-database
  3. http://stackoverflow.com/questions/16221563/whats-the-point-of-multiple-redis-databases
  4. https://www.quora.com/What-are-5-mistakes-to-avoid-when-using-Redis
  5. http://lzone.de/cheat-sheet/Redis

 

StackExchange.Redis Basics

  1. The central object in StackExchange.Redis is the ConnectionMultiplexer class in the StackExchange.Redis namespace; this is the object that hides away the details of multiple servers. Because the ConnectionMultiplexer does a lot, it is designed to be shared and reused between callers. You should not create a ConnectionMultiplexer per operation. It is fully thread-safe and ready for this usage.
  2. Accessing a redis database is as simple as:
    • IDatabase db = redis.GetDatabase();

The object returned from GetDatabase is a cheap pass-thru object, and does not need to be stored

  1. RedisKey:
    • StackExchange.Redis represents keys by the RedisKey type. The good news, though, is that this has implicit conversions to and from both string and byte[], allowing both text and binary keys to be used without any complication
  1. RedisValue: 
    • Values can also need to represent typed primitive data – most commonly (in .NET terms) Int32Int64Double or Boolean. Because of this, RedisValue provides a lot more conversion support thanRedisKey
    • Note that while the conversions from primitives to RedisValue are implicit, many of the conversions from RedisValue to primitives are explicit: this is because it is very possible that these conversions will fail if the data does not have an appropriate value.
    • Note additionally that when treated numerically, redis treats a non-existent key as zero; for consistency with this, nil responses are treated as zero:

db.KeyDelete(“abc”);
int i = (int)db.StringGet(“abc”); // this is ZERO

  • If you need to detect the nil condition, then you can check for that:

db.KeyDelete(“abc”);
var value = db.StringGet(“abc”);
bool isNil = value.IsNull; // this is true

or perhaps more simply, just use the provided Nullable<T> support:

db.KeyDelete(“abc”);
var value = (int?)db.StringGet(“abc”); // behaves as you would expect

Documentation:

(colored ones just mean I have tried that documentation to some extent)

 

Code:

  1. FeaturizeBasic
  2. Redis Hashes Using StackExchange.Redis
  3. Redis Sets Using StackExchange.Redis

 

References:

  1. http://panuoksala.blogspot.com/2015/01/redis-hashes-and-net.html
Advertisements

Redis : Basics

Recently I have been playing around with Redis. In particular I have been trying how to integrate Redis with web services in Azure.

Redis is actually more like a data structures server, supporting very interesting data structures, and operations on them.

I am listing down some data structures, and corresponding when using these data structures.

  1. Strings
  2. Lists
  3. Sets
  4. Sorted Sets
    • Useful when fast access to the middle of a large collection of elements is important
  5. Hashes
    • Hashes are maps between string fields and string values, so they are the perfect data type to represent objects.
  6. Bit arrays
  7. Hyper Log Log

 

DECR, DECRBY, DEL, EXISTS, EXPIRE, GET, GETSET, HDEL, HEXISTS, HGET, HGETALL, HINCRBY, HKEYS, HLEN, HMGET, HMSET, HSET, HVALS, INCR, INCRBY, KEYS, LINDEX, LLEN, LPOP, LPUSH,LRANGE, LREM, LSET, LTRIM, MGET, MSET, MSETNX, MULTI, PEXPIRE, RENAME, RENAMENX, RPOP, RPOPLPUSH, RPUSH, SADD, SCARD, SDIFF, SDIFFSTORE, SET, SETEX, SETNX, SINTER, SINTERSTORE, SISMEMBER, SMEMBERS, SMOVE, SORT, SPOP, SRANDMEMBER, SREM, SUNION, SUNIONSTORE, TTL, TYPE, ZADD,ZCARD, ZCOUNT, ZINCRBY, ZRANGE, ZRANGEBYSCORE, ZRANK, ZREM, ZREMRANGEBYSCORE, ZREVRANGE, ZSCORE

 

Code on GitHub:

References:

Web Services : Data flow between the UI view and the backend APIs

These two articles helped me a lot in understanding how to handle data passing between the UI view and the backend APIs.

References:

  1. https://developer.mozilla.org/en-US/docs/Web/Guide/HTML/Forms/Sending_and_retrieving_form_data
  2. http://www.asp.net/web-api/overview/advanced/sending-html-form-data-part-1
  3. http://carehart.org/blog/client/index.cfm/2007/1/2/form_self_post

 

Azure Machine Learning : Model Retraining

References:

  1. https://azure.microsoft.com/en-us/documentation/articles/machine-learning-retrain-models-programmatically/
  2. https://azure.microsoft.com/en-us/blog/retraining-and-updating-azure-machine-learning-models-with-azure-data-factory/
  3. https://gallery.cortanaanalytics.com/Tutorial/No-code-Batch-Scoring-and-Retraining-1
  4. https://azure.microsoft.com/en-us/documentation/articles/machine-learning-consume-web-services/

Visual Studio : Project Structure

Have always been a bit curious to understand about VS’s project structure. Also, the use of config files and stuff. So here goes :

  • I create a solution ‘vsprojstructure’
    • I eventually wan to create a git repo out of it.
  • class library ‘configlib’
    • It has an app.config file
  • class library ‘algolib’
    • when algolib is independent, it doesn’t know anything about ‘configlib’
    • when i add ‘configlib’ as a reference, 2 things happen:
      • VS automatically updates the ‘Project Dependencies’ for ‘algolib’ project. It marks that ‘algolib’ depends on ‘configlib’
      • when i build the project, it copies ‘configlib’s dll and pdb files inside ‘algolibs’ bin\Debug folder
      • Note : it does not copy the .config file. This is worrisome
  • Now add a console app called ‘demo’
    • ‘demo’ has ‘algolib’
    • when i build the project, it copies ‘algolib’ and ‘configlib’s dll and pdb files inside ‘demos’  bin\Debug folder. However compilation fails (see code why)
    • So i also add ‘configlib’ as a reference. Compilation succeeds now.
    • However, it shows ‘AppSettings is empty’. This makes sense because as we noted before, the .config was never copied
  • It seems the only way to solve this problem is via post build events.
    • See reference.
    • Using build events.
    • This explains why in many cases we see projects having post build events writing to a common folder
  • Also, note by default the ‘demo’ exe will try to read its own exe. The .config file in the configlib directory needs to be crafted to ensure it reads its own .config file

 

Examples:

1.

robocopy $(SolutionDir)configlib\bin\Debug\ $(TargetDir) configlib.dll.config /xo /xn
IF %ERRORLEVEL% GEQ 8 exit 1
exit 0

2.

robocopy $(ProjectDir)dependencies $(SolutionDir)bin\$(Configuration) /xo /xn
IF %ERRORLEVEL% GEQ 8 exit 1
exit 0

Pro Tip:

  • A critical aspect is to understand the OutputPath parameter in the .config file
  • For each project, this is what controls where the output is going to.

Code:

Github

References:

 

 

Threading. Singletons. Generics. Locking Bug!

I recently came across this very interesting bug.  Note: the problem requires understanding of threads, generics, inheritance and locks.

In particular look at the Locking Bug code. Specifically, notice how the locking behaviour changes based on the types used for the generic base class.

  • When the types are same, the base class object used for locking is the same.
    • So locking can be enforced by doing something like
      • private static readonly object Padlock = new object();
      • lock (Padlock)
  • However, when the types are different, the compiler creates two completely different memory objects, hence the locking does not come into play
    • A possible solution is to create a separate internal class and enforce locking using the typeof that class
      • internal class ForLocking {}
      • lock (typeof(ForLocking))
    • If you dont prefer using typeof, another option is to create a separate class and have a static lockObject in it.
      • See code.
    • http://stackoverflow.com/questions/8185004/lockx-vs-locktypeofx

Code:

References: