Is there a speed different between servicestack.redis GetAll().where() and GetByIds()

Question

I was using servicestack.redis recently, and I need query from IRedisTypedClient. I know all data is in memory, but still want to know, is there a speed different between GetAll().Where() and GetByIds()?

GetAll() and GetByIds() are two methods provided by servicestack.redis.

Use GetAll() can continue search in result(by lambda), that means I can use some custom conditions, but I don't know, whether that will load all data from redis memory then search in IEnumable<T>, and whether the search speed will slower than GetByIds().

@BugFinder you are right, i'll try it tonight, when i find out tell your guys. — Bucketcode, Aug 16 '16 at 09:00

score 2 · Answer 1 · edited Aug 16 '16 at 12:52

I just did a experiment, I stored 1 million object(ps:there is a servicestack's bug, can only store about half million object once).

queried with these two methods.

DateTime beginDate = DateTime.Now;
Debug.WriteLine("查询1开始");`  
Website site = WebsiteRedis.GetByCondition(w => w.Name == "网址2336677").First();
double time = (DateTime.Now - beginDate).TotalMilliseconds;
Debug.WriteLine("耗时：" + time + "ms");

DateTime beginDate2 = DateTime.Now;
Debug.WriteLine("查询2开始");
Website site2 = WebsiteRedis.GetByID(new Guid("29284415-5de0-4781-bea4-5e01332814b2"));
double time2 = (DateTime.Now - beginDate2).TotalMilliseconds;
Debug.WriteLine("耗时：" + time2 + "ms");

Result is
GetAll().Where() - takes 19 seconds,
GetById()- take 190ms.

I guess it's because servicestack use object id as redis key, so never use GetAll().Where() as query, every object should related with id and use GetById() as query. GetAll() should use on object with less records.

I edit it now, still have some problem with markdown code, but ok to see. — Bucketcode, Aug 16 '16 at 12:49

score 2 · Answer 2 · edited May 23 '17 at 12:32

You can have a look at the implementations of GetAll and GetByIds to see how they work.

GetByIds just converts all Ids to a fully qualified Key which each entry is stored under then calls GetValues() which creates a single MGET request to fetch all the values:

public IList<T> GetByIds(IEnumerable ids)
{
    if (ids != null)
    {
        var urnKeys = ids.Map(x => client.UrnKey<T>(x));
        if (urnKeys.Count != 0)
            return GetValues(urnKeys);
    }

    return new List<T>();
}

public IList<T> GetAll()
{
    var allKeys = client.GetAllItemsFromSet(this.TypeIdsSetKey);
    return this.GetByIds(allKeys.ToArray());
}

GetAll fetches all the Ids from the TypeIdsSetKey (i.e. Redis SET containing all ids for that Type) then calls GetByIds().

So GetByIds is faster because it makes one less call to Redis, but together they only make 2 Redis operations.

Note they both return an in memory .NET List<T> so you can use LINQ to further filter the returned results, but it returns all results for that Type and the filtering is performed on the client so this isn't efficient for large datasets. Instead you should look at creating manual indexes using Redis SETs for common queries.

awesome, i just reading the question you linked in answer, and checking your profile, turn out you answer me at the same time. — Bucketcode, Aug 16 '16 at 14:31

Is there a speed different between servicestack.redis GetAll().where() and GetByIds()

2 Answers2