Solutionizing .NET

Dynamic LINQ (DLINQ) is a LINQ extension provided in the VS 2008 Samples. Scott Guthrie provides a good overview here: Dynamic LINQ (Part 1: Using the LINQ Dynamic Query Library), but the executive summary is that it implements certain query operations on IQueryable (the non-generic variety), with filtering, grouping and projection specified with strings rather than statically-typed expressions.

I’ve never had a use for it, but a question on Stack Overflow caused me to take a second look…

…the selected groupbyvalue (Group) will always be a string, and the sum will always be a double, so I want to be able to cast into something like a List, where Result is an object with properties Group (string) and TotalValue (double).

Before we can solve the problem, let’s take a closer look at why it is being asked…

DynamicExpression.CreateClass

We can use the simplest of dynamic queries to explore a bit:

[Test]
public void DLINQ_IdentityProjection_ReturnsDynamicClass()
{
    IQueryable nums = Enumerable.Range(1, 5).AsQueryable();
    IQueryable q = nums.Select("new (it as Value)");
    Type elementType = q.ElementType;

    Assert.AreEqual("DynamicClass1", elementType.Name);
    CollectionAssert.AreEqual(new[] { typeof(int) },
        elementType.GetProperties().Select(p => p.PropertyType).ToArray());
}

DLINQ defines a special expression syntax for projection that is used to specify what values should be returned and how. it refers to the current element, which in our case is an int.

The result in question comes from DynamicQueryable.Select():

public static IQueryable Select(this IQueryable source, string selector, params object[] values)
{
    LambdaExpression lambda = DynamicExpression.ParseLambda(source.ElementType, null, selector, values);
    return source.Provider.CreateQuery(
        Expression.Call(
            typeof(Queryable), "Select",
            new Type[] { source.ElementType, lambda.Body.Type },
            source.Expression, Expression.Quote(lambda)));
}

The non-generic return type suggest that the type of the values returned is unknown at compile time. If we check an element’s type at runtime, we’ll see something like DynamicClass1. Tracing down the stack from DynamicExpression.ParseLambda(), we eventually find that DynamicClass1 is generated by a call to DynamicExpression.CreateClass() in ExpressionParser.ParseNew(). CreateClass() in turn delegates to a static ClassFactory which manages a dynamic assembly in the current AppDomain to hold the new classes, each generated by Reflection.Emit. The resulting type is then used to generate the MemberInit expression that constructs the object.

Dynamic to Static

While dynamic objects are useful in some situations (thus support in C# 4), in this case we want to use static typing. Let’s specify our result type with a generic method:

IQueryable<TResult> Select<TResult>(this IQueryable source, string selector, params object[] values);

We just need a mechanism to insert our result type into DLINQ to supersede the dynamic result. This is surprisingly easy to implement, as ParseLambda() already accepts a resultType argument. We just need to capture it…

private Type resultType;
public Expression Parse(Type resultType)
{
    this.resultType = resultType;
    int exprPos = token.pos;
    // ...

…and then update ParseNew() to use the specified type:

Expression ParseNew()
{
    // ...
    NextToken();
    Type type = this.resultType ?? DynamicExpression.CreateClass(properties);
    MemberBinding[] bindings = new MemberBinding[properties.Count];
    for (int i = 0; i < bindings.Length; i++)
        bindings[i] = Expression.Bind(type.GetProperty(properties[i].Name), expressions[i]);
    return Expression.MemberInit(Expression.New(type), bindings);
}

If resultType is null, as it is in the existing Select() implementation, a DynamicClass is used instead.

The generic Select<TResult> is then completed by referencing TResult as appropriate:

public static IQueryable<TResult> Select<TResult>(this IQueryable source, string selector, params object[] values)
{
    LambdaExpression lambda = DynamicExpression.ParseLambda(source.ElementType, typeof(TResult), selector, values);
    return source.Provider.CreateQuery<TResult>(
        Expression.Call(
            typeof(Queryable), "Select",
            new Type[] { source.ElementType, typeof(TResult) },
            source.Expression, Expression.Quote(lambda)));
}

With the following usage:

public class ValueClass { public int Value { get; set; } }

[Test]
public void DLINQ_IdentityProjection_ReturnsStaticClass()
{
    IQueryable nums = Enumerable.Range(1, 5).AsQueryable();
    IQueryable<ValueClass> q = nums.Select<ValueClass>("new (it as Value)");
    Type elementType = q.ElementType;

    Assert.AreEqual("ValueClass", elementType.Name);
    CollectionAssert.AreEqual(nums.ToArray(), q.Select(v => v.Value).ToArray());
}

Note that the property names in TResult must match those in the Select query exactly. Changing the query to “new (it as value)” results in an unhandled ArgumentNullException in the Expression.Bind() call seen in the for loop of ParseNew() above, as the “value” property cannot be found.

Selecting Anonymous Types

So we can select dynamic types or existing named types, but what if we want to have the benefits of static typing without having to declare a dedicated ValueClass, as we can with anonymous types and normal static LINQ? As a variation on techniques used elsewhere, let’s can define an overload of Select() that accepts an instance of the anonymous type whose values we will ignore but using its type to infer the desired return type. The overload is trivial:

public static IQueryable<TResult> Select<TResult>(this IQueryable source, TResult template, string selector, params object[] values)
{
    return source.Select<TResult>(selector, values);
}

With usage looking like this (note the required switch to var q):

[Test]
public void DLINQ_IdentityProjection_ReturnsStaticClass()
{
    IQueryable nums = Enumerable.Range(1, 5).AsQueryable();
    var q = nums.Select(new { Value = 0 }, "new (it as Value)");
    Type elementType = q.ElementType;

    Assert.IsTrue(elementType.Name.Contains("AnonymousType"));
    CollectionAssert.AreEqual(nums.ToArray(), q.Select(v => v.Value).ToArray());
}

However, if we try the above we encounter an unfortunate error:

The property ‘Int32 Value’ has no ‘set’ accessor

As you may or may not know, anonymous types in C# are immutable (modulo changes to objects they reference), with their values set through a compiler-generated constructor. (I’m not sure if this is true in VB.) With this knowledge in hand, we can update ParseNew() to check if resultType has such a constructor that we could use instead:

    // ...
    Type type = this.resultType ?? DynamicExpression.CreateClass(properties);

    var propertyTypes = type.GetProperties().Select(p => p.PropertyType).ToArray();
    var ctor = type.GetConstructor(propertyTypes);
    if (ctor != null)
        return Expression.New(ctor, expressions);

    MemberBinding[] bindings = new MemberBinding[properties.Count];
    for (int i = 0; i < bindings.Length; i++)
        bindings[i] = Expression.Bind(type.GetProperty(properties[i].Name), expressions[i]);
    return Expression.MemberInit(Expression.New(type), bindings);
}

And with that we can now project from a dynamic query onto static types, both named and anonymous, with a reasonably natural interface.

Due to licensing I can’t post the full example, but if you’re at all curious about Reflection.Emit or how DLINQ works I would encourage you to dive in and let us know what else you come up with. Things will get even more interesting with the combination of LINQ, the DLR and C# 4’s dynamic in the coming months.