JsonConverter applied to class causes JsonConverter applied to property to be ignored

Question

I defined two JsonConverter classes. One I attach to the class, the other I attach to a property of that class. If I only attach the converter to the property, it works fine. As soon as I attach a separate converter to the class, it ignores the one attached to the property. How can I make it not skip such JsonConverterAttributes?

Here is the class-level converter (which I adapted from this: Alternate property name while deserializing). I attach it to the test class like so:

[JsonConverter(typeof(FuzzyMatchingJsonConverter<JsonTestData>))]

Then here is the FuzzyMatchingJsonConverter itself:

using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
using Optimizer.models;
using System;
using System.Collections.Generic;
using System.Linq;

namespace Optimizer.Serialization
{
    /// <summary>
    /// Permit the property names in the Json to be deserialized to have spelling variations and not exactly match the
    /// property name in the object. Thus puntuation, capitalization and whitespace differences can be ignored.
    /// 
    /// NOTE: As implemented, this can only deserialize objects from a string, not serialize from objects to a string.
    /// </summary>
    /// <seealso cref="https://stackoverflow.com/questions/19792274/alternate-property-name-while-deserializing"/>
    public class FuzzyMatchingJsonConverter<T> : JsonConverter
    {
        /// <summary>
        /// Map the json property names to the object properties.
        /// </summary>
        private static DictionaryToObjectMapper<T> Mapper { get; set; } = null;

        private static object SyncToken { get; set; } = new object();

        static void InitMapper(IEnumerable<string> jsonPropertyNames)
        {
            if (Mapper == null)
                lock(SyncToken)
                {
                    if (Mapper == null)
                    {
                        Mapper = new DictionaryToObjectMapper<T>(
                            jsonPropertyNames,
                            EnumHelper.StandardAbbreviations,
                            ModelBase.ACCEPTABLE_RELATIVE_EDIT_DISTANCE,
                            ModelBase.ABBREVIATION_SCORE
                        );
                    }
                }
            else
            {
                lock(SyncToken)
                {
                    // Incremental mapping of additional attributes not seen the first time for the second and subsequent objects.
                    // (Some records may have more attributes than others.)
                    foreach (var jsonPropertyName in jsonPropertyNames)
                    {
                        if (!Mapper.CanMatchKeyToProperty(jsonPropertyName))
                            throw new MatchingAttributeNotFoundException(jsonPropertyName, typeof(T).Name);
                    }
                }
            }
        }

        public override bool CanConvert(Type objectType) => objectType.IsClass;

        /// <summary>
        /// If false, this class cannot serialize (write) objects.
        /// </summary>
        public override bool CanWrite { get => false; }

        /// <summary>
        /// Call the default constructor for the object and then set all its properties,
        /// matching the json property names to the object attribute names.
        /// </summary>
        /// <param name="reader"></param>
        /// <param name="objectType">This should match the type parameter T.</param>
        /// <param name="existingValue"></param>
        /// <param name="serializer"></param>
        /// <returns>The deserialized object of type T.</returns>
        public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
        {
            // Note: This assumes that there is a default (parameter-less) constructor and not a constructor tagged with the JsonCOnstructorAttribute.
            // It would be better if it supported those cases.
            object instance = objectType.GetConstructor(Type.EmptyTypes).Invoke(null);
            JObject jo = JObject.Load(reader);
            InitMapper(jo.Properties().Select(jp => jp.Name));

            foreach (JProperty jp in jo.Properties())
            {
                var prop = Mapper.KeyToProperty[jp.Name];
                prop?.SetValue(instance, jp.Value.ToObject(prop.PropertyType, serializer));
            }
            return instance;
        }

        public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
        {
            throw new NotImplementedException();
        }
    }
}

Do not get bogged down with DictionaryToObjectMapper (it is proprietary, but uses fuzzy matching logic to deal with spelling variations). Here is the next JsonConverter, that will change "Y", "Yes", "T", "True", etc into Boolean true values. I adapted it from this source: https://gist.github.com/randyburden/5924981

using System;
using System.Linq;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;

namespace Optimizer.Serialization
{
    /// <summary>
    /// Handles converting JSON string values into a C# boolean data type.
    /// </summary>
    /// <see cref="https://gist.github.com/randyburden/5924981"/>
    public class BooleanJsonConverter : JsonConverter
    {
        private static readonly string[] Truthy = new[] { "t", "true", "y", "yes", "1" };
        private static readonly string[] Falsey = new[] { "f", "false", "n", "no", "0" };

        /// <summary>
        /// Parse a Boolean from a string where alternative spellings are permitted, such as 1, t, T, true or True for true.
        /// 
        /// All values that are not true are considered false, so no parse error will occur.
        /// </summary>
        public static Func<object, bool> ParseBoolean
            = (obj) => { var b = (obj ?? "").ToString().ToLower().Trim(); return Truthy.Any(t => t.Equals(b)); };

        public static bool ParseBooleanWithValidation(object obj)
        {
            var b = (obj ?? "").ToString().ToLower().Trim();
            if (Truthy.Any(t => t.Equals(b)))
                return true;
            if (Falsey.Any(t => t.Equals(b)))
                return false;
            throw new ArgumentException($"Unable to convert ${obj}into a Boolean attribute.");
        }

        #region Overrides of JsonConverter

        /// <summary>
        /// Determines whether this instance can convert the specified object type.
        /// </summary>
        /// <param name="objectType">Type of the object.</param>
        /// <returns>
        /// <c>true</c> if this instance can convert the specified object type; otherwise, <c>false</c>.
        /// </returns>
        public override bool CanConvert(Type objectType)
        {
            // Handle only boolean types.
            return objectType == typeof(bool);
        }

        /// <summary>
        /// Reads the JSON representation of the object.
        /// </summary>
        /// <param name="reader">The <see cref="T:Newtonsoft.Json.JsonReader"/> to read from.</param>
        /// <param name="objectType">Type of the object.</param>
        /// <param name="existingValue">The existing value of object being read.</param>
        /// <param name="serializer">The calling serializer.</param>
        /// <returns>
        /// The object value.
        /// </returns>
        public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
            => ParseBooleanWithValidation(reader.Value);


        /// <summary>
        /// Specifies that this converter will not participate in writing results.
        /// </summary>
        public override bool CanWrite { get { return false; } }

        /// <summary>
        /// Writes the JSON representation of the object.
        /// </summary>
        /// <param name="writer">The <see cref="T:Newtonsoft.Json.JsonWriter"/> to write to.</param><param name="value">The value.</param><param name="serializer">The calling serializer.</param>
        public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
        {
            //TODO: Implement for serialization
            //throw new NotImplementedException("Serialization of Boolean");
            // I have no idea if this is correct:
            var b = (bool)value;
            JToken valueToken;
            valueToken = JToken.FromObject(b); 
            valueToken.WriteTo(writer);
        }

        #endregion Overrides of JsonConverter
    }
}

And here is how I created the test class used in my unit tests:

[JsonConverter(typeof(FuzzyMatchingJsonConverter<JsonTestData>))]
public class JsonTestData: IEquatable<JsonTestData>
{
    public string TestId { get; set; }
    public double MinimumDistance { get; set; }

    [JsonConverter(typeof(BooleanJsonConverter))]
    public bool TaxIncluded { get; set; }

    [JsonConverter(typeof(BooleanJsonConverter))]
    public bool IsMetsFan { get; set; }

    [JsonConstructor]
    public JsonTestData()
    {
        TestId = null;
        MinimumDistance = double.NaN;
        TaxIncluded = false;
        IsMetsFan = false;
    }

    public JsonTestData(string testId, double minimumDistance, bool taxIncluded, bool isMetsFan)
    {
        TestId = testId;
        MinimumDistance = minimumDistance;
        TaxIncluded = taxIncluded;
        IsMetsFan = isMetsFan;
    }

    public override bool Equals(object obj) => Equals(obj as JsonTestData);

    public bool Equals(JsonTestData other)
    {
        if (other == null) return false;
        return ((TestId ?? "") == other.TestId)
            && (MinimumDistance == other.MinimumDistance)
            && (TaxIncluded == other.TaxIncluded)
            && (IsMetsFan == other.IsMetsFan);
    }

    public override string ToString() => $"TestId: {TestId}, MinimumDistance: {MinimumDistance}, TaxIncluded: {TaxIncluded}, IsMetsFan: {IsMetsFan}";

    public override int GetHashCode()
    {
        return -1448189120 + EqualityComparer<string>.Default.GetHashCode(TestId);
    }
}

Answer 1

The reason that [JsonConverter(typeof(BooleanJsonConverter))] as applied to your properties is not working is that you have supplied a JsonConverter for the containing type, and are not calling the applied converter(s) for its members inside your ReadJson() method.

When a converter is not applied to a type, prior to (de)serialization Json.NET uses reflection to create a JsonContract for the type that specifies how to map the type from and to JSON. In the case of an object with properties, a JsonObjectContract is generated that includes methods to construct and populate the type and lists all the members of the type to be serialized, including their names and any applied converters. Once the contract is built, the method JsonSerializerInternalReader.PopulateObject() uses it to actually deserialize an object.

When a converter is applied to a type, all the logic described above is skipped. Instead JsonConverter.ReadJson() must do everything including deserializing and setting all member values. If those members happen to have converters applied, ReadJson() will need to notice this fact and manually invoke the converter. This is what your converter needs to be doing around here:

        foreach (JProperty jp in jo.Properties())
        {
            var prop = Mapper.KeyToProperty[jp.Name];
            // Check for and use [JsonConverter(typeof(...))] if applied to the member.
            prop?.SetValue(instance, jp.Value.ToObject(prop.PropertyType, serializer));
        }

So, how to do this? One way would be to use c# reflection tools to check for the appropriate attribute(s). Luckily, Json.NET has already done this for you in constructing its JsonObjectContract; you can get access to it from within ReadJson() simply by calling:

var contract = serializer.ContractResolver.ResolveContract(objectType) as JsonObjectContract;

Having done that you can use the already-constructed contract to guide your deserialization.

Since you don't provide a working example of your FuzzyMatchingJsonConverter I've created something similar that will allow both snake case and pascal case properties to be deserialized into an object with camel case naming:

public abstract class FuzzyMatchingJsonConverterBase : JsonConverter
{
    protected abstract JsonProperty FindProperty(JsonObjectContract contract, string propertyName);

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        var contract = serializer.ContractResolver.ResolveContract(objectType) as JsonObjectContract;
        if (contract == null)
            throw new JsonSerializationException(string.Format("Contract for type {0} is not a JsonObjectContract", objectType));

        if (reader.TokenType == JsonToken.Null)
            return null;

        if (reader.TokenType != JsonToken.StartObject)
            throw new JsonSerializationException(string.Format("Unexpected token {0}", reader.TokenType));

        existingValue = existingValue ?? contract.DefaultCreator();

        while (reader.Read())
        {
            switch (reader.TokenType)
            {
                case JsonToken.Comment:
                    break;
                case JsonToken.PropertyName:
                    {
                        var propertyName = (string)reader.Value;
                        reader.ReadAndAssert();
                        var jsonProperty = FindProperty(contract, propertyName);
                        if (jsonProperty == null)
                            continue;
                        object itemValue;
                        if (jsonProperty.Converter != null && jsonProperty.Converter.CanRead)
                        {
                            itemValue = jsonProperty.Converter.ReadJson(reader, jsonProperty.PropertyType, jsonProperty.ValueProvider.GetValue(existingValue), serializer);
                        }
                        else
                        {
                            itemValue = serializer.Deserialize(reader, jsonProperty.PropertyType);
                        }
                        jsonProperty.ValueProvider.SetValue(existingValue, itemValue);
                    }
                    break;
                case JsonToken.EndObject:
                    return existingValue;
                default:
                    throw new JsonSerializationException(string.Format("Unexpected token {0}", reader.TokenType));
            }
        }
        throw new JsonReaderException("Unexpected EOF!");
    }

    public override bool CanWrite { get { return false; } }

    public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
    {
        throw new NotImplementedException();
    }
}

public abstract class FuzzySnakeCaseMatchingJsonConverterBase : FuzzyMatchingJsonConverterBase
{
    protected override JsonProperty FindProperty(JsonObjectContract contract, string propertyName)
    {
        // Remove snake-case underscore.
        propertyName = propertyName.Replace("_", "");
        // And do a case-insensitive match.
        return contract.Properties.GetClosestMatchProperty(propertyName);
    }
}

// This type should be applied via attributes.
public class FuzzySnakeCaseMatchingJsonConverter : FuzzySnakeCaseMatchingJsonConverterBase
{
    public override bool CanConvert(Type objectType)
    {
        throw new NotImplementedException();
    }
}

// This type can be used in JsonSerializerSettings.Converters
public class GlobalFuzzySnakeCaseMatchingJsonConverter : FuzzySnakeCaseMatchingJsonConverterBase
{
    readonly IContractResolver contractResolver;

    public GlobalFuzzySnakeCaseMatchingJsonConverter(IContractResolver contractResolver)
    {
        this.contractResolver = contractResolver;
    }

    public override bool CanConvert(Type objectType)
    {
        if (objectType.IsPrimitive || objectType == typeof(string))
            return false;
        var contract = contractResolver.ResolveContract(objectType);
        return contract is JsonObjectContract;
    }
}

public static class JsonReaderExtensions
{
    public static void ReadAndAssert(this JsonReader reader)
    {
        if (reader == null)
            throw new ArgumentNullException();
        if (!reader.Read())
            throw new JsonReaderException("Unexpected EOF!");
    }
}

Then you would apply it as follows:

[JsonConverter(typeof(FuzzySnakeCaseMatchingJsonConverter))]
public class JsonTestData
{
    public string TestId { get; set; }

    public double MinimumDistance { get; set; }

    [JsonConverter(typeof(BooleanJsonConverter))]
    public bool TaxIncluded { get; set; }

    [JsonConverter(typeof(BooleanJsonConverter))]
    public bool IsMetsFan { get; set; }
}

Notes:

• I avoided pre-loading the JSON into an intermediate JToken hierarchy, since there was no need to do so.

• You don't provide a working example of your own converter so I can't fix it for you in this answer, but you would want to make it inherit from FuzzyMatchingJsonConverterBase and then write your on version of protected abstract JsonProperty FindProperty(JsonObjectContract contract, string propertyName);.

• You might also need to check and use other properties of JsonProperty such as JsonProperty.ItemConverter, JsonProperty.Ignored, JsonProperty.ShouldDeserialize and so on. But if you do, you may eventually end up duplicating the entire logic of JsonSerializerInternalReader.PopulateObject().

• A null JSON value should be checked for near the beginning of ReadJson().

Sample working .Net fiddle that shows that the following JSON can be deserialized successfully, and thus that both the type and member converters are getting invoked:

{
  "test_id": "hello",
  "minimum_distance": 10101.1,
  "tax_included": "yes",
  "is_mets_fan": "no"
}