How to deserialize JSON with duplicate property names in the same object

Question

I have a JSON string that I expect to contain duplicate keys that I am unable to make JSON.NET happy with.

I was wondering if anybody knows the best way (maybe using JsonConverter? ) to get JSON.NET to change a JObject's child JObjects into to JArrays when it sees duplicate key names ?

// For example: This gives me a JObject with a single "JProperty\JObject" child. var obj = JsonConvert.DeserializeObject<object>("{ \"HiThere\":1}"); // This throws: // System.ArgumentException : Can not add Newtonsoft.Json.Linq.JValue to Newtonsoft.Json.Linq.JObject. obj = JsonConvert.DeserializeObject<object>("{ \"HiThere\":1, \"HiThere\":2, \"HiThere\":3 }");

The actual JSON I am trying to deserialize is much more complicated and the duplicates are nested at multiple levels. But the code above demonstrates why it fails for me.

I understand that the JSON is not recommended which is why I am asking if JSON.NET has a way to work around this. For argument's sake let's say I do not have control over the JSON. I actually do use a specific type for the parent object but the particular property that is having trouble will either be a string or another nested JSON object. The failing property type is "object" for this reason.

Actually, JSON objects with duplicate names are perfectly legal. ECMA-404/json.org are silent on the issue. Meanwhile RFC7159 Section 4 says The names within an object SHOULD be unique (SHOULD = recommended, not MUST) along with some discussion of the implications. — Serguei
– Serguei, Commented Jun 17, 2016 at 17:18

TylerH · Accepted Answer · 2023-09-20 20:15:50Z

While a JObject cannot contain properties with duplicate names, the JsonTextReader used to populate it during deserialization does not have such a restriction (this makes sense if you think about it: it's a forward-only reader; it is not concerned with what it has read in the past).

Here is some code that will populate a hierarchy of JTokens, converting property values to JArrays as necessary if a duplicate property name is encountered in a particular JObject.

Since I don't know your actual JSON and requirements, you may need to make some adjustments to it, but it's something to start with at least.

Here's the code:

public static JToken DeserializeAndCombineDuplicates(JsonTextReader reader) { if (reader.TokenType == JsonToken.None) { reader.Read(); } if (reader.TokenType == JsonToken.StartObject) { reader.Read(); JObject obj = new JObject(); while (reader.TokenType != JsonToken.EndObject) { string propName = (string)reader.Value; reader.Read(); JToken newValue = DeserializeAndCombineDuplicates(reader); JToken existingValue = obj[propName]; if (existingValue == null) { obj.Add(new JProperty(propName, newValue)); } else if (existingValue.Type == JTokenType.Array) { CombineWithArray((JArray)existingValue, newValue); } else // Convert existing non-array property value to an array { JProperty prop = (JProperty)existingValue.Parent; JArray array = new JArray(); prop.Value = array; array.Add(existingValue); CombineWithArray(array, newValue); } reader.Read(); } return obj; } if (reader.TokenType == JsonToken.StartArray) { reader.Read(); JArray array = new JArray(); while (reader.TokenType != JsonToken.EndArray) { array.Add(DeserializeAndCombineDuplicates(reader)); reader.Read(); } return array; } return new JValue(reader.Value); } private static void CombineWithArray(JArray array, JToken value) { if (value.Type == JTokenType.Array) { foreach (JToken child in value.Children()) array.Add(child); } else { array.Add(value); } }

And here's a demo:

class Program { static void Main(string[] args) { string json = @" { ""Foo"" : 1, ""Foo"" : [2], ""Foo"" : [3, 4], ""Bar"" : { ""X"" : [ ""A"", ""B"" ] }, ""Bar"" : { ""X"" : ""C"", ""X"" : ""D"" }, }"; using (StringReader sr = new StringReader(json)) using (JsonTextReader reader = new JsonTextReader(sr)) { JToken token = DeserializeAndCombineDuplicates(reader); Dump(token, ""); } } private static void Dump(JToken token, string indent) { Console.Write(indent); if (token == null) { Console.WriteLine("null"); return; } Console.Write(token.Type); if (token is JProperty) Console.Write(" (name=" + ((JProperty)token).Name + ")"); else if (token is JValue) Console.Write(" (value=" + token.ToString() + ")"); Console.WriteLine(); if (token.HasValues) foreach (JToken child in token.Children()) Dump(child, indent + " "); } }

Output:

Object Property (name=Foo) Array Integer (value=1) Integer (value=2) Integer (value=3) Integer (value=4) Property (name=Bar) Array Object Property (name=X) Array String (value=A) String (value=B) Object Property (name=X) Array String (value=C) String (value=D)

Thanks Brian Rogers - You've filled in some gaps in my understanding of the JsonTextReader. I actually have a JsonConverter already assigned to my "object" property that I was using to abstract away the traces of JSON.NET. But in my unit test I was just trying to deserialize my "bad" JSON directly without the converter so it didn't occur to me that I could edit that to resolve the issue. I will post the code below as an additional answer.

Brian Booth · Accepted Answer · 2013-12-21 16:43:08Z

Brian Rogers - Here is the helper function of the JsonConverter that I wrote. I modified it based on your comments about how a JsonTextReader is just a forward-reader doesn't care about duplicate values.

private static object GetObject(JsonReader reader) { switch (reader.TokenType) { case JsonToken.StartObject: { var dictionary = new Dictionary<string, object>(); while (reader.Read() && (reader.TokenType != JsonToken.EndObject)) { if (reader.TokenType != JsonToken.PropertyName) throw new InvalidOperationException("Unknown JObject conversion state"); string propertyName = (string) reader.Value; reader.Read(); object propertyValue = GetObject(reader); object existingValue; if (dictionary.TryGetValue(propertyName, out existingValue)) { if (existingValue is List<object>) { var list = existingValue as List<object>; list.Add(propertyValue); } else { var list = new List<object> {existingValue, propertyValue}; dictionary[propertyName] = list; } } else { dictionary.Add(propertyName, propertyValue); } } return dictionary; } case JsonToken.StartArray: { var list = new List<object>(); while (reader.Read() && (reader.TokenType != JsonToken.EndArray)) { object propertyValue = GetObject(reader); list.Add(propertyValue); } return list; } default: { return reader.Value; } } }

DarcyThomas · Accepted Answer · 2013-12-21 02:50:44Z

You should not be using a generic type of object, it should be a more specific type.

However you json is malformed which is you rmain problem

You have :

"{ \"HiThere\":1, \"HiThere\":2, \"HiThere\":3 }"

But it should be:

"{"HiTheres": [{\"HiThere\":1}, {\"HiThere\":2}, {\"HiThere\":3} ]}"

Or

"{ \"HiThereOne\":1, \"HiThereTwo\":2, \"HiThereThree\":3 }"

You json is one object with 3 fields with all the same name ("HiThere"). Which wont work.

The json I have shown gives: An array (HiTheres) of three objects each with a field of HiThere Or One object with three field with different names. (HiThereOne, HiThereTwo, "HiThereThree)

Have a look at http://jsoneditoronline.org/index.html And http://json.org/

Did you use a proper type? e.g., JsonConvert.DeserializeObject<HiThereClass>()
Although you're right about the duplicate names, "{"HiTheres": [\"HiThere\":1, \"HiThere\":2, \"HiThere\":3 ]}" is still not valid JSON.

Collectives™ on Stack Overflow

How to deserialize JSON with duplicate property names in the same object

3 Answers 3

1 Comment

Comments

3 Comments

Linked

Hot Network Questions