C.E.'s answer is probably the best, relying as it does on a well-constructed Java library with serious people behind it, but I thought it'd be nice to have a pure Mathematica solution as well, so I sunk some time into building an XMLGraph object which I put here.
The basic idea is to store the two pieces of data I talked about before, a Graph that holds all of the structural relationships, and an Association encoding all of the data as a flat data structure.
Each node then gets a name, e.g. "head:ee1d9544" which goes into the Graph and an entry in the Association like:
<| "Parent" -> "html:80da223b", "Type" -> "head", "Name" -> "head:ee1d9544", "Meta" -> <||>, "Children" -> {} |>
This way it's easy to mutate the object by changing entries in the flat association and it's easy to query the Graph to get out structure.
I then used this to implement conversion from XMLElement and CSS selectors, so here's a little rundown of what we can do (although development work isn't finished):
Get["https://github.com/b3m2a1/mathematica-tools/raw/master/XMLGraph.m"]; testXML = Import["https://developer.github.com/apps/building-oauth-apps/understanding-scopes-for-oauth-apps/#available-scopes", {"HTML", "XMLObject"} ]; g = XMLGraph[testXML]
Then we can query, say, all the code elements that follow a td element (this was what originally spurred me to make this). The Mathematica code for this query in the standard way looks like:
t1 = Cases[testXML, XMLElement["td", _, code_] :> FirstCase[code, XMLElement["code", _, {s_, ___}] :> s, Nothing, \[Infinity]], \[Infinity]];
The graph query looks like:
t2 = First /@ Values@ g@"Children"[g@"Select"["td code"]]; DeleteDuplicates@Sort@t1 == DeleteDuplicates@Sort@t2 True
Performance-wise we're a bit slower:
t1 = Cases[testXML, XMLElement["td", _, code_] :> FirstCase[code, XMLElement["code", _, {s_, ___}] :> s, Nothing, \[Infinity]], \[Infinity]]; // RepeatedTiming {0.00045, Null} t2 = First /@ Values@ g@"Children"[g@"Select"["td code"]]; // RepeatedTiming {0.0091, Null}
But other things like property look-ups are so much faster like this. Also we can do queries over the Association instead of the graph (when possible) which is competitive with Cases:
Cases[testXML, XMLElement["code", _, _], \[Infinity]]; // RepeatedTiming {0.00015, Null} nodes = g@"Select"["code"]; // RepeatedTiming {0.00077, Null}
And once we have this set of nodes it's easy and fast to get all sorts of information out:
g@"Children"[nodes]; // RepeatedTiming {0.00012, Null} g@"Parent"[nodes]; // RepeatedTiming {0.000082, Null} g@"Attribute"[nodes, "colspan"]; // RepeatedTiming {0.000084, Null}
This is rather more challenging to do well with XMLElement.
We can also do easy modifications like this:
newG = g@"ModifyNodes"[ Thread[nodes -> <|"colspan" -> 2|>] ]; g@"Attribute"[nodes, "colspan"] // Take[#, 5] & <|"td:ae4c93a0" -> "1", "td:c3fbb1ce" -> "1", "td:77adbfa1" -> "1", "td:3efd9c29" -> "1", "td:69452f14" -> "1"|> newG@"Attribute"[nodes, "colspan"] // Take[#, 5] & <|"td:ae4c93a0" -> 2, "td:c3fbb1ce" -> 2, "td:77adbfa1" -> 2, "td:3efd9c29" -> 2, "td:69452f14" -> 2|>
Or we can change some types and dump to XML:
mods = {newG["Root"] -> <|"Type" -> "xml"|>}; newG@"ModifyNodes"[mods]@"XML" // Shallow XMLElement["xml", <|"version" -> "-//W3C//DTD HTML 4.01 Transitional//EN", "lang" -> "en", "prefix" -> "og: http://ogp.me/ns#", {"http://www.w3.org/2000/xmlns/", "xmlns"} -> "http://www.w3.org/1999/xhtml"|>, {XMLElement[<<3>>], XMLElement[<<3>>]}]
The full list of supported methods is somewhat in flux, but it'll be exposed eventually and every method has a Symbol backing it (e.g. XMLGraph`Package`SelectNodes for "Select") so that's always possible to look at.
As a final usage note, with the structure as it's set-up the storage can always be handled by as HashTable which would allow this XMLGraph to be a truly mutable object in the way the object-oriented interface suggests it could be.
Hopefully this gives a sense of how this may be done and why this type of structure is useful. And of course, this is a preliminary package, so performance enhancements are always possible.
