14
$\begingroup$

Let's use the example Dataset:

dataset = Dataset[{ <|"a" -> 1, "b" -> "x", "c" -> {1}|>, <|"a" -> 2, "b" -> "y", "c" -> {2, 3}|>, <|"a" -> 3, "b" -> "z", "c" -> {3}|>, <|"a" -> 4, "b" -> "x", "c" -> {4, 5}|>, <|"a" -> 5, "b" -> "y", "c" -> {5, 6, 7}|>, <|"a" -> 6, "b" -> "z", "c" -> {}|>}]

And data for a new field "d" that I'd like to add:

d = {6, 5, 4, 3, 2, 1}

I can add the field by completely unpackaging and repackaging the data:

Dataset[Map[Association,Transpose[Append[Transpose[Normal[Normal[dataset]]], Thread["d"->d]]]]]

There must be a simpler way! I would like to do this routinely and with large datasets, so I'm looking for something more compact and potentially much more efficient. What am I missing?

Improve this question
$\endgroup$
2
  • $\begingroup$ Related: how-can-i-add-a-column-into-a-existing-dataset. $\endgroup$ Commented Nov 3, 2014 at 21:11
  • $\begingroup$ @SjoerdC.deVries, that is helpful, but appears to only apply when the new column is some function of the existing ones. $\endgroup$ Commented Nov 3, 2014 at 21:14

4 Answers 4

Reset to default
7
$\begingroup$

With this auxiliary function:

tr = Transpose[#, AllowedHeads -> All] &;

you can do

dataset[tr /* Append["d" -> {6, 5, 4, 3, 2, 1}] /* tr]

The formatting of the result won't be as nice as the original, because of type inference limitations, but the result is correct.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Could you please show how to generalize to ds=ExampleData[{"Dataset", "Planets"}] and, say, we want to add a column with the distance from Earth ? $\endgroup$ Commented Nov 4, 2014 at 12:10
  • $\begingroup$ @b.gatessucks It's a bit harder. This is what I came up with (used consecutive integers for distances, to make it simple): ds[tr /* Append["distance" -> AssociationThread[Normal@ds[Keys], {1, 2, 3, 4, 5, 6, 7, 8}]] /* tr] $\endgroup$ Commented Nov 4, 2014 at 12:21
  • $\begingroup$ Many thanks, much better than de-/re-constructing the dataset from scratch. Hopefully it won't be too long until this kind of operation is built-in. $\endgroup$ Commented Nov 4, 2014 at 12:51
  • 1
    $\begingroup$ @b.gatessucks Indeed, it looks like this operation must be built-in. $\endgroup$ Commented Nov 4, 2014 at 13:18
7
$\begingroup$ 
dataset[MapThread[Append, {#, "d" -> d // Thread}] &]

or

dataset[Join[#, "d" -> d // Thread /* Map[Association], 2] &]

or

Module[{ds = Normal@dataset}, ds[[All, "d"]] = d; Dataset@ds]
Improve this answer
$\endgroup$
6
$\begingroup$

Perhaps this:

Module[{i = 1}, dataset[All, <| #, "d" -> d[[i++]] |> &]]

dataset screenshot

Improve this answer
$\endgroup$
3
$\begingroup$

Either

MapIndexed[Append[#1, "d" -> d[[#2[[1]]]]] &, dataset]

or

MapIndexed[Insert[#1, "d" -> d[[#2[[1]]]], -1] &, dataset]

works

Mathematica graphics

Improve this answer
$\endgroup$
3
  • $\begingroup$ Off-beat question in the image above, what is meant by 3 levels and 27 elements? $\endgroup$ Commented Jan 14, 2015 at 13:09
  • $\begingroup$ @user1585715 3 is the max depth (that would be the depth of the list elements in c), and 27 is the number of leaves in the datastructure. $\endgroup$ Commented Jan 14, 2015 at 15:35
  • $\begingroup$ Sjoerd - Thanks very much! That makes sense. $\endgroup$ Commented Jan 14, 2015 at 17:08

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.