0

I'm using Olivere v7.

I'm trying to Merge _id & _source (from the ElasticSearch call, which returns a JSON response) into a re-ordered struct, and deliver results in that format.

This is what my actual code is:

elasticsearch, err := //actual request to ES using olivere fmt.Println(elasticsearch.Hits.Hits)

and it successfully makes a call and prints this:

{ "_score": 11.019884, "_index": "test", "_type": "_doc", "_id": "20", "_seq_no": null, "_primary_term": null, "_source": { "name": "Michael J.", "age": "22" } }, { "_score": 11.019884, "_index": "test", "_type": "_doc", "_id": "21", "_seq_no": null, "_primary_term": null, "_source": { "name": "Michael Jae.", "age": "18" } }, { "_score": 11.019884, "_index": "test", "_type": "_doc", "_id": "52", "_seq_no": null, "_primary_term": null, "_source": { "name": "Michael Jay.", "age": "69" } }, { "_score": 11.019884, "_index": "test", "_type": "_doc", "_id": "33", "_seq_no": null, "_primary_term": null, "_source": { "name": "Michael Jo.", "age": "25" } }

Instead, i would like to only print "_id", "name", "age"

So the desired output based on the same query above would be:

{ "id": "20", "name": "Michael J.", "age": "22" }, { "id": "21", "name": "Michael Jae.", "age": "18" }, { "id": "52", "name": "Michael Jay.", "age": "69" }, { "id": "33", "name": "Michael Jo.", "age": "25" }

I've written this code:

elasticsearch, err := //actual request to ES using olivere type StructuredResponse struct { ID string `json:"id"` Email string `json:"email"` Name string `json:"name"` } var SR []StructuredResponse for _, hit := range elasticsearch.Hits.Hits { source, err := json.Marshal(hit.Source) if err != nil { panic(err) } var SR2 StructuredResponse err = json.Unmarshal(source, &SR2) if err != nil { panic(err) } SR = append(SR, SR2) } fmt.Println(SR)

But i'm lost, im not sure what the next step would be or what to do from here.

Update:

elasticsearch.Hits.Hits points to this struct within Olivere's package:

// SearchHit is a single hit. type SearchHit struct { Score *float64 `json:"_score,omitempty"` // computed score Index string `json:"_index,omitempty"` // index name Type string `json:"_type,omitempty"` // type meta field Id string `json:"_id,omitempty"` // external or internal Uid string `json:"_uid,omitempty"` // uid meta field (see MapperService.java for all meta fields) Routing string `json:"_routing,omitempty"` // routing meta field Parent string `json:"_parent,omitempty"` // parent meta field Version *int64 `json:"_version,omitempty"` // version number, when Version is set to true in SearchService SeqNo *int64 `json:"_seq_no"` PrimaryTerm *int64 `json:"_primary_term"` Sort []interface{} `json:"sort,omitempty"` // sort information Highlight SearchHitHighlight `json:"highlight,omitempty"` // highlighter information Source json.RawMessage `json:"_source,omitempty"` // stored document source Fields map[string]interface{} `json:"fields,omitempty"` // returned (stored) fields Explanation *SearchExplanation `json:"_explanation,omitempty"` // explains how the score was computed MatchedQueries []string `json:"matched_queries,omitempty"` // matched queries InnerHits map[string]*SearchHitInnerHits `json:"inner_hits,omitempty"` // inner hits with ES >= 1.5.0 Nested *NestedHit `json:"_nested,omitempty"` // for nested inner hits Shard string `json:"_shard,omitempty"` // used e.g. in Search Explain Node string `json:"_node,omitempty"` // used e.g. in Search Explain // HighlightFields // SortValues // MatchedFilters }
Improve this question
3
  • Why the current solution doesn't satisfy you ? What output do you have ? Commented Jan 12, 2021 at 8:10
  • @aureliar the current solution does not actually add the ID anywhere. It prints the email and name fields, but leaves the ID field empty. Commented Jan 12, 2021 at 8:12
  • Isn't the name of ID in the resulting elasticsearch output _id? Then it should be json:"_id". But that will not work, as _id is one level up to the rest of the data. Also, why are you marshalling it back to json? Isn't the ID already available on the hit directly? Commented Jan 12, 2021 at 9:17

2 Answers 2

Reset to default
2

I think it should work like this:

var srs []StructuredResponse for _, hit := range elasticsearch.Hits.Hits { var sr StructuredResponse if err := json.Unmarshal(hit.Source, &sr); err != nil { panic(err) } sr.ID = hit.Id srs = append(srs, sr) }

The ID is already available in the hit. See https://github.com/olivere/elastic/blob/82129300722c28a88207fd5dfed442668d9e264d/search.go#L749

Improve this answer
Sign up to request clarification or add additional context in comments.

5 Comments

Correct! This worked like a charm! If it's not too much to ask, as im still learning - could you please explain what i was doing wrong and what you did?
1) you do not need to marshal the hit.Source. It is already marshalled as it is of type json.RawMessage (see SearchHit struct in your question). 2) there is no id in the source. It only contains name and age. The id is in the hit and that is already unmarshalled by olivere's package.
What I did: Unmarshal the hit.Source to a StructuredResponse variable (no ID in source) and after I take the Id from the hit and additionally write it to the struct variable.
Does that make sense?
Ah, yes that makes sense! I need to read more into slices, appending, & how marshal/unmarshal works, and when to use it. I missed the RawMessage part, but the appending part is a tad bit confusing. I really appreciate you! @Tehsphinx
0

Assuming the fields of elasticsearch.Hits.Hits are exported, you shouldn't need to Marshal the output.

for _, hit := range elasticsearch.Hits.Hits { SR = append(SR, StructuredResponse{ID: hit.ID, Email: hit.Email, Name: Hit.Name}) }

This will give you an slice with a complete set of items (SR). At that point you should be able to Marshal that slice into JSON, if that's what you wanted to do.

Improve this answer

4 Comments

Getting SR2 is not a type. Also, they are not exported.
Sorry I should have used StructuredResponse instead of SR2, updated the example.
! That's starting to make sense! Thanks! My second concern is, would calling hit.email result in an error? since it is nested inside the struct. I've updated my question to also include the struct within Olivere's package
hit.ID is actually hit.Id and hit.Email and hit.Name do not exist. You have to unmarshal hit.Source to get them. See my answer.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.