0

is it possible to change the resultset structure when retrieving data from elastic search?

the problem ist, the timeseries data are sometimes from 3000-8000 records which are a json array with json objects in it ... parsing it in this case not really efficient or necessary so i thought - could a resultset be transformed to just lets say a simple json object with an array of time and array of values? nothing more? i could do this in java or php but since we want to have an efficent way of dealing with large datasets we are currently evaluating our options.

Improve this question

2 Answers 2

Reset to default
1

You can control what elasticsearch returns using source filtering:

https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-source-filtering.html

It can let you pick which part of the indexed document it will return, which, depending on your index structure could be an array of times and values, or at least, very easily mapped to it using the language of your choice.

Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

i'm not sure if i can create somethin like this {"time":["2017-12-12 11:11:00","2017-12-12 11:11:01"],"value":["0.245","0.314"]} which would be s simple json without metadata, _source etc.
Yeah I don't think you can get rid of all the metadata.
0

Another possibility is to use scripting to control the results. If you map the result in this way, you should be able to get the hits object to be a JSON array of key: value.

Improve this answer

4 Comments

i read a lot around this topic but i really didn't find a solution for this issue. is it really that unusual to fetch 5000 records from ES and process it within a reasonable amout of time?
Well, it's definitely not the primary purpose behind elasticsearch, which is a full text search engine..
good point! what would you suggest in such a case? mysql, postgre etc. are not really fast enough or suitable for such a task ...
thanks for your comment, this brought me to some alternatives like these blog.netsil.com/… ... guess ES is an option but maybe we reconsider the choice in this special area of application. thanks again!

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.