2

I'm going to make sure this stays specific and not flagged as too broad.

I'm putting together an approach to store data in Heroku. This data is relatively simple in nature - it's key/value pairs, where the key is a unique ID, and the value is JSON, somewhere around 50k lines long. The primary requirement is fast retrieval of the value (JSON), by the unique key.

I've heard of three data services that Heroku offers: Postgres, Redis, and Kafka. My specific question is: solely based on the criteria above (the nature of the data being key/value pairs with JSON as the value, and the quickest retrievel), is one of these data services a better choice than the others?

A few side notes - CDN is also being evaluated, but for the sake of keeping this question specific and not being closed, looking for input on Heroku here :) Also, the volume of data will be several million rows, so let's also exclude the possibility of storing this data in Salesforce as well

Improve this question
2
  • How about big objects? Commented Aug 9, 2018 at 17:01
  • @RedDevil this is a really interesting idea and I hadn't thought of it before - I just did some brief research. A possible issue is that we can expect the JSON to be upwards of 1M characters. Even if we broke that up across multiple long text fields, the limit on an object appears to be 1638400. Really appreciate this idea though, and we're going to run with it in a parallel analysis Commented Aug 9, 2018 at 17:37

1 Answer 1

Reset to default
1

A few of us discussed this on the Heroku Support team, including our lead Data Support Engineer. We all came up with a handful of ways to do this, but we limited it down to Postgres or Redis, with Postgres coming out on top. As long as you're not trying to query on that big piece of JSON -- just looking it up by the key -- Postgres should work just fine for you.

Improve this answer
1
  • Thank you very much for taking the time to respond! Commented Sep 20, 2018 at 21:03

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.