Skip to main content
Software Engineering

You are not logged in. Your edit will be placed in a queue until it is peer reviewed.

We welcome edits that make the post easier to understand and more valuable for readers. Because community members review edits, please try to make the post substantially better than how you found it, for example, by fixing grammar or adding additional resources and hyperlinks.

Required fields*

5
  • Better, it's a data-centric solution at least, but it's still overkill Commented Sep 5, 2015 at 5:30
  • I'm using an sqlite for time being as it's easy to start with. Could easily switch to MySQL (or PostgreSQL). If using a no-SQL backend would net anything to make this work even better, I'm all ears. Commented Sep 5, 2015 at 7:31
  • Well, my thinking was mainly that you’re converting it to rows in a database anyway. If you don’t need to run it from multiple processes concurrently, I don’t think you want to switch to something more heavyweight than SQLite. Commented Sep 5, 2015 at 7:35
  • No need for concurrent processing. But I need to store the data about resources somewhere. An SQL db seemed like a good choice, However, there is nothing preventing me from loading data into any datatype for processing deltas. All I want at the end of the each extract run is to figure out what's new, what remained same, and what has disappeared. I can figure out how to update records as necessary from this info. Commented Sep 5, 2015 at 7:43
  • After you’ve parsed the data and put it into the database, it’s probably simpler to write a query than implement an algorithm. That said, if you do want to code it, the algorithm you want is set difference and there’s an implementation in the C++ STL that you can use to do it in a single line once you’ve put both sets of data into the container of your choice, probably a Vector. Commented Sep 5, 2015 at 7:53