Tuesday, January 5, 2016

Data-fixin' down-low on the Production Data

You have this:

Note that the date is wrong: these Top5s stocks should be for 2016-01-04, yesterday, not today! AND we need to fix this by COB today, so our entire database does not become corrupted.

What to do?

So, we have three options: 

1. revert the database from the daily save. Safe. 
2. Remove that date and all its child nodes, ugh, and not so safe (children are children of every day). 
3. Or, reset that date.

Let's do 3. for a change: the hot data fix.

match (d:Day { month: 1})
set d.date = "2016-01-04"
set d.day = 4
return d

(n.b.: the above query works ONLY because there is only one trading day this new month (and year))

Fixed! Hot data fixes with Graph databases can be painless and easy, and you have visual proof your fix worked, too!

Post Script, 2016-01-05

Uh, oh! After adding next day we have this!

Data Corruption! Two days are sharing overlapping market caps, prices, and volumes! Root cause? I corrected the Day-node only, but the three child nodes (PRICE, VOLUME, MKT_CAP) have a date attribute that I did not correct in the data correction above! Oh, no! Data corruption!

What do we do? Besides run around in circles in a panic for a while?

Well, we restore from backup before this mess started and then re-add each days data using the correctly supplied dates, ... as we should have done in the first case instead of all this super-cool 'hot data fix'-approach. That's what we should have done.

Done. Corrected. VOLUME, PRICE, and MKT_CAP child nodes are now properly separated, and the stocks now fall under their respective day's categories.

Restoring from backup database with corrected data blog post forthcoming.

No comments:

Post a Comment