Share a Public-Acess/Read-only Clone of your Graph Database

Problem

So, you want to show off the amazing tech you've been developing to your potential buyers using Graph Database technology. How do you do it safely?

Well, the safest way is not to and you can show off power-point presentations to your heart's content.

But the power-point presentation, no matter how slick, is nothing to what your graph database can do for your customers, and you know it, and I know it. Believe me, I've been involve in generating twenty power point presentations, and the customers just don't get it until you put the graph database in their hands and let them play with it (that's what I do, and that's how we got the sale, and then the follow-on contract).

Solution

So, back to sharing your graph database safely. How do you go about doing that?

We use grapheneDB.com as our DaaS over neo4j graph database, and they provide tools that makes sharing a copy of your database safely a cinch. Let's see how.

Let's say you have this graph database in your private access that you wish to share:

You're not going to share your database (you could, but we won't in this walkthrough), instead, we'll make a clone of that database. Here's how.  As you see, you have an [upgrade/clone] option for your database, select that.

Clone Database

By selecting [upgrade/clone] you have two options: clone from running database that involves downtime (that's why I'm doing this on a Sunday, see), or clone database from a backup. 

As you've been making daily backups (link) – right? – you can select the clone from a backup option. This is available at the professional editions. As I have a developer database here, I'll be cloning directly from this currently running database.

That's the top-half of the page, the second half is choosing what the cloned database will look like, which will be usually be the same configuration as your source database. The one field you need to supply is the new database name. Since this is a public access database, I'm prefixing the new name with 'PUBLIC': a little reminder-tag for me.

Once you have the configuration you desire for the new database select [Create database]. You get a nice little warning. Select [Continue].

Then you get the spinny-box as it creates the new database...

la-di-dah! Then you have your new database! YAY!

Configure New/Cloned Database to Read-Only Mode

But you're not done yet! We wish to provide public access to our database, but we don't want our adoring public to be creating THREE BILLION NODES on their own and drive you out of business.

Not that they would do that, ever, but still.

So, let's make this public-facing clone safe for access by making it read-only. Go to the Configure-tab on your new PUBLIC database. There you see that the read-only mode is "DISABLED":

Let's change that. Select [Edit configuration], and then select the "Enable"-link under Read only mode. Once you do that, select [Apply and restart].

You get a warning-dialog, so, again, select the red [Apply and restart] in this dialog:

You again get a 'restarting database'-spinny:

Then a confirmation at the top of your browser that your database has been restarted with the new configuration:

So that now, when you select the Overview-tab, you are reinforced with the confirmation that your database-clone is now in read-only mode:

Confirmed Read-Only

Let's confirm this by trying to create a node in the neo4j browser. Select [Launch] in your Overview-tab. Since this is the first time for this brand-new (cloned) database, you get the standard disclaimer-dialog:

Select [Launch browser], then, create some J-Random node named 'Foo' or whatevs. You, thankfully, get back the 'Could not create token'-message. You've verified your database is in read-only mode.

So, now you can share your database, as it's a clone of your original database (I leave data-masking for you to figure out), and it's in read-only mode.

Share Cloned Read-Only Database

But how do you share your database?

Go to the Connection-tab for your database and there it contains the connection information, including username and password as well as the connection URI or the REST URL for accessing your database.

BOOM-BAM, BABY! You now have provided a public-access read-only copy of your database! And, as it's provided by grapheneDB.com DaaS, either by AWS or Azure (your choice; grapheneDB allows either), it is demoable anywhere you have internet access.

You are in business! It's up to you how you wish to present the data (transformed, masked, or straight up) or to scale the data (a part of your database or all of it) as your public-access database.

Now: go to! Go to!

Fixing the Data Fix: Restoring from Backup

Data fix, part II, or: how to fix the data fix

So, yesterday, we looked at doing a hot data fix, and it ... worked, or it seemed to, but then, after adding the new day's data, instead of getting something like this:

this happened:

Oh, no! Now we have a data corruption, where stocks from the previous day and the current day are intermingled (the second graph-diagram) under one set of MKT_CAP, PRICE, and VOLUME headings instead of being separated into their respective days (as the first graph-diagram shows).

What to do? We have basically two options:

1. tease apart all the data relations for those two days, adding in the correct headings for each day and then partitioning the stocks to their respective days from the saved queries stored in the log files.

... you do save your log files don't you?

Basically, this is another hot fix. Possible? Yes. Doable? Yes. Risky?

... well: yes.

2. Or, we could make sure the data-load cypher queries are now correct offline (they are, I verified), blow away the current, corrupted database, restore from backup before the corrupt occurred and reload the corrected data from the stored cypher queries.

Basically, restore from backup.

Doable? Let's check. Do we have a backup? Well, thanks to grapheneDB.com, a DaaS ('Data as a Service'), we do, as they do a daily backup and save a week's worth of them for you under their professional plan. But before we start the restore, let's just be sure and save off our database as an export. You know: just in case we lose everything.

Exporting a database

How do we do that?

Simple. We go to the admin page which has the Export Database-option:

Then we select that option:

A database with six-month's worth of top 5s – under 2,000 nodes and under 2,000 relations – takes no time to prepare for export.  And then, we download that export:

Restoring from Backup

Now that we have a local export of our database, let's restore from out backup. 

The corruption occurred on 2016-01-05, so we use the 2016-01-04 backup:

... and accept the warning that we're about to wipe our database, because, yes, we want to eliminate the data corruption I introduced:

Then grapheneDB.com tells you, that, yes, your database is restored from backup:

Which is all well and good, but I'll do due-diligence, myself and confirm that the database is restored, firstly, and to the state before the corruption occurred.

Yup!

(Actually, I checked further into the database, and you should, too, to the level of your assurance, but this is not in the scope of this article.)

Data Correction from the point of the Restore

Okay, database restored to it's old state. WHEW! So we're back at ground zero, as it were, pre-2016. Now, let's reload our corrected 2016 data. To do that, I wrote a little utility that converts stock Top5s to Cypher-queries in JSON, called 'jsoner.sh.' I save off the top 5s to a daily record, the format of which is, e.g., this:

date>kind:Leaders|Losers

2015-12-31

Mkt_Cap:EPD,LBTYK,WMB,AAPL,GOOG|GOOGL,MSFT,AMZN

Price:NM-G,SWN,WPX,BCOM,LTRPB|AXON,NK,TXMD

Volume:GE,BAC,AAPL,KMI,QQQ,SPY,MSFT,SIRI,SUNE,FCX

2016-01-04

Mkt_Cap:EPD,BXLT,WFC-L,AMZN,GOOGL|GOOG,BABA,WFC

Price:SUNE,EPE,CHK,QUNR,AXON|RARE,NK,TEAM

Volume:SPY,BP,SAN,BHP,RIG,RDS.A,EEM,RIO

2016-01-05

Mkt_Cap:LLY,BABA,WMT,NTT,CHL|TOT,RDS.B,PTR,SAN

Price:SWHC,RGR,SHI,FLIR,GPRO|STRZB,CHK-D,BCOM,XLRN,EPE

Volume:ARRS,SPY,QQQ,XIV,SUNE,GPRO,SWHC,FCS,GDX

jsoner.sh scans the above and converts them to properly-formatted Cypher queries, ensuring the stocks fall into the the right heading for the appropriate day. So, let's run jsoner on 2016-01-04 and re-enter into the database the first corrected day's data:

Good. Let's verify those Top 5s are in and under the correct day:

And, you see at the bottom of the screen, I visually verified that the volume category is also now correct with the its appropriate date. It was incorrect before, showing 2016-01-05, and this was the root cause that lead to the data corruption before. Now we see that it is correct. YAY!

Okay, now let's reenter the corrected second day's data, that is the data for 2015-01-05 now:

And, we verify that the new data are in place, properly factored into the right days:

And so they are.

Database fully restored from a backup, ready, again, to receive the latest daily Top5s securities from the stock market.

We are back in business. YAY!