Offline Data Storage and Retrieval

[MosesFernandes]

If we need to repurpose the SAW node with archiver, what about existing SAW node data? 

Can we accomodate the SAW node data on offline storage?

If stored offline, how it can be retreived?

[SravanKoneti1]

Hi Moses,

Generally, SAW nodes in Cluster maintain the replica of data on each SAW node. If you are removing one SAW node, the data still be available in other nodes. But SAW cluster needs minimum 3 SAW nodes to work.

[MosesFernandes]

Hi Sravan,

Thanks for responding on my query. In the current architechture, we have 3 hybrids and 3 SAW nodes. So in the proposed architechture, we will repurpose SAW 1 and SAW 2 with ESA 1 and ESA 2, SAW 3 with Archiver. 

For this, we need to accomodate the SAW data. If it is stored offline how and where it can be done. Also how can we retrieve the same?

[SravanKoneti1]

Hi Moses,

Thanks for more details. I believe, Avro files can be read using API built with different programming languages like java,python,..etc.,

So, you can move 1 Avro file to offline storage and try reading it with API. If that is readable, you can build analytics on that data.

Reference:

Apache Avro - Wikipedia 

avro-cli-examples/README.md at master · miguno/avro-cli-examples · GitHub 

Reading and Writing Avro Files from the Command Line - Michael G. Noll 

[MosesFernandes]

Hi Sravan,

Thanks for your generous response. It was of great help.

There is a query about warehouse. What is the expiry period of Data Warehouse(SAW nodes)?