Cygnus - Data Persistence using Apache Flume
Cygnus is a connector in charge of persisting context data sources into other third-party databases and storage systems, creating a historical view of the context. Internally, Cygnus is based on Apache Flume, Flume is a data flow system based on the concepts of flow-based programming. It supports powerful and scalable directed graphs of data routing, transformation, and system mediation logic. It was built to automate the flow of data between systems. While the term 'dataflow' can be used in a variety of contexts, we use it here to mean the automated and managed flow of information between systems.
Academy Courses
Lesson 1. Cygnus Introduction
By following this course, you will learn about Cygnus, our connector able to create historics from Orion context data. FAQ, architecture, basic and advanced configuration, and detailed sink catalogue.
Lesson 2. Persisting to HDFS using Cygnus
This video presentation explains how to use Cygnus to persist data for Big Data Analytics.
Step-by-Step Tutorials
Data Persistence using Cygnus is described in the following step-by-step tutorial:
- FIWARE 301: Persisting Context Data using Apache Flume (MongoDB, MySQL, PostgreSQL)