Data Ingestion

Data Ingestion is the process of moving streaming or batched data from pre-existing databases to different storage media and data sources, transforming to a desirable data format or loading data to a new system for fast and efficient processing by machine learning applications. When it comes to parallel data processing tasks, in which the problem should be decomposed in to small data sets and processed independently the ingestion technology can be used to perform Extract, Transform and Load (ETL) tasks and data integration. We specialise in building such pipelines to process both unstructured/structured data. The pipelines can be executed by many supported back-ends including, Apache Apex, Flink, Spark and Google Cloud Data Flow.