mFlow enables fast implementation, efficient control, management and monitoring of the ETL processes execution
ETL process involves collecting a complete, consistent set of data from multiple source systems. It is necessary to ensure proper ordering and control of all the interdependencies between individual transformations in order to load the data warehouse successfully.
Usual pitfalls in the design and management of ETL processes are:
- Process development time takes up significant developer resources
- Non-transparency of processes, their interdependence and performance results
- Lack of adaptive parallelism
- No support for process origin and impact reports (Lineage / Impact Analysis)
- Problematic migration between environments and the inability to gradually replace ETL tools
Increased productivity
- Creating an entire process tree of several hundred processes in a couple of hours
- Adaptive changes are measured in minutes
- Process creation: Individual (GUI), Bulk (SQL)
- There is no deploy
- Testing of process variants: selection of series / parallels is reduced to changing one attribute (update)
Flexibility
- Modular development and testing
- Easy implementation of changes
- Independence of the ETL tool
- Flexible API addition – (combining ETL tools, gradual migration enabled)
Performance
- Intelligent adaptive parallelism
- The maximum number of simultaneous processes can be defined
- Queuing
- Maximum utilization of resources
- It is possible to define the order of entering the Queue
Supervision and maintenance
- Transparency (Lineage & Impact analysis)
- Load process monitoring via web browser
- Automation – (Re) start load with one command
- Unified logging (processed lines, errors)
- Interactive ad hoc reports
