Streaming Integrator
WSO2 Streaming Integrator includes a stream flow designer and a stream processing engine with strong monitoring and analytics functions. The streaming runtime capabilities enable users to treat all data sources as streams of data, apply stream processing operations, and publish to one or more destinations. It also comes with advanced error handling and correction.
WSO2 Streaming Integrator allows you to implement streaming ETL, change data capture (CDC), and process large files and real-time APIs. You can connect and realize event-driven architectures with distributed streaming systems such as Kafka, Amazon SQS, and more.
Streaming ETL with CDC
- Treat all data sources as streams of data
- Debezium-based CDC with Oracle, MSSQL, MySQL, and more
- Wizard-based ETL task designer
- Error handling including error capture and replay
- Integrate with files, Kafka, HTTP endpoints, RDBMS, AWS S3, and more
- Hierarchical monitoring dashboards for file, CDC, and DB related stats and figures
Large File Processing with MFT
- Process and transfer large files
- Performance: Read a 100GB file with 10 million lines in just 20 mins
- Supports many data formats: CSV, XML, JSON, Binary, Avro, Key-Value pairs, Strings using Regex, and Protobuf
- Built-in real-time file and directory event listeners
- MFT capabilities to build complex enterprise scenarios
Event Stream Integration
- Built-in connectors to integrate various event sources and event sinks
- At least once delivery guarantee with exactly-once processing
- Supports transforming, enriching, cleansing, aggregating, and correlating event streams with insights
Real-Time APIs
- Expose any data source as a real-time streaming API with WebSockets and gRPC streaming
- Full lifecycle API management for streaming APIs with WSO2 API Manager
Visual Tooling
- Capabilities to design, develop, test, and deploy streaming integration flows
- Drag-and-drop streaming flow designer
- Streaming ETL task wizard
- Source editor with IntelliSense
- Support for event and feed simulation
- Configurable connectors to integrate various event sources and event sinks
Monitoring
- Monitoring metrics for CDC, File, and DB Interactions
- Capabilities to drill down into event analytics information from the dashboard at the server, application, event source, and event sink levels
- Real-time insights into performance, usage, and errors
- Visualization with built-in charts
An overview of
WSO2 Streaming Integrator
