Streaming Integrator


WSO2 Streaming Integrator includes a stream flow designer and a stream processing engine with strong monitoring and analytics functions. The streaming runtime capabilities enable users to treat all data sources as streams of data, apply stream processing operations, and publish to one or more destinations. It also comes with advanced error handling and correction.

WSO2 Streaming Integrator allows you to implement streaming ETL, change data capture (CDC), and process large files and real-time APIs. You can connect and realize event-driven architectures with distributed streaming systems such as Kafka, Amazon SQS, and more.

Download  
Streaming Integrator
Streaming ETL with CDC

Streaming ETL with CDC

  • Treat all data sources as streams of data
  • Debezium-based CDC with Oracle, MSSQL, MySQL, and more
  • Wizard-based ETL task designer
  • Error handling including error capture and replay
  • Integrate with files, Kafka, HTTP endpoints, RDBMS, AWS S3, and more
  • Hierarchical monitoring dashboards for file, CDC, and DB related stats and figures

Large File Processing with MFT

  • Process and transfer large files
    • Performance: Read a 100GB file with 10 million lines in just 20 mins
    • Supports many data formats: CSV, XML, JSON, Binary, Avro, Key-Value pairs, Strings using Regex, and Protobuf
  • Built-in real-time file and directory event listeners
  • MFT capabilities to build complex enterprise scenarios
Large File Processing with MFT
Event Stream Integration

Event Stream Integration

  • Built-in connectors to integrate various event sources and event sinks
  • At least once delivery guarantee with exactly-once processing
  • Supports transforming, enriching, cleansing, aggregating, and correlating event streams with insights

Real-Time APIs

  • Expose any data source as a real-time streaming API with WebSockets and gRPC streaming
  • Full lifecycle API management for streaming APIs with WSO2 API Manager
Real-Time APIs
Visual Tooling

Visual Tooling

  • Capabilities to design, develop, test, and deploy streaming integration flows
  • Drag-and-drop streaming flow designer
  • Streaming ETL task wizard
  • Source editor with IntelliSense
  • Support for event and feed simulation
  • Configurable connectors to integrate various event sources and event sinks

Monitoring

  • Monitoring metrics for CDC, File, and DB Interactions
  • Capabilities to drill down into event analytics information from the dashboard at the server, application, event source, and event sink levels
  • Real-time insights into performance, usage, and errors
  • Visualization with built-in charts
Monitoring
An overview of

WSO2 Streaming Integrator


Success Stories