The Apache Software Foundation, the all-volunteer developers, stewards, and incubators of more than 350 open-source projects and initiatives, announced today Apache InLong as a Top-Level Project. The project name carries significant meaning. InLong is a divine beast in Chinese mythology who guides the river into the sea, and it is regarded as a metaphor for the InLong system for reporting data streams.
One-stop integration framework for massive data
InLong has been widely adopted across various industries, including advertising, payment, social, gaming, AI, and others. The Apache InLong project was originally called TubeMQ, focusing on high-performance, low-cost message queuing services.
To further release the surrounding ecological capabilities of TubeMQ, the community upgraded the project to InLong. It integrates the entire processes of collecting, aggregating, storing, and sorting data processing. It is simple, flexible, stable, and reliable. Features include:
- Ease of use: Apache InLong is a SaaS-based service platform. Users can easily and quickly report, transfer, and distribute data by publishing and subscribing to data based on topics.
- Stability & Reliability: Apache InLong is derived from the online production environment. It delivers high-performance processing capabilities for 100 trillion-level data streams and highly reliable services for 100 billion-level data streams.
- Comprehensive features: Apache InLong supports various data access methods and can be integrated with different types of Message Queue (MQ). It also provides real-time data extract, transform, and load (ETL) and sorting capabilities based on rules. Apache InLong also allows users to plug features to extend system capabilities.
- Service integration: Apache InLong provides unified system monitoring and alert services. It provides fine-grained metrics to facilitate data visualization. Users can view the running status of queues and topic-based data statistics in a unified data metric platform. Users can also configure the alert service based on their business requirements so that users can be alerted when errors occur.
- Scalability: Apache InLong adopts a pluggable architecture that allows you to plug modules into the system based on specific protocols. Users can replace components and add features based on their business requirements.
In addition, the project recently released v1.2.0-incubating, the twelfth release of Apache InLong. Improvements include:
- Refactoring the InLong Sort module to support Transform, such as String Split, Data Filter, Regular Join, etc.
- Adding 8+ data nodes including MongoDB, SqlServer, Greenplum, Oracle DB, etc.
- Optimizing the initialization process of data nodes such as Iceberg, ClickHouse, and Hive.
- Supporting MySQL to collect Binlog from the specified offset.
- Supporting the plug-in expansion of different types of message queues.
- Supporting the management of multiple clusters and data nodes.