stop managing data pipelines.
start shipping great products.
the complexity of bringing together streaming, datalake, and schema management is holding your team back.
wings is your open-source streaming platform. it uses zero-copy iceberg storage to eliminate data duplication, giving you the power to own your entire data stack and simplify your architecture.
query engine
open table catalog
in-memory processing
storage format
core features
zero copy iceberg topics
wings stores topics into an apache iceberg catalog. stream data with wings, or query using any apache iceberg reader.
fine grained partitioning
partition data based on your analytical use case, not your operational needs. partition data based on your schema on any key you choose, allowing for efficient querying and analysis.
native schema management
define the arrow schema for your topics, wings takes care of enforcing the schema on ingestion. evolve schemas as your business needs change without the cost of migrating existing data.
open source
wings is open source with an apache 2.0 license. you can self-host it on your own infrastructure, so your data and metadata never leaves your network.
architecture
separate data and metadata
we separate data from metadata, and storage from compute. we leverage cloud-native services for storage of data and metadata to simplify operations.
arrow flight sql
wings uses arrow flight for efficient data transfer. all data and metadata is a sql query away thanks to arrow flight sql.
multi tenant
wings is designed to support multiple tenants on a single cluster. projects can use different object storage backends. topics can write live data to express buckets, and use cheaper standard storage for compacted data.
distributed cache
we leverage liquidcache to implement distributed pushdown caching and reduce the number of object store requests, improving latency and reducing cost.
extensible auth and secret management
wings interacts with external services to provide authentication, authorization, and secret management. object store configuration is safely stored in external vaults.
rust core, typescript sdk
wings is implemented in rust, leveraging the arrow and datafusion ecosystem. we provide a typescript sdk for best-in-class developer experience.