Spiral

We are hiring DevRel engineers in NYC, SF, or London! If you've wanted to pull your hair out while... - waiting on data loaders - writing video or sensor data pipelines - or watching expensive GPUs sit idle on I/O ...and you like helping others solve it... we want to meet you! Link below 👇

A warm welcome to Spiral as a Supporter of #EuroRust26! Spiral builds modern data analytics infrastructure, including their open-source Vortex file format (a Linux Foundation project) and SpiralDB data warehouse. They develop with Rust for its performance, correctness, and portability across platforms and languages– essentially, Rust's goals align strongly with Spiral’s technical priorities. See you at EuroRust, Spiral! 🦀✨ #RustLang #EuroRust26

Working on world models? If you enjoyed Nicholas Gates's deep dive on training models with video, then come here him unpack it on Tuesday evening (June 16th) in SF! We're hosting an event with krea.ai and Tigris Data featuring lightning talks from frontier AI labs and infra providers including Chroma, Metamorphic, and more. Link below 👇

📅 Upcoming event— we’re teaming up with krea.ai and Tigris Data hosting a tech meetup in SF next week! We're bringing together engineers, researchers, and builders to network and dig into one of the most pressing challenges in AI infrastructure: internet-scale distributed data systems for AI research. Whether you're working on data pipelines, storage architecture, or just curious about how we move and manage data at the scale AI demands — come join us. https://luma.com/2z8lp24e

We are heading to Denver for CVPR this week! We're co-hosting an AI for Science happy hour with Metamorphic tomorrow (Thursday, June 4th) — bites, drinks, and zero slides. If you work on multimodal data at scale or have interest in scientific AI, come say hi! Link in comments

We are heading to Denver for CVPR this week! We're co-hosting an AI for Science happy hour with Metamorphic tomorrow (Thursday, June 4th) — bites, drinks, and zero slides. If you work on multimodal data at scale or have interest in scientific AI, come say hi! Link in comments

Cloudflare folks in London & New York – we’re actively hiring. We are building the foundational data infrastructure for physical AI, tackling bottleneck after bottleneck for the world's best research teams. If you're excited about AI's impact on the physical world, from robotics to biology, we are growing our team of expert systems & ML infra engineers. If you were impacted by the layoffs at Cloudflare and think you might be a fit, please DM us.

You can have your cake and decompress it too 🍰 On TPC-H at scale factor 10, Vortex files are 38% smaller and decompress 10–25x faster than Parquet with ZSTD, without using any general-purpose compression. Nicholas Gates wrote about how Vortex uses BtrBlocks-style codec selection to beat Parquet+ZSTD on both size and speed. Link in comments.

Today's data infrastructure was built for humans. AI systems need something entirely different, and we think that mismatch is the invisible bottleneck slowing the AI revolution. That's why we're leading Spiral's Series A. Modern AI systems can process information at unprecedented speeds, but legacy data infrastructure creates a fundamental bottleneck, forcing advanced systems to wait while outdated formats and architectures struggle to keep pace. Will Manning, Robert Kruszewski, and Nicholas Gates at Spiral built Vortex to remove that constraint. Their breakthrough technology delivers 200x faster random-access reads with GPU-native decompression that streams data directly from S3 into GPU memory, enabling AI systems to access data at the speed they require. Vortex has now been contributed to the Linux Foundation with widespread industry adoption. And Spiral goes beyond fixing speed. They're building a complete data platform designed specifically for AI applications at massive scale. This is what happens when you build data infrastructure for the future. Explore why we invested by Quentin Clark and Max Rimpel in the comments.

Current data systems were built for humans reading dashboards. But AI doesn't want summaries—it wants everything. An H100 can consume 4 million images per second. Most sit idle 70% of the time, waiting for data. We built Spiral to fix this: ⚡ 10-20x faster than existing solutions 🎯 Direct S3 to GPU data loading ✨ Unified system for all data types 🔒 Enterprise-grade security built in Already backed by Microsoft, Snowflake, and Palantir. $22M raised from Amplify Partners and General Catalyst. Full announcement: https://lnkd.in/ePPwkXXM #DataInfrastructure #AI #MachineLearning #DataEngineering