Challenge

Kalvad faced significant challenges with the high costs of managing large volumes of log data. Their existing system was becoming increasingly inadequate due to the costly infrastructure and intensive operational requirements.

Why Quickwit

Kalvad chose Quickwit over Elasticsearch for several compelling reasons:

• Support for Large Data Volumes: Quickwit's architecture, which separates compute and storage, makes it possible to scale to petabytes effortlessly.
• Extend retention without adding complexity: Quickwit stores all data on object storage, so increasing retention does not impact cluster management.
• Reduce infrastructure costs: All index data is on object storage, so you don’t need to duplicate data on costly SSDs. Plus, the compression ratio was 2x better than that of Elasticsearch.
• Reduce maintenance costs: Quickwit has a stateless architecture, and all index data is stored on object storage. This makes cluster operations seamless.
• Great integration with OSS Tools: Quickwit integrates well with Grafana, Vector, and Open Telemetry APIs.

Implementation and Benefits

Quickwit's cloud-native architecture proved to be a game-changer for Kalvad. They migrated to Quickwit on private cloud, public cloud, and air-gapped environments, using ZFS machines and the object storage service Garage.

The migration to Quickwit has delivered multiple operational benefits:

• Scaling up from 2TB to 100TB effortlessly: Kalvad is now managing clusters totalling 100TB without any operational issues.
• Reducing the total cost of ownership by 10x: Overall operational costs have been reduced dramatically, with a tenfold decrease in computing, storage, and engineering costs.
• Increasing retention from 90 days to 2 years: This extension is crucial for analyzing long-term trends and seasonal variations in our applications, providing Kalvad with richer insights and a more comprehensive understanding of our data.
• Enhanced search capabilities: Data analysts and developers have more data to play with, and thanks to easy-to-scale search, this facilitates more innovative use of data resources while keeping the system cost-effective.

Summary

Kalvad’s decision to adopt Quickwit for log management has been transformational. It has enabled the company to achieve remarkable improvements in efficiency, cost savings, and operational agility. Kalvad will be able to scale its log management system even further to support its clients' needs.