As companies build extensive data systems to support modern applications, data management costs have become a significant concern. Efforts to build AI - whether experimental or in production - adds further strain with the need for immediate access to textual, image, video, or other modalities of data, making data storage and extraction costlier.

However, beyond storage costs, labor costs for high-skilled AI engineers are also a large and growing line item. Creating complex data systems that integrate various applications takes significant time and effort, resulting in a tangled web of data transfers. This complexity makes setup, maintenance, and error resolution challenging and time-consuming.

In this post, we explore total cost of ownership when it comes to a database for AI applications. This includes hard and soft costs. We’ll also explore how purpose-built databases that consolidate storage costs and  operational overhead under a single hood, can save companies money on both categories, particularly labor. In fact, under extremes, a purpose-built database can reduce costs by 10x.

The Hard Costs

Hard costs form the foundation of database expense discussions. A typical data backend might use various tools:

Using some combination of these tools often generates four different types of hard costs:

1. Barrier Costs : Upfront or subscription fees for proprietary products.
2. Storage Costs : Charges per byte stored, higher for standard databases and lower for optimized data stores.
3. CPU Costs : Fees for CPU time used in data processing, sometimes for meeting format expectations that vary for the various data tools
4. Network Costs : Costs for data transfer between nodes, varying for ingress and egress rates.

Combined, these costs really add up. One way to think about it is that each byte ingested into the storage system carries multiple micro-costs. This is magnified when building AI pipelines  as it can sometimes lead to copies of the same data in order to saturate large scale model training, local copies for experimentation to avoid frequently accessing this web of data systems, or a collection of overlapping datasets as data scientists search for representative data for better AI outcome. This invariably leads to redundant and unnecessary expenses.

The Soft Costs

The significant expense, however, lies in the engineering labor required to set up and maintain these systems. Engineers need to configure databases, connect nodes, write transformation code, manage security, and handle ETL processes. Maintenance involves updating schemas, applying security patches, potentially mismatching API updates, and ensuring data consistency across systems. Training new engineers and managing permissions add further complexity and time costs.

Human errors add another layer of cost, potentially leading to data loss or compliance violations. When data scientists and ML engineers are forced to deal with such traditional data systems, dataset preparation instead of analysis can consume most of their time. If models don’t perform as expected, tracking down possible error in the data web potentially from incorrect conversions or mismatched data types is nearly impossible or can consume a lot of precious team hours. Any disruption in this do-it-yourself spaghetti solution can require filing tickets with multiple vendors navigating through their variety of support systems.

The Opportunity Cost

Even with improving off-the-shelf traditional and generative AI models, desired AI outcomes to enable AI use cases in production still require access to proprietary data. While traditional applications could be implemented with siloed data organizations, AI applications require access to data from multiple sources and modalities to achieve near-human outcomes. Complex data architectures delay their implementation sometimes as long as 6-12 months. Because it’s harder to query and visualize the underlying data used to train AI applications or surface the right responses when working with Gen AI applications, it can make it harder for data science and ML teams to achieve the desired level of accuracy and performance for their models. This difficulty in visualizing the data is somewhat intentional as a way to protect PII , but the downside is that it makes it harder to tweak AI models, which can often result in go-to-market delays. These delays amount to a real cost if another competitor is more agile at implementing AI, beating a company to market.

Enter a Purpose-Driven Database

We built ApertureDB, a purpose-built database for multimodal data, to address these challenges and reduce costs.

Hard Cost Savings

• Efficient tech spend : Instead of splitting storage of small units (text, often stored in a Postgres-like solution, sometimes multiple to suit the data types) from larger units (images or video, often stored in an S3-like solution), ApertureDB consolidates data types behind a unified API. Reduced reliance on multiple third-party applications lowers subscription costs since you only need one solution for all of your complex multimodal processing needs.
• Resource efficiency:  With the ability to preprocess or augment data on the fly, where often its downsized for ML use cases, this means less network costs, lower unit costs (less duplication), and simplified compliance.
• Resource Utilization : Finally, there is the savings on bare-metal efficiency. By processing data rapidly on the fly and in parallel via a purpose-built database, GPUs are kept busy and saturated, maximizing utility.  

Soft Cost Savings

• Simplified Management : One database means fewer connections, less setup, and fewer maintenance tasks. There is only one  database to learn, significantly less connections to set-up, and a lot fewer work-around scripts.
• Reduced Errors and Data Loss : Lower risk of duplication and errors, minimizing time spent on corrections. If you ask data teams what they waste their time on , it’s handling one-off requests of data getting incorrectly duplicated or destroyed. With a single place of storage, this is no longer a huge issue.
• Ease of Use : Streamlined training and permission management save time and reduce complexity. There is also only one  database to issue updates and patches to, one  database to train new employees around, one  database to grant access to, and one  database to monitor for any errors.
• Focused Effort : With the ability to index, query, and easily visualize data using its blended vector search and graph filtering, data science and ML teams can easily prepare and understand their datasets. Data teams can focus on optimizing AI accuracy and performance rather than managing a complex data system.

Opportunity Cost Elimination

A purpose-built database accelerates AI implementation by simplifying data integration and access. For instance, running multimodal AI analysis is dramatically easier with databases like ApertureDB because it consolidates data; otherwise, analysis needs to be conducted independently and then crudely collapsed into a single query script.

The simplified data access with ApertureDB enables faster deployment of AI applications, giving companies a competitive edge.

Real-World Evidence

We’ve thus far provided a very theoretical argument to why a purpose-built database saves on costs. However, we also have empirical evidence to support that claim! We’ve observed teams deploy ApertureDB, a multimodal database, into production. We’ve paid particular attention to their net savings. Let’s visit the migration experiences of two very  different companies.

A Fortune 100 big box retailer

First is a Fortune 500 company that featured a massive online storefront with a 5-10 person visual AI team at their disposal. The team’s job was to understand why consumers weren’t seeing proper recommendations when browsing products on the online storefront.

They discovered a few subproblems:

1. It was difficult to obtain permissions to read the correct product and asset management tables, as Fortune 500s are naturally protective about data.
2. Simple tasks would often take hours (and sometimes days) because data was located in various places.
3. Data science teams would download data once and continuing using it without refreshing to avoid the delays
4. Embeddings were extracted by various teammates but not consolidated resulting in varying recommendations and making it very difficult to debug if the recommendations were not as expected.

Apart from the hard cost of local dataset copies and wasted processing efforts in extracting embeddings repeatedly without a way to collaborate, these resulted in soft costs to the data team productivity by creating day-to-day headaches and a huge opportunity cost in lost revenue (hard to quantify revenue gains from models that were delayed or not deployed) due to a disconnected system .

Moving to ApertureDB did require setting up ETL from various systems but needed to be automated once. However, the biggest benefits of the move to ApertureDB were solutions to the prior problems:

1. Permissions were simplified to just managing RBAC for one  system
2. Previously tedious tasks of collecting data from multiple tools translated to writing simple queries to one system, reducing the time needed to under an hour
3. Data teams no longer needed to create local copies since they could get pre-processed, regularly refreshed data on-the-fly in the format expected
4. Embeddings (even from different models for comparisons) could be indexed in one  location, together with the corresponding images, annotations, and metadata.

Now with fewer hiccups, the team was able to turn their attention to improving their models’ efficacy. The final result? Customers received better recommendations leading to better revenue potential.

Badger Technologies

Badger Technologies (Badger) is a retail robotics company that enables retailers to monitor product placements via a motorized, tower-like robot. Badger robots pass through aisles and ingests data from an array of cameras and use their onboard computer vision models to convert those images to an array of embeddings. They then need high throughput vector classification to identify the products and match them to their locations and pricing, generating alerts in case of mismatches. On the model preparation side, Badger team also needed a way to store images for store products, label them and train the models to deploy on their robots.

This created a multimodal data problem. Specifically:

1. On the store side, Badger needed thousands of vector classifications performed per second from a large number of stores, reliably
2. On the model preparation side, Badger needed a unified location to store their image data, speed up labeling tasks, and simplify dataset preparation for model training.

As described in this case study , their previous vector database solutions could not keep up with the scale causing a slowdown in their growth, leading to revenue loss. With a complex manual solution for data labeling where data was scattered in various file storage locations, data labeling was often delayed by months causing a lot of wasted people-time.  

After implementing ApertureDB, Badger was able to improve its store performance by more than 2.5x. Today, Badger is able to handle a whopping 10,000 queries per second without any instability. This is possible because of ApertureDB’s scalable multimodal vector database. With ApertureDB’s ML integrations with labeling and training frameworks, Badger team got the opportunity to unify their image and label storage into one database, which meant significant simplification of their data collection and model training pipelines.

Dataset Preparation Speedup for Fast-Growing Startups

We saw similar video labeling, training and inference examples with a frictionless checkout company, Zippin, and a Biotech stealth company. They were both dealing with large collections of videos in storage buckets, either labeled or requiring an integration with a labeling pipeline, needing to extract interesting frames, and keep costs down all while growing rapidly. Use of ApertureDB has made it possible for them to get their data requirements met in half the time and half the amount of resources, both extremely crucial benefits for startups.  

ApertureDB RAG-based Documentation Chatbot

ApertureDB team wanted to experiment with our vector search (and knowledge graph capabilities in the coming days) to build a query-response chatbot to help our users. With our LangChain RAG implementation, it took us less than a week to crawl our website and documentation pages, index the necessary embeddings,  and add a plug-in our documentation page  (alpha version right now). There are of course more improvements that can be done by experimenting with various LLMs, token sizes, and knowledge graph constructions, but data management did not slow us down. We are happy with the improved user experience and are now able to improve our documentation if user queries indicate missing pages (a new blog coming soon).

Why Go Purpose-Built?

Purpose-built databases offer significant savings in both hard and soft costs, as well as eliminate the opportunity costs associated with delayed AI implementation. They provide:

• Lower storage costs
• Reduced reliance on multiple products
• Simplified infrastructure management
• Minimized error-related disruptions
• Streamlined training and permission management
• Faster AI deployment

To learn more about how a purpose-built database for multimodal analytics can save costs, book a demo here. We have built an industry-leading database for multimodal AI to future-proof data pipelines as multimodal AI methods evolve. Stay informed about our journey by subscribing to our blog.

I want to acknowledge the insights and valuable edits from Mathew Pregasen and JJ Nguyen.

‍