The AI Datastore
The AI Datastore for Schemas, BLOBs, and Predictions. Use with your apps or integrate built-in Human Supervision, Data Workflow, and UI Catalog to get the most value out of your AI Data.
New DLv2 License
Learn more about Diffgram License v2 (DLv2) and the Contributor License (CL).
Use Cases
All AI companies have AI Data: Schemas, Blobs and Predictions. Is your volume of AI data increasing exponentially? Diffgram’s AI Datastore provides a single place to put this data. Use with your apps or integrate our built-in Human Supervision, Data Workflow, and UI Catalog to get the most value out of your AI Data.
Use with your AI Apps - One place for Compliant PII AI data.
Human Supervision (Data Labeling) - Label all media types and scale your annotation.
AI Data Application Workflow - Move data between your AI Apps and control your AI through a friendly UI/UX exp.
UI Catalog - Visually Explore your AI Datastore.
Use with your apps
- Single place to put, organize, and access multiple Schemas, Predictions and Blobs.
- Business objects to represent AI data in a meaningful way to your business. For example combining multiple media types, such as the front and back of a drivers license. Insert records and define records from multiple sources. Query unified representations.
- High Performance inserts and queries across complex heterogeneous data. Engineer friendly primitives. Economical lifecycle and long term storage options.
- Sync multiple existing databases, like MongoDB.
- Many operations and functions including multiple datasets, tagging, moving files, copying files, and more
Do you think about these questions?
- Do you have an ever-growing volume of Blobs, Schemas, and Predictions?
- Do you expect that your volume of AI predictions, tools, users or data is going to increase?
- Are you concerned about implications of new tools to create/build AI?
- Are you using multiple Blob stores, schemas, and prediction generating systems?
- Do you know how much AI data you have? Or how much is being effectively used? Do you know what your missing? How Siloed is your AI data?
Human Supervision (Data Labeling)
Label all media types and scale your annotation.
- The most advanced labeling toolset in the world.
- Get complete visibility and control of every part of your labeling operations.
- Annotate nearly Anything. Images, Video, Text, Geospatial, Audio, 3D, Compound/Multi-Modal, Conversations and more. Auto Annotation, including with SAM (coming soon). For Supervised Machine Learning, GenAI, and Every Media Type.
- Deploy to your Frontline staff exactly what human supervision you want. The only solution for deploying Training Data into your business processes at a large scale. with complete Tasks system, reporting, and robust customization options.
- Built for the Enterprise. Not just Data Science. Flexible, Customizable, & API/SDK Driven. Fully integrated on top of Diffgram AI Datastore and Diffgram AI Workflow. Display ML outputs and annotated data in your application, collect human feedback and supervision, and integrate with your ML smarts.
AI Data Application Workflow
Before Workflow teams would often have to create many one-off scripts and complicated Rube-Goldberg type machines to make even the most basic AI Apps work. These ungainly contraptions often have huge security and compliance gaps, are completely unmaintainable, and took ages to spin up in the first place. Workflow changes all that, moving a previously opaque and mystical process into a friendly UI/UX experience, making AI Data movement as easy as Dropbox, and all with an eye to long term maintenance and economics.
Move AI Data between AI apps
Move AI Data between AI apps in a secure, compliant, and safe way without manually transforming, writing one-off scripts, or spinning up annoying functions.
- Use new AI tools without fear of data loss, duplications, and breaches.
- Easily visualize your AI Data flow through a friendly UI/UX experience.
- Unleash your experts to create new projects faster, maintain and update existing ones, and save valuable engineering cycles.
Visualize your AI Data flow through a friendly UI/UX experience.
UI Catalog
Visually Explore your AI Datastore.
- Explore, Discovery, Curate, and Use your Unstructured Data
- Search and visualize all of your unstructured data in one place.
- Share and act on insights faster.
- Analyze model results. Visually compare model runs and versions.
- All your data, metadata, labels, and predictions at your fingertips you can make better decisions to unblock your AI initiatives.
As your org grows it’s common for folks doing the curation to be different from the ones doing the annotation or model training. Catalog unlocks the value in your existing data by surfacing existing opportunities and saving on unnecessary work. Easily search for existing data and predictions. Diffgram Catalog is built for ease of use. No coding skills required.
Migrate
Migrate from Labelbox and other legacy tools
Our Story
We solved Human Supervision side. Then we realized that people need to see the catalog of their predictions. Once they can see it we got requests to integrate the data with more apps. So we solved these three things. We built it all. And then we realized the common thread between all of them is the AI Datastore. Now we are building The AI Datastore 2.0. We are making all the great use cases for using AI data in your apps, with built-in human supervision, with Data Workflows, and UI Catalogs and more.
The AI Datastore Explained in more Detail
Diffgram AI Datastore is a new core technology that uses multiple sub technologies, such as databases, queues, integrations etc. The heart is the storage of Schemas (Business Objects), Blobs, Predictions, and how those three relate to each other. The initial use cases focus on out of the box support for customer AI apps (e.g. vision models or LLMs), and built-in apps as human supervision, visually exploring datasets, workflow, and integrations with other apps.
One part of the new core storage layer is the ability to have complex definitions for external BLOBs, such as on AWS, Azure, GCP, and true on prem with MinIO. All existing Blobs get upgraded in a new PII Compliant way for AI purposes, meaning that end users can securely access Blobs while maintaining PII compliance. The second key part is Business objects to represent AI data in a meaningful way to your business. The third is Predictions with overall performance inserts and queries, along with economical lifecycle and long term storage options.
The storage layer features common Datastore functions such as the ability to copy, move, and manipulate customer defined business objects in special Diffgram defined datasets. A key value add is the ability for these business objects, e.g. a Drivers License, to be worked with as one object, while Diffgram transparently handles references multiple BLOBs, Schemas, and Predictions sets. In this context "moving" data, means moving it within the Diffgram Datastore structure, but it may or may not mean external movement. In the case of Workflow, handling that in/out external movement is a key value add.
Dive into the Diffgram AI Datastore universe where sophistication meets simplicity. Unlock the true potential of your AI applications today and explore new possibilities, powered by cutting-edge technology and built on the solid foundation of security and compliance.
Updated about 1 year ago