Skip to main content

Smart Search on Inputs-Manager

Use vector search to find, sort, and filter what you need in your data

The Clarifai’s Smart Search feature on the Input-Manager screen allows you to harness the power of AI to search through unstructured or structured data with ease.

Whether you're searching for images, videos, or text data, Smart Search empowers you to quickly find the content you need based on visual, semantic and/or conceptual understanding.

With Smart Search, you can unlock the full potential of your data assets, making it easier than ever to locate and retrieve the data of interest. For example, you can use the search results within your app to accelerate data labeling for custom model training tasks.

The Input-Manager screen is the main page showcasing all the inputs in your app. You can access it by clicking the Inputs option in your app's collapsible left sidebar. The screen has two modes: Input Mode (default) and Object Mode.

info
  • When performing a Smart Search with custom concepts, ensure that these concepts are first trained using an embedding-classifier model (transfer-learning model). Without this training, the search query may not work. In the Input-Manager screen, concepts that have already been trained with a model are marked with a blue circle, while untrained concepts are marked with a grey circle.
  • When you upload inputs to our platform, we use your app’s base workflow to index them. This immediately makes them searchable by concept, annotation, or other advanced parameters. You can update the search index by clicking the Update button in the search bar, then clicking the reindex button in menu that drops down.

Fully-Managed Vector Search Engine​

Our Smart Search feature leverages vector search capabilities to power the search experience. Vector search is a type of search engine that uses vectors to search and retrieve text, images, and videos.

Vector embeddings are numerical “representations” of unstructured data, which enable their meaning to be encoded and processed mathematically. By converting the data into vectors, which is a language native to computers, we can efficiently perform search operations on them.

Instead of traditional keyword-based search, where exact matches are sought, vector search allows for searching based on visual and/or semantic similarity by calculating distances between vector embedding representations of the data.

Powered by a Vector Database​

Our vector search engine uses deep learning embedding models to first analyze the visual features of each input, such as color, shape, and texture. This process, known as feature extraction, generates a corresponding vector representation for each piece of unstructured data.

The embedding models then index these vector representations and store them in our vector database (also called a vector store or a semantic search engine).

When a user performs a search, their query is also converted into a vector representation. The vector DB then searches for the vector representations that are most similar to the query vector representation. The results are then displayed to the user.

By using our vector search as a service, you can get more relevant search results, faster search times, and scalable performance.

Simplifies Smart Search Integration​

Choosing Clarifai's turnkey smart search solution is better than building your own from scratch.

Without Clarifai, you would need to do the following in order to build smart search features into your own solutions:

  • Set up your own vector database instance. This involves choosing a database platform, installing the software, and configuring the database.
  • Build out the entire pipeline for turning images/text into embeddings. This involves using a computer vision or natural language processing (NLP) library to extract features from images or text, and then converting those features into a vector representation.
  • Insert or query the vector database. This involves using the database's API to add new data or search for existing data.

This process can be very time-consuming and complex, especially for developers who are not familiar with vector databases or machine learning.

Clarifai eliminates the need for developers to do all of this work by providing an out of the box solution for building state-of-the-art smart search capabilities.

We offer the following types of Smart Search options in our platform.

  • Smart Image Search — Allows you to retrieve images sorted by their visual relevance to a query in the form of:

    • Image — Provide a reference image of interest to compare inputs against.
    • Concept — Provide a trained concept to compare input predictions against.
    • Caption — Provide a full-text description to compare inputs against.

  • Smart Object Search — Allows you to retrieve annotated objects (bounding boxes within images) sorted by their visual relevance to a query in the form of:

    • Image — Provide a reference image of interest to compare inputs against.
    • Concept — Provide a trained concept to compare input predictions against.
    • Caption — Provide a full-text description to compare inputs against.

  • Smart Text Search — Allows you to retrieve text data sorted by their content and semantic similarity to a query in the form of:

    • Text — Provide a text description to compare input predictions against.
    • Concept — Provide a trained concept to compare input predictions against.
© 2024 Clarifai, Inc. All rights reserved