Skip to main content

Release 10.2

Release Date: March 5th, 2024



New FeatureImprovementBug FixEnterprise Only
new-featureimprovementbugenterprise

Evaluation

StatusChangeDetails
new-feature [Developer Preview]Introduced a module for evaluating large language models (LLMs)You can use the module to evaluate the performance of LLMs against standardized benchmarks alongside custom criteria to gain deep insights into their strengths and weaknesses.

Here are some of its key features:

  • Evaluate across 100+ tasks covering diverse use cases like RAG, classification, casual chat, content summarization, and more. Each use case provides the flexibility to choose from relevant evaluation classes like Helpfulness, Relevance, Accuracy, Depth, and Creativity. You can further enhance the customization by assigning user-defined weights to each class.
  • Define weights on each evaluation class to create custom weighted scoring functions. This lets you measure business-specific metrics and store them for consistent use. For example, for RAG-related evaluation, you may want to give zero weight to Creativity and more weights for Accuracy, Helpfulness, and Relevance.
  • Save the best performing prompt-model combinations as a workflow with a single click for future reference.

New Published Models

StatusChangeDetails
new-featurePublished new models

(Clarifai-hosted models are the ones we host within our Clarifai Cloud. Wrapped models are those hosted externally, but we deploy them on our platform using their third-party API keys.)
  • Wrapped Google Gemini Pro Vision, which was created from the ground up to be multimodal (text, images, videos) and scale across a wide range of tasks.
  • Wrapped Claude 3 Opus, a state-of-the-art, multimodal language model (LLM) with superior performance in reasoning, math, coding, and multilingual understanding.
  • Wrapped Claude 3 Sonnet, a multimodal LLM balancing skills and speed, excelling in reasoning, multilingual tasks, and visual interpretation.
  • Wrapped Qwen1.5-72B-Chat, which leads in language understanding, generation, and alignment, setting new standards in conversational AI and multilingual capabilities, outperforming GPT-4, GPT-3.5, Mixtral-8x7B, and Llama2-70B on many benchmarks.
  • Wrapped DeepSeek-Coder-33B-Instruct, a SOTA 33 billion parameter code generation model, fine-tuned on 2 billion tokens of instruction data, offering superior performance in code completion and infilling tasks across more than 80 programming languages.
  • Clarifai-hosted Gemma-2b-it, a part of Google DeepMind's lightweight, Gemma family LLM, offering exceptional AI performance on diverse tasks by leveraging a training dataset of 6 trillion tokens, focusing on safety and responsible output.
  • Clarifai-hosted Gemma-7b-it, an instruction fine-tuned LLM, lightweight, open model from Google DeepMind that offers state-of-the-art performance for natural language processing tasks, trained on a diverse dataset with rigorous safety and bias mitigation measures.
  • Clarifai-hosted DeciLM-7B-Instruct, a state-of-the-art, efficient, and highly accurate 7 billion parameter LLM, setting new standards in AI text generation.

Python SDK

StatusChangeDetails
improvementMade enhancements to the Python SDK
  • Updated and cleaned the requirements.txt file for the SDK.
  • Fixed an issue where a failed training job led to a bug when loading a model in the Clarifai-Python client library, and concepts were replicated when their IDs did not match.
improvementMade enhancements to the RAG (Retrieval Augmented Generation) feature
  • Enhanced the RAG SDK's upload() function to accept the dataset_id parameter.
  • Enabled custom workflow names to be specified in the RAG SDK's setup() function.
  • Fixed scope errors related to the user and now_ts variables in the RAG SDK by correcting their definition placement, which was previously inside an if statement.
  • Added support for chunk sequence numbers in the metadata when uploading chunked documents via the RAG SDK.

Community UI

StatusChangeDetails
new-featureAdded a notification for remaining time for free deep training
  • Added a notification at the upper-right corner of the Select a model type page about the number of hours left for deep training your models for free.
new-featureAdded feedback form
  • Added feedback form links to the header and listings pages of models, workflows, and modules. This enables registered users to provide general feedback or request a specific model.
new-featureAdded a display of inference pricing per request
  • The model and workflow pages now display the price per request for both logged-in and non-logged-in users.
improvementImplemented progressive image loading for images
  • Progressive image loading displays low-resolution versions of images initially, gradually replacing them with higher-resolution versions as they become available. It solves page load issues and preserves image sharpness.
improvementReplaced spaces with dashes in IDs
  • When updating User, App, or any other resource IDs, spaces will be replaced with dashes.
improvementUpdated links
  • Updated the text and link for the Slack community in the navbar's info popover to 'Join our Discord Channel.' Similarly, updated the link similar to it at the bottom of the landing page to direct to Discord.
  • Removed the "Where's Legacy Portal?" text.
improvementDisplay name in PAT toast notification
  • We've updated the account security page to display a PAT name instead of PAT characters in the toast notification.
bugFixed an issue with the Create an App modal
  • Previously, errors displayed in the App ID section could overlap, making them difficult to read. We fixed it.

Mobile

StatusChangeDetails
improvementImproved the mobile onboarding flow
  • Made minor updates to mobile onboarding.
improvementImproved sidebar appearance
  • Enhanced sidebar appearance when folded in mobile view.

Apps

StatusChangeDetails
improvementAdded an option to edit the scopes of a collaborator
  • You can now edit and customize the scopes associated with a collaborator’s role on the App Settings page.

Leaderboard

StatusChangeDetails
bugFixed an issue with inconsistency in the Leaderboard sorting
  • Previously, there was an issue where the leaderboard filtering was not functioning correctly under certain situations. We fixed it.
bugFixed an issue with the Leaderboard where duplicate items caused confusion and usability concerns
  • Previously, duplicate items could appear in the dataset sidebar, leading to uncertainty when selecting them and confusion with the hover state. We’ve rectified this to enhance clarity and user experience.

Input-Manager

StatusChangeDetails
bugFixed issues with uploading bulk inputs
  • We fixed an issue where it was not possible to upload multiple files with the same name.
  • We fixed an issue where it was not possible to reupload failed or canceled uploads.
bugImprovements made to text preview on text tiles in the Input-Manager
  • Previously, when a single string (a single word) was excessively long, it remained unbroken and did not wrap onto separate lines. Additionally, adjusting the zoom level of the grid did not affect the amount of text displayed. We fixed the issues, and the texts are now displayed as desired.

Input-Viewer

StatusChangeDetails
bugFixed an issue with renaming a concept in the Input-Viewer
  • Previously, if you added a concept and renamed the concept, you could not use the highlighted input box to “filter by concept” for the new relation that was created. We fixed the issue, and you can now filter by concept using the original name as well as the new relation that was created for it.
bugFixed an issue with the gearbox icon in the Input-Viewer
  • Previously, opening the gearbox popover, selecting a model or workflow, and then closing the popover resulted in the gearbox icon remaining gray, indicating that nothing was selected inside the popover. We fixed the issue, and now the gearbox icon turns blue when an item is selected within its popover.
bugFixed an issue with entering the same name of a concept
  • For example, previously, if you generated predictions from classifications — let’s say one of the predicted concepts is “sky.” If you edited “sky” to “wall,” a new relation was created. However, if you edited the concept name again to “sky,” a new relation was erroneously created. We fixed the issue, and now since the relation and the originally predicted concept are the same, we remove the “wall” relation.
bugFixed an issue where deleting generated annotations was possible in the Input-Viewer
  • Previously, users could delete AI-assisted annotation suggestions made using the General-detection workflow, regardless of whether they were their own or not. We fixed the issue, and now users can only delete their own annotations.
bugFixed an issue that triggered an 'unsaved' warning when navigating away from the Input-Viewer
  • This issue caused confusion as the browser mistakenly indicated unsaved changes when attempting to reload the page. We fixed it.

Labeling Tasks

StatusChangeDetails
bugFixed an issue with AI-assisted bounding box annotations in the Labeling Tasks screen
  • Previously, when multiple AI-assisted annotations were present for the same concept in an input, clicking the check mark to accept one of them could result in the name of the selected annotation changing unexpectedly. We fixed it.

Organization Settings and Management

StatusChangeDetails
bugFixed an issue with the cancel button functionality when deleting an organization
  • Previously, the cancel button on the dialog for deleting an organization did not function as expected. We fixed the issue. Additionally, we have added the organization name to the dialog, ensuring users know precisely what they are deleting.

Models

StatusChangeDetails
improvementEnabled deletion of associated model assets when removing a model annotation
  • Now, when deleting a model annotation, the associated model assets are also marked as deleted.

Workflows

StatusChangeDetails
improvementImproved model selection
  • Made improvements to the model selection drop-down list on the workflow builder.
bugFixed an issue with creating workflows
  • Previously, users experienced an issue where the input node would move unexpectedly when models were dropped into the graph during the initial workflow creation process. This required users to manually adjust the location of the input node before proceeding with workflow creation. We have fixed this issue to ensure a smoother workflow creation experience.
bugFixed an issue where underlines showed up for some buttons
  • Previously, certain buttons, such as those found on the workflow page, were displaying underlines unnecessarily. We fixed it.
bugFixed an issue with the RAG Prompter model template
  • Previously, when creating a RAG Prompter model with a custom template and assigning it to a workflow via an API call, the API call result indicated that the Prompter model template was updated. However, the updated template was not visible in the workflow UI. We fixed the issue.

Modules

StatusChangeDetails
bugFixed issues with some modules

Others

StatusChangeDetails
bugFixed an issue with the pending email verification modal
  • Previously, when an email address was long, it could extend beyond the boundary of the 'Please verify your email' modal. We fixed it.
bugFixed an issue with SSO authenticationYou can log in to the Clarifai platform via an SSO method (either GitHub or Google), complete the SSO authentication steps, and get redirected to Clarifai seamlessly.

  • Previously, if you failed to complete the SSO login within 500 seconds, the login session could expire, resulting in an 'Invalid or expired oauth request state' error. To address this, we have extended the expiration time to allow users enough time to complete any authentication steps with their SSO provider.