Skip to main content

Release 9.10

Release Date: November 7th, 2023


New FeatureImprovementBug FixEnterprise Only
new-featureimprovementbugenterprise

API​

StatusChangeDetails
new-featureImproved the Platform Tracker performance with a detect-track workflow
  • Introduced the state-of-the-art BYTE-Track, an online multi-object tracking system built upon the principles of Simple Online and Realtime Tracking (SORT). With BYTE-Track, users can seamlessly integrate it into their detect-track workflows, unlocking advanced capabilities for efficient object tracking.

Python SDK​

StatusChangeDetails
new-featureAdded model inference parameters customization
  • You can now configure inference parameters such as temperature, max tokens, and more, depending on the specific model you are using, for both text-to-text and text-to-image generative tasks. This empowers you to customize and fine-tune your model interactions to better suit your individual needs.
new-featureAdded a robust search interface within the Python SDK for image and text inputsThe SDK now supports vector search (ranking) capabilities and offers advanced filtering options by parameters.

  • You can flexibly refine search results using a variety of criteria, including concepts, image bytes, image URLs, text descriptions, embedded metadata tags, and geo points (longitude and latitude, with radius limits).
  • The search interface also supports AND and OR operators for complex queries.
  • The SDK has also been updated to include schema validation checks to ensure data integrity and search accuracy.
You can get examples of how the search functionality works here.
bugFixed SDK model output info parameters bugs related to clarifai-grpc version 9.9.0
  • This update ensures the SDK resolves the model output info error when invoking the 'postmodel' endpoint with specific parameters.
bugFixed the issue of duplicate input IDs when uploading two similar datasets to the same app
  • We now append a random value to each input ID during dataset upload. This ensures that input IDs remain unique, allowing users to work with similar inputs across multiple datasets within the same app without encountering conflicts.

Integrations​

StatusChangeDetails
new-featureIntroduced Clarifai and Databricks integrationThis integration is achieved via the Clarifai Python SDK and it is available here.

  • This integration enables developers to efficiently manage unstructured data and computing tasks while leveraging Clarifai's computer vision and natural language capabilities.
  • It facilitates seamless data ingestion and movement between Databricks and Clarifai.

PAT​

StatusChangeDetails
new-featureAdded ability to automatically generate a Personal Access Token (PAT) when you create an account
  • Previously, only app-specific keys were automatically generated when you created an app. A PAT will also now be generated for you during account creation.

New Published Models​

StatusChangeDetails
new-featurePublished several new, ground-breaking models
  • Wrapped Nougat-base, a Meta AI-developed visual transformer model that converts document images, including complex math equations, into structured text, offering advancements in academic paper parsing.
  • Wrapped Mistral-7B-OpenOrca, a high-performing large language model achieved by fine-tuning the Mistral-7B base model using the OpenOrca dataset.
  • Wrapped Zephyr-7B-alpha, a 7 billion parameter model, fine-tuned on Mistral-7b and outperformed the Llama2-70B-Chat on MT Bench.
  • Wrapped OpenHermes-2-mistral-7B, a 7 billion LLM fine-tuned on Mistral with 900,000 entries of primarily GPT-4 generated data from open datasets.
  • Wrapped Whisper-large-v2, a versatile pre-trained ASR and speech translation model trained on multilingual data without requiring fine-tuning.
  • Wrapped SSD-1B, a diffusion-based text-to-image model—it's 50% smaller and 60% faster than SDXL 1.0.
  • Wrapped Jina-embeddings-v2, an English text embedding model by Jina AI. It’s based on the Bert architecture with an 8192-sequence length, outperforming OpenAI's embedding model in various metrics.

Models​

StatusChangeDetails
improvementImproved min_value range for consistency across all model types
  • For embedding-classifiers, we’ve standardized min_value to have a range of 0 to 1 with a step size of .01. For most of the other model types, we’ve standardized it to have a range of 0 to 100 with a step size of .1.
improvementMade time information changes to the Centroid Tracker model
  • We’ve made significant improvements to the Centroid Tracker, specifically within the "time_info" section. We added "start_time" and "end_time" to provide precise information regarding when an object was detected and when detection ceased.
improvementMade improvements to the Model-Viewer’s version table
  • We made the changes to make the table more consistent with the evaluation leaderboard. It now provides users with a cohesive and familiar interface.
  • We relocated evaluation actions from a separate module to the table to enhance the user experience.
improvementMade significant improvements to enhance the dataset and concept selection process when training models
  • Model builders who haven't yet created datasets or dataset versions can now conveniently choose the 'app default dataset' in the model training editor screen. This option provides visibility into the labeled input counts, allowing users to verify their data before initiating the training process.
  • The concept selection interface now displays the labeled input count for each concept. This feature helps users prevent training concepts without adequate labeled inputs and simplifies the process of identifying data imbalances, all without the need to navigate away from the screen.
bugFixed a modal reappearance issueIf you click the “Use Model” button on an individual model’s page, the “Call by API / Use in a Workflow” modal appears. You can then click the “Copy Code” button to copy the displayed code for integration into your own use case.

  • Previously, if you clicked “Copy Code” and clicked “Use Model” again, the modal would not reappear. We fixed the issue.
bugFixed an issue with selecting a model version on the publish screen
  • Previously, if you tried choosing a different model version on the publish screen, it resulted in a failure. It occurred when trying to edit the visibility of models with multiple versions. We fixed the issue.
bugFixed an issue where the model page closed unexpectedly
  • Previously, if you clicked the “View JSON” button to view the prediction output of embedding models in the onboarding flow, the model pop-up could close unexpectedly. We fixed the issue.

Workflows​

StatusChangeDetails
bugFixed an issue with processing videos using the Universal workflow
  • The Universal workflow now processes videos as desired.
bugFixed an issue where it was not possible to copy a workflow without first changing its ID
  • Previously, you could successfully copy a workflow only after changing the copied workflow ID. You can now copy an existing workflow, even if you keep the same workflow name during the copying process, such as from "(workflow name)-copy" to "(workflow name)-copy."

Listing Resources​

StatusChangeDetails
improvementAdded ability to view whether a resource is available publicly or privately
  • When listing your own resources, such as models, we've added an icon that clearly indicates whether they are private or shared within the Community.
improvementAdded starring option to modules
  • Similar to other resources, you can now mark modules as favorites by using the star icon.
improvementImproved the accessibility of starred resources
  • Previously, you could only access starred resources by navigating to the top-right profile menu and selecting the “starred” option. You can now easily access both your own and Community resources by choosing either the “All” or “Starred” view on the main screen for listing resources, making it more intuitive to find what you need.

License Types​

StatusChangeDetails
improvementAdded several new license types
  • If you want to select a license type for your resource, we've expanded your options to provide a diverse range that can cater to your unique preferences.

Organization Settings and Management​

StatusChangeDetails
improvementEnhanced searching for organization members
  • You can now search for organization members using both their first name and last name, individually or in combination.
improvementAdjusted a team's app view of organization apps
  • We removed 'App name,' added a non-sortable 'App description' with a maximum of two lines, introduced 'Date created,' and optionally included 'Last updated' if the information is available via the API.
StatusChangeDetails
improvementMade searchability enhancements on the Community platform
  • You can now enjoy an upgraded experience when searching by resource ID, user ID, short description, and even markdown notes. These enhancements ensure that you can find the exact information you need more efficiently and accurately.

Input-Manager​

StatusChangeDetails
improvementImplemented caching of input thumbnails throughout Input-Manager and Input-Viewer
  • This caching mechanism significantly enhances the overall efficiency of our system by minimizing the need to repeatedly load or generate thumbnails, resulting in faster response times and smoother interactions for our users.
improvementEnhanced user experience during smart searches
  • Instead of blocking user actions, we now display a non-intrusive loading overlay. This overlay will be visible during search requests within the Input-Manager, ensuring that the search grid results remain accessible without disruption.
  • Fixed an the issue where getBaseWorkflowHasMultimodalEmbedder erroneously returned 'false' when the component was not loaded. We modified the component’s behavior and it now returns 'undefined' when it is not loaded. When the return value is 'undefined,' we now disable the search bar to ensure users can only conduct searches of the correct type.
improvementImproved the behavior of the input upload job monitor in the Input-ManagerIf you upload inputs on the Input-Manager, a small sidebar window appears at the bottom-right corner of the screen, providing you with real-time status updates on the upload process. There is also a checkbox in the pop-up window, allowing you to tailor your monitoring preferences to better suit your needs.

  • If the checkbox is checked, the upload monitor will initiate polling. It will also immediately update the input list as new inputs become available.
  • If the checkbox is unchecked, polling will continue. However, the input list will only be updated once ALL jobs have been completed. Previously, there was an issue where unchecking the checkbox would halt polling, preventing updates.
improvementPrevented manual page refresh during input uploads
  • We now prevent users from refreshing the page while inputs are still uploading. We display a modal that prompts the user to confirm whether they want to reload the page or not. This ensures users are aware of ongoing uploads and helps avoid unintended disruptions caused by manual page refreshes.
bugFixed issues with fetching input counts
  • We resolved issues related to the REST API request for fetching counts, which previously lacked proper filters as validated by the backend. The problems led to error responses (400/500 errors).
bugFixed an issue that caused a "No inputs found" message to appear
  • Previously, if you conducted an image search in the Input-Manager, selected "unlabeled" in the Labels menu, and then refreshed the page, you could get the error message. We fixed it.
bugFixed a "Duplicate input ID" error that was preventing the creation of datasets
  • This issue occurred when attempting to bulk add 25+ inputs to a dataset by selecting all of them in the Input-Manager page using the checkbox at the top, leading to an error message despite no actual duplicates being present. Notably, the error did not occur when you created a dataset before uploading the inputs and subsequently adding the same inputs from the 'upload inputs' modal. We fixed the issue.
bugFixed an issue when using the OR operator to filter inputs in the Input-ManagerThe OR operator lets you filter inputs that do not match the given value.

  • Previously, when the operator was selected, the z-index of the menu overlay could disrupt the user interface elements. We fixed the issue.
bugFixed an issue with showing annotations on the Object Mode in the Input-Manager
  • Previously, after conducting an image search in the Input Mode and subsequently switching to the Object Mode, the annotation tags failed to display as expected. We fixed the issue.
bugFixed an issue where newly created concepts could not be deselectedYou can create a new concept in the “Labels” section of the Input-Manager. The newly created concept will be automatically selected.

  • Previously, you could not deselect the newly created concept unless you selected another concept first. We fixed the issue and you can now unselect a concept that you have just created.
bugFixed an issue with showing error message when uploading inputs
  • If a user is unable to upload an image, we now clearly communicate the cause of the CORS error.
bugFixed an issue with the clear button in the “Labels” section of the Input-Manager
  • Previously, the clear button could not deselect selected concepts. We fixed the issue and you can now use it to reset and remove all selected concepts.

Input-Viewer​

StatusChangeDetails
bugFixed an issue with the Input-Viewer gallery
  • Previously, if you selected an item with index 0 in the Input-Manager and then accessed the Input-Viewer, scrolling down on the left sidebar failed to load subsequent inputs in the gallery. We fixed the issue.
bugFixed an issue that caused delayed loading of video thumbnails in the Input-Viewer
  • We've implemented various actions to ensure video thumbnails in the Input Viewer's left-side gallery do not experience significant delays when retrieved from the server.
bugFixed an issue that caused unintended closure of model/workflow selection modal in AI AssistIf you want to choose a model or workflow to assist in input labeling within the Input-Viewer, navigate to the settings for AI Assist. Click the “Select Model or Workflow” search box, and then click the “View all” button. A modal window will open, enabling you to search for your desired model or workflow.

  • Previously, that modal could close inadvertently and the page could be refreshed. We fixed the issue and the modal now remains open without any unexpected closures. You can type in the search bar without triggering unintended events in other components.
bugFixed incorrect overlay display in Predict mode in Input-Viewer
  • The background overlay is now correctly displayed only when a user is in Annotate mode and in drawing mode without selecting a concept from the top bar.
bugFixed an issue where the tip popover meant for region annotations appeared when you switched to Predict mode
  • The popover is now shown exclusively when you’re in Annotate mode and you’ve not created any annotations within your application.

Labeling Tasks​

StatusChangeDetails
bugFixed an issue that caused duplicated AI-assisted suggestions in the right-hand sidebar of the Labeling Tasks screen
  • Prediction suggestions in the Labeler’s right-hand sidebar are no longer duplicated.
  • We also fixed an issue that caused discrepancies between the API predictions and those displayed in the right-hand panel.
bugFixed an issue that prevented suggestions with scores exceeding 0.995 from being displayed in the Labeling Tasks screen
  • High-quality prediction suggestions are now visible within the Labeler, allowing you to make informed and accurate labeling decisions.
bugFixed an issue that caused a mismatch between bounding box predictions and actual labels
  • Previously, there was an inconsistency between the concepts listed in the right-hand sidebar and the corresponding bounding boxes visible in the Labeler's canvas area. We fixed the issue to ensure that the concepts and bounding boxes are now in perfect synchronization.
bugFixed an issue with the partitioned worker labeling strategy
  • Previously, there were problems with labeling results when tasks were distributed among multiple workers. We fixed the issue, ensuring accurate and consistent labeling outcomes in such scenarios.
bugFixed issues regarding the malfunction of the "APPROVE ALL" button and the uncertainty surrounding the functionality of the "APPROVE USER" button in labeler review-annotation mode
  • The buttons now operate as expected, and uncertainties regarding the approval status have been eliminated. This enhancement ensures a smoother and more reliable user experience during the process of reviewing annotations.
bugFixed an issue where it was not possible to submit labeled inputs for some labeling tasks
  • Previously, you could not submit labeled inputs for certain labeled tasks. When working on some labeling tasks, if you loaded the input, selected the relevant concepts, and attempted to submit your labeled input, you could encounter an error. We fixed the issue.

Onboarding Flow​

StatusChangeDetails
improvementReordered the 'Use Model' and 'Use Workflow' tabs in the onboarding flow
  • In the 'Use Model' or 'Use Workflow' pop-up, we moved 'Call by API' to the top position and made 'Python' the first choice.
  • We applied the changes within the 'Use Model' pop-up, 'Use Workflow' pop-up, and in the onboarding version of 'Use Model.'