Release 9.10

Release Date: November 7th, 2023

---

New Feature	Improvement	Bug Fix	Enterprise Only

API

Status	Change	Details
	Improved the Platform Tracker performance with a detect-track workflow	Introduced the state-of-the-art BYTE-Track, an online multi-object tracking system built upon the principles of Simple Online and Realtime Tracking (SORT). With BYTE-Track, users can seamlessly integrate it into their detect-track workflows, unlocking advanced capabilities for efficient object tracking.

Python SDK

Status	Change	Details
	Added model inference parameters customization	You can now configure inference parameters such as temperature, max tokens, and more, depending on the specific model you are using, for both text-to-text and text-to-image generative tasks. This empowers you to customize and fine-tune your model interactions to better suit your individual needs.
	Added a robust search interface within the Python SDK for image and text inputs	The SDK now supports vector search (ranking) capabilities and offers advanced filtering options by parameters. You can flexibly refine search results using a variety of criteria, including concepts, image bytes, image URLs, text descriptions, embedded metadata tags, and geo points (longitude and latitude, with radius limits). The search interface also supports AND and OR operators for complex queries. The SDK has also been updated to include schema validation checks to ensure data integrity and search accuracy. You can get examples of how the search functionality works here.
	Fixed SDK model output info parameters bugs related to clarifai-grpc version 9.9.0	This update ensures the SDK resolves the model output info error when invoking the 'postmodel' endpoint with specific parameters.
	Fixed the issue of duplicate input IDs when uploading two similar datasets to the same app	We now append a random value to each input ID during dataset upload. This ensures that input IDs remain unique, allowing users to work with similar inputs across multiple datasets within the same app without encountering conflicts.

Integrations

Status	Change	Details
	Introduced Clarifai and Databricks integration	This integration is achieved via the Clarifai Python SDK and it is available here. This integration enables developers to efficiently manage unstructured data and computing tasks while leveraging Clarifai's computer vision and natural language capabilities. It facilitates seamless data ingestion and movement between Databricks and Clarifai.

PAT

Status	Change	Details
	Added ability to automatically generate a Personal Access Token (PAT) when you create an account	Previously, only app-specific keys were automatically generated when you created an app. A PAT will also now be generated for you during account creation.

New Published Models

Status	Change	Details
	Published several new, ground-breaking models	Wrapped Nougat-base, a Meta AI-developed visual transformer model that converts document images, including complex math equations, into structured text, offering advancements in academic paper parsing. Wrapped Mistral-7B-OpenOrca, a high-performing large language model achieved by fine-tuning the Mistral-7B base model using the OpenOrca dataset. Wrapped Zephyr-7B-alpha, a 7 billion parameter model, fine-tuned on Mistral-7b and outperformed the Llama2-70B-Chat on MT Bench. Wrapped OpenHermes-2-mistral-7B, a 7 billion LLM fine-tuned on Mistral with 900,000 entries of primarily GPT-4 generated data from open datasets. Wrapped Whisper-large-v2, a versatile pre-trained ASR and speech translation model trained on multilingual data without requiring fine-tuning. Wrapped SSD-1B, a diffusion-based text-to-image model—it's 50% smaller and 60% faster than SDXL 1.0. Wrapped Jina-embeddings-v2, an English text embedding model by Jina AI. It’s based on the Bert architecture with an 8192-sequence length, outperforming OpenAI's embedding model in various metrics.

Models

Status	Change	Details
	Improved min_value range for consistency across all model types	For embedding-classifiers, we’ve standardized min_value to have a range of 0 to 1 with a step size of .01. For most of the other model types, we’ve standardized it to have a range of 0 to 100 with a step size of .1.
	Made time information changes to the Centroid Tracker model	We’ve made significant improvements to the Centroid Tracker, specifically within the "time_info" section. We added "start_time" and "end_time" to provide precise information regarding when an object was detected and when detection ceased.
	Made improvements to the Model-Viewer’s version table	We made the changes to make the table more consistent with the evaluation leaderboard. It now provides users with a cohesive and familiar interface. We relocated evaluation actions from a separate module to the table to enhance the user experience.
	Made significant improvements to enhance the dataset and concept selection process when training models	Model builders who haven't yet created datasets or dataset versions can now conveniently choose the 'app default dataset' in the model training editor screen. This option provides visibility into the labeled input counts, allowing users to verify their data before initiating the training process. The concept selection interface now displays the labeled input count for each concept. This feature helps users prevent training concepts without adequate labeled inputs and simplifies the process of identifying data imbalances, all without the need to navigate away from the screen.
	Fixed a modal reappearance issue	If you click the “Use Model” button on an individual model’s page, the “Call by API / Use in a Workflow” modal appears. You can then click the “Copy Code” button to copy the displayed code for integration into your own use case. Previously, if you clicked “Copy Code” and clicked “Use Model” again, the modal would not reappear. We fixed the issue.
	Fixed an issue with selecting a model version on the publish screen	Previously, if you tried choosing a different model version on the publish screen, it resulted in a failure. It occurred when trying to edit the visibility of models with multiple versions. We fixed the issue.
	Fixed an issue where the model page closed unexpectedly	Previously, if you clicked the “View JSON” button to view the prediction output of embedding models in the onboarding flow, the model pop-up could close unexpectedly. We fixed the issue.

Workflows

Status	Change	Details
	Fixed an issue with processing videos using the Universal workflow	The Universal workflow now processes videos as desired.
	Fixed an issue where it was not possible to copy a workflow without first changing its ID	Previously, you could successfully copy a workflow only after changing the copied workflow ID. You can now copy an existing workflow, even if you keep the same workflow name during the copying process, such as from "(workflow name)-copy" to "(workflow name)-copy."

Listing Resources

Status	Change	Details
	Added ability to view whether a resource is available publicly or privately	When listing your own resources, such as models, we've added an icon that clearly indicates whether they are private or shared within the Community.
	Added starring option to modules	Similar to other resources, you can now mark modules as favorites by using the star icon.
	Improved the accessibility of starred resources	Previously, you could only access starred resources by navigating to the top-right profile menu and selecting the “starred” option. You can now easily access both your own and Community resources by choosing either the “All” or “Starred” view on the main screen for listing resources, making it more intuitive to find what you need.

License Types

Status	Change	Details
	Added several new license types	If you want to select a license type for your resource, we've expanded your options to provide a diverse range that can cater to your unique preferences.

Organization Settings and Management

Status	Change	Details
	Enhanced searching for organization members	You can now search for organization members using both their first name and last name, individually or in combination.
	Adjusted a team's app view of organization apps	We removed 'App name,' added a non-sortable 'App description' with a maximum of two lines, introduced 'Date created,' and optionally included 'Last updated' if the information is available via the API.

Search

Status	Change	Details
	Made searchability enhancements on the Community platform	You can now enjoy an upgraded experience when searching by resource ID, user ID, short description, and even markdown notes. These enhancements ensure that you can find the exact information you need more efficiently and accurately.

Input-Manager

Status	Change	Details
	Implemented caching of input thumbnails throughout Input-Manager and Input-Viewer	This caching mechanism significantly enhances the overall efficiency of our system by minimizing the need to repeatedly load or generate thumbnails, resulting in faster response times and smoother interactions for our users.
	Enhanced user experience during smart searches	Instead of blocking user actions, we now display a non-intrusive loading overlay. This overlay will be visible during search requests within the Input-Manager, ensuring that the search grid results remain accessible without disruption. Fixed an the issue where getBaseWorkflowHasMultimodalEmbedder erroneously returned 'false' when the component was not loaded. We modified the component’s behavior and it now returns 'undefined' when it is not loaded. When the return value is 'undefined,' we now disable the search bar to ensure users can only conduct searches of the correct type.
	Improved the behavior of the input upload job monitor in the Input-Manager	If you upload inputs on the Input-Manager, a small sidebar window appears at the bottom-right corner of the screen, providing you with real-time status updates on the upload process. There is also a checkbox in the pop-up window, allowing you to tailor your monitoring preferences to better suit your needs. If the checkbox is checked, the upload monitor will initiate polling. It will also immediately update the input list as new inputs become available. If the checkbox is unchecked, polling will continue. However, the input list will only be updated once ALL jobs have been completed. Previously, there was an issue where unchecking the checkbox would halt polling, preventing updates.
	Prevented manual page refresh during input uploads	We now prevent users from refreshing the page while inputs are still uploading. We display a modal that prompts the user to confirm whether they want to reload the page or not. This ensures users are aware of ongoing uploads and helps avoid unintended disruptions caused by manual page refreshes.
	Fixed issues with fetching input counts	We resolved issues related to the REST API request for fetching counts, which previously lacked proper filters as validated by the backend. The problems led to error responses (400/500 errors).
	Fixed an issue that caused a "No inputs found" message to appear	Previously, if you conducted an image search in the Input-Manager, selected "unlabeled" in the Labels menu, and then refreshed the page, you could get the error message. We fixed it.
	Fixed a "Duplicate input ID" error that was preventing the creation of datasets	This issue occurred when attempting to bulk add 25+ inputs to a dataset by selecting all of them in the Input-Manager page using the checkbox at the top, leading to an error message despite no actual duplicates being present. Notably, the error did not occur when you created a dataset before uploading the inputs and subsequently adding the same inputs from the 'upload inputs' modal. We fixed the issue.
	Fixed an issue when using the OR operator to filter inputs in the Input-Manager	The OR operator lets you filter inputs that do not match the given value. Previously, when the operator was selected, the z-index of the menu overlay could disrupt the user interface elements. We fixed the issue.
	Fixed an issue with showing annotations on the Object Mode in the Input-Manager	Previously, after conducting an image search in the Input Mode and subsequently switching to the Object Mode, the annotation tags failed to display as expected. We fixed the issue.
	Fixed an issue where newly created concepts could not be deselected	You can create a new concept in the “Labels” section of the Input-Manager. The newly created concept will be automatically selected. Previously, you could not deselect the newly created concept unless you selected another concept first. We fixed the issue and you can now unselect a concept that you have just created.
	Fixed an issue with showing error message when uploading inputs	If a user is unable to upload an image, we now clearly communicate the cause of the CORS error.
	Fixed an issue with the clear button in the “Labels” section of the Input-Manager	Previously, the clear button could not deselect selected concepts. We fixed the issue and you can now use it to reset and remove all selected concepts.

Input-Viewer

Status	Change	Details
	Fixed an issue with the Input-Viewer gallery	Previously, if you selected an item with index 0 in the Input-Manager and then accessed the Input-Viewer, scrolling down on the left sidebar failed to load subsequent inputs in the gallery. We fixed the issue.
	Fixed an issue that caused delayed loading of video thumbnails in the Input-Viewer	We've implemented various actions to ensure video thumbnails in the Input Viewer's left-side gallery do not experience significant delays when retrieved from the server.
	Fixed an issue that caused unintended closure of model/workflow selection modal in AI Assist	If you want to choose a model or workflow to assist in input labeling within the Input-Viewer, navigate to the settings for AI Assist. Click the “Select Model or Workflow” search box, and then click the “View all” button. A modal window will open, enabling you to search for your desired model or workflow. Previously, that modal could close inadvertently and the page could be refreshed. We fixed the issue and the modal now remains open without any unexpected closures. You can type in the search bar without triggering unintended events in other components.
	Fixed incorrect overlay display in Predict mode in Input-Viewer	The background overlay is now correctly displayed only when a user is in Annotate mode and in drawing mode without selecting a concept from the top bar.
	Fixed an issue where the tip popover meant for region annotations appeared when you switched to Predict mode	The popover is now shown exclusively when you’re in Annotate mode and you’ve not created any annotations within your application.

Labeling Tasks

Status	Change	Details
	Fixed an issue that caused duplicated AI-assisted suggestions in the right-hand sidebar of the Labeling Tasks screen	Prediction suggestions in the Labeler’s right-hand sidebar are no longer duplicated. We also fixed an issue that caused discrepancies between the API predictions and those displayed in the right-hand panel.
	Fixed an issue that prevented suggestions with scores exceeding 0.995 from being displayed in the Labeling Tasks screen	High-quality prediction suggestions are now visible within the Labeler, allowing you to make informed and accurate labeling decisions.
	Fixed an issue that caused a mismatch between bounding box predictions and actual labels	Previously, there was an inconsistency between the concepts listed in the right-hand sidebar and the corresponding bounding boxes visible in the Labeler's canvas area. We fixed the issue to ensure that the concepts and bounding boxes are now in perfect synchronization.
	Fixed an issue with the partitioned worker labeling strategy	Previously, there were problems with labeling results when tasks were distributed among multiple workers. We fixed the issue, ensuring accurate and consistent labeling outcomes in such scenarios.
	Fixed issues regarding the malfunction of the "APPROVE ALL" button and the uncertainty surrounding the functionality of the "APPROVE USER" button in labeler review-annotation mode	The buttons now operate as expected, and uncertainties regarding the approval status have been eliminated. This enhancement ensures a smoother and more reliable user experience during the process of reviewing annotations.
	Fixed an issue where it was not possible to submit labeled inputs for some labeling tasks	Previously, you could not submit labeled inputs for certain labeled tasks. When working on some labeling tasks, if you loaded the input, selected the relevant concepts, and attempted to submit your labeled input, you could encounter an error. We fixed the issue.

Onboarding Flow

Status	Change	Details
	Reordered the 'Use Model' and 'Use Workflow' tabs in the onboarding flow	In the 'Use Model' or 'Use Workflow' pop-up, we moved 'Call by API' to the top position and made 'Python' the first choice. We applied the changes within the 'Use Model' pop-up, 'Use Workflow' pop-up, and in the onboarding version of 'Use Model.'