Use AI to help you build AI. Auto annotation uses your model predictions to label your training data
This tutorial demonstrates how auto-annotation workflows can be configured within Clarifai Portal. With auto-annotation, you can use model predictions to label your inputs. Auto-annotation can help you to prepare training data, or assign other useful labels and metadata to your inputs.
When a concept is predicted by a model, it is predicted with a confidence score between 0 and 1. When your model predictions are confident (close to 1), you can have your data automatically labeled with that concept. When your predictions are less-than-confident, you can have your input sent to a human being for review.
This enables you to speed-up and scale-up your annotation process while ensuring quality standards.
Create Concepts, Label and Train a Custom Model
To begin, create your application and upload your inputs.
Upload files from your computer, or add image and video URLs.
Create the concepts that you would like to use for your model. Click create new concept in the lefthand sidebar. In this tutorial we'll create concepts that describe people based on activities they do in the park:
Label your inputs with the appropriate concepts by clicking the checkmark box.
Navigate to the Model Gallery
Now it is time to create some custom models.
Create a Context-Based Classifier
A Context-Based Classifier lets you create a custom classification model that will predict the concepts you have created. Choose a
DISPLAY NAME and click in the
OUTPUT_INFO.DATA.CONCEPTS box to select the concepts that you would like included in your model. Finally, click "Create Model" and then click "Train Model" in the next page.
GREATER THAN and
LESS THAN Concept Thresholder models
Concept Thresholders help you route your data based on the confidence of your predictions. Choose a
DISPLAY NAME and add the concepts you would like to route under the
OUTPUT_INFO.DATA heading. You can choose separate concept thresholds for each of your custom concepts. Create one model with high
CONCEPT THRESHOLDS and select
OUTPUT_INFO.PARAMS.CONCEPT_THRESHOLD_TYPE. Create a second Concept Thresholder model with low
CONCEPT THRESHOLDS and select
LESS_THAN model will help you screen out concepts that are predicted with very low confidence.
WRITE SUCCESS and
WRITE PENDING Annotation Writer models
The Annotation Writer writes each annotation with a specific concept, status and user. Create one Annotation Writer that will write the annotation with the
ANNOTATION_SUCCESS status and choose the user (possibly yourself) under
OUTPUT_INFO.PARAMS.ANNOTATION_USER_ID. Create a second Annotation Writer that will write with the
Create the workflow
We will now join all the models together into a single workflow. In the Clarifai platform, the outputs from one model can be used as inputs to another model. Different models accept and produce different types of inputs and outputs.
The Context-Based Classifier will return list of concepts for each of your inputs. These inputs will then be sent to the Concept Thresholder models.
GREATER THAN model will filter out the concept if it lower than the threshold and send the final concept list to
WRITE SUCCESS model. Concepts will then be written to the input with
SUCCESS status. You can train or search on these concepts immediately.
LESS THAN model will filter out the concepts with low prediction scores. The remaining concept list will be sent to
WRITE PENDING. Concepts will then be written to the input with
Make the new workflow your app's default
Make this the default workflow in the app, so it will run every time we add an input and execute the auto annotation process. If the workflow is not the default workflow of your app you can still use PostWorkflowResults on new inputs to check that you configured the workflow graph and your models properly but the data will not be written to the DB. This is recommended before making it your default workflow and adding inputs to you app.
Add new images
Adding new images will trigger the default workflow in your app.
View Annotations in Explorer
Now you can view your annotations in Explorer. Just drag your mouse over the "tag" icon to see annotations associated with the image.