Skip to main content

Tasks : Create, Get, Update, Delete

Group your labeling work into tasks that can be delegated


Task is the work that needs to be done for labeling the inputs in an app. It's a powerful way to help your team annotate inputs fast and efficiently.

These are some parameters you can specify when working with tasks:

  • Task type — It can be:
    • CONCEPTS_CLASSIFICATION — Concepts classification tasks annotate concepts for the overall image, frame of video, or section of text.
    • BOUNDING_BOX_DETECTION — Bounding box detection tasks annotate rectangular bounding box regions around each concept in an image, frame of video, or section of text.
    • POLYGON_DETECTION — Polygon detection tasks annotate free-form regions around concepts in an image, frame of video, or section of text.
  • Worker — Task worker includes information about the workers who will work on the task.
  • Concept IDs — List of concept IDs used in the work on the task. The concepts should already be existing in your app.
  • Task worker strategy — It can be:
    • PARTITIONED — The inputs will be partitioned in several partitions. Each worker will label one or more input partitions.
    • FULL — Each worker will label all inputs from the input source.
  • Input source — It can be:
    • ALL_INPUTS — Use all inputs in the app.
    • SAVED_SEARCH — Use the inputs from a saved search. You also need to specify the id of the saved search.
    • DATASET — Use inputs from a dataset.
  • sample_ms — Used in video model predictions. It specifies the sample delay for video predictions (1 frame per N milliseconds).
  • Review strategy — It can be:
    • NONE — No review is needed.
    • MANUAL — Manual review strategy.
    • CONSENSUS — Consensus review strategy.
  • Partitioned strategy info — It can be:
    • EVENLY — Each worker will label (approximately) the same number of inputs.
    • WEIGHTED — Each worker will have an assigned weight.
  • Workers per input — The number of workers who will label each input.
info

The initialization code used in the following examples is outlined in detail on the client installation page.

Create

To create a new task in your app, you POST the task information to the v2/task endpoint.

Non-Assigned Task

A task should be assigned to a list of users, but it's not required. The following code will create a non-assigned task.

#################################################################################
# In this section, we set the user authentication, app ID, and concept ID.
# Change these strings to run your own example.
#################################################################################

USER_ID = 'YOUR_USER_ID_HERE'
# Your PAT (Personal Access Token) can be found in the Account's Security section
PAT = 'YOUR_PAT_HERE'
APP_ID = 'YOUR_APP_ID_HERE'
# Change this to create your own non-assigned task
CONCEPT_ID = 'water'

##########################################################################
# YOU DO NOT NEED TO CHANGE ANYTHING BELOW THIS LINE TO RUN THIS EXAMPLE
##########################################################################

from clarifai_grpc.channel.clarifai_channel import ClarifaiChannel
from clarifai_grpc.grpc.api import resources_pb2, service_pb2, service_pb2_grpc
from clarifai_grpc.grpc.api.status import status_code_pb2

channel = ClarifaiChannel.get_grpc_channel()
stub = service_pb2_grpc.V2Stub(channel)

metadata = (('authorization', 'Key ' + PAT),)

userDataObject = resources_pb2.UserAppIDSet(user_id=USER_ID, app_id=APP_ID)

post_tasks_response = stub.PostTasks(
service_pb2.PostTasksRequest(
user_app_id=userDataObject, # The userDataObject is created in the overview and is required when using a PAT
tasks=[
resources_pb2.Task(
type='CONCEPTS_CLASSIFICATION',
name='Annotate ' + CONCEPT_ID,
worker=resources_pb2.TaskWorker(
strategy='FULL'
),
concepts=[
resources_pb2.TaskConcept(
concept=resources_pb2.Concept(
id=CONCEPT_ID
)
)
],
input_source=resources_pb2.TaskInputSource(
type='ALL_INPUTS'
),
sample_ms=1000,
review=resources_pb2.TaskReview(
strategy='NONE'
)
)
]
),
metadata=metadata
)

if post_tasks_response.status.code != status_code_pb2.SUCCESS:
print(post_tasks_response.status)
raise Exception("Post tasks failed, status: " + post_tasks_response.status.description)

Assigned Task

A task should be assigned to a list of users. These users will do the labeling work, so they're also called workers. A task may also be assigned to a list of users for review purposes.

#################################################################################
# In this section, we set the user authentication, app ID, and details for
# assigning a task. Change these strings to run your own example.
#################################################################################

USER_ID = 'YOUR_USER_ID_HERE'
# Your PAT (Personal Access Token) can be found in the Account's Security section
PAT = 'YOUR_PAT_HERE'
APP_ID = 'YOUR_APP_ID_HERE'
# Change these to assign your own task
CONCEPT_ID = 'water'
WORKER_USER_ID = 'WORKER_USER_ID_HERE'
REVIEWER_USER_ID = 'REVIEWER_USER_ID_HERE' # User who will review this task

##########################################################################
# YOU DO NOT NEED TO CHANGE ANYTHING BELOW THIS LINE TO RUN THIS EXAMPLE
##########################################################################

from clarifai_grpc.channel.clarifai_channel import ClarifaiChannel
from clarifai_grpc.grpc.api import resources_pb2, service_pb2, service_pb2_grpc
from clarifai_grpc.grpc.api.status import status_code_pb2

channel = ClarifaiChannel.get_grpc_channel()
stub = service_pb2_grpc.V2Stub(channel)

metadata = (('authorization', 'Key ' + PAT),)

userDataObject = resources_pb2.UserAppIDSet(user_id=USER_ID, app_id=APP_ID)

post_tasks_response = stub.PostTasks(
service_pb2.PostTasksRequest(
user_app_id=userDataObject, # The userDataObject is created in the overview and is required when using a PAT
tasks=[
resources_pb2.Task(
type='CONCEPTS_CLASSIFICATION',
name='Annotate ' + CONCEPT_ID,
worker=resources_pb2.TaskWorker(
strategy='FULL',
workers=[
resources_pb2.Worker(
user=resources_pb2.User(
id=WORKER_USER_ID
)
)
]
),
concepts=[
resources_pb2.TaskConcept(
concept=resources_pb2.Concept(
id=CONCEPT_ID
)
)
],
input_source=resources_pb2.TaskInputSource(
type='ALL_INPUTS'
),
sample_ms=1000,
review=resources_pb2.TaskReview(
strategy='MANUAL',
manual_strategy_info=resources_pb2.TaskReviewManualStrategyInfo(
sample_percentage=0.5
),
users=[
resources_pb2.User(
id=REVIEWER_USER_ID
)
]
)
)
]
),
metadata=metadata
)

if post_tasks_response.status.code != status_code_pb2.SUCCESS:
print(post_tasks_response.status)
raise Exception('Post tasks failed, status: ' + post_tasks_response.status.description)

Task With Partitioned Worker Strategy

The previous tasks were created with full worker strategy.

{
"strategy": "FULL"
}

In case of FULL worker strategy, each worker will work on all inputs selected in the input source.

If you wish the work to be distributed between workers, then you can select the PARTITIONED worker strategy.

In the following example, there are two workers:

  • workers_per_input: each input will be assigned to 1 worker
  • weights.user_id_1: the first worker will get 90% of inputs
  • weights.user_id_2: the second worker will get 10% of inputs
#################################################################################
# In this section, we set the user authentication, app ID, and details for
# creating a task. Change these strings to run your own example.
#################################################################################

USER_ID = 'YOUR_USER_ID_HERE'
# Your PAT (Personal Access Token) can be found in the Account's Security section
PAT = 'YOUR_PAT_HERE'
APP_ID = 'YOUR_APP_ID_HERE'
# Change these to create your own task with partitioned worker strategy
CONCEPT_ID = 'water'
USER_ID_1 = 'USER_ID_1_HERE'
USER_ID_2 = 'USER_ID_2_HERE'

##########################################################################
# YOU DO NOT NEED TO CHANGE ANYTHING BELOW THIS LINE TO RUN THIS EXAMPLE
##########################################################################

from clarifai_grpc.channel.clarifai_channel import ClarifaiChannel
from clarifai_grpc.grpc.api import resources_pb2, service_pb2, service_pb2_grpc
from clarifai_grpc.grpc.api.status import status_code_pb2
from google.protobuf.struct_pb2 import Struct

channel = ClarifaiChannel.get_grpc_channel()
stub = service_pb2_grpc.V2Stub(channel)

params = Struct()
params.update({
USER_ID_1: 90,
USER_ID_2: 10
})

metadata = (('authorization', 'Key ' + PAT),)

userDataObject = resources_pb2.UserAppIDSet(user_id=USER_ID, app_id=APP_ID)

post_tasks_response = stub.PostTasks(
service_pb2.PostTasksRequest(
user_app_id=userDataObject, # The userDataObject is created in the overview and is required when using a PAT
tasks=[
resources_pb2.Task(
type='CONCEPTS_CLASSIFICATION',
name='Annotate ' + CONCEPT_ID,
worker=resources_pb2.TaskWorker(
strategy='PARTITIONED',
users=[
resources_pb2.User(id=USER_ID_1),
resources_pb2.User(id=USER_ID_2)
],
partitioned_strategy_info=resources_pb2.TaskWorkerPartitionedStrategyInfo(
type='WEIGHTED',
workers_per_input=1,
weights=params
)
),
concepts=[
resources_pb2.TaskConcept(
concept=resources_pb2.Concept(id=CONCEPT_ID)
)
],
input_source=resources_pb2.TaskInputSource(type='ALL_INPUTS'),
sample_ms=1000,
review=resources_pb2.TaskReview(strategy='NONE')
)
]
),
metadata=metadata
)

if post_tasks_response.status.code != status_code_pb2.SUCCESS:
print(post_tasks_response.status)
raise Exception('Post tasks failed, status: ' + post_tasks_response.status.description)
info
  • It is not required for the weights to add up to 100. For example, the weights [9, 1] are equivalent with weights [90, 10].
  • The partitioning is approximate. This means that the number of assigned inputs to each worker may have a small error margin, but it will be close to the assigned weight percentage.

Task With Consensus Review

The previous tasks were created with no review or manual review strategy.

{
"strategy": "MANUAL"
}

We recommend to create tasks with CONSENSUS review strategy. When enough workers label an input in the same way, it will automatically be approved, with no need for the reviewer to spend time to check. In this way, the reviewer will be able to focus on the inputs where the workers don't agree.

Note that an approval threshold must be set. It is the number of labelers that need to agree in order to automatically approve an annotation.

For example, in case of 3 workers and approval_threshold set to 2, if an input is labeled in the same way by 2 workers, they form a majority and the group reaches a consensus.

#################################################################################
# In this section, we set the user authentication, app ID, and details for
# creating a task. Change these strings to run your own example.
#################################################################################

USER_ID = 'YOUR_USER_ID_HERE'
# Your PAT (Personal Access Token) can be found in the Account's Security section
PAT = 'YOUR_PAT_HERE'
APP_ID = 'YOUR_APP_ID_HERE'
# Change these to create your own task with consensus review
CONCEPT_ID = 'water'
USER_ID_1 = 'USER_ID_1_HERE'
USER_ID_2 = 'USER_ID_2_HERE'
USER_ID_3 = 'USER_ID_3_HERE'
USER_ID_4 = 'USER_ID_4_HERE'

##########################################################################
# YOU DO NOT NEED TO CHANGE ANYTHING BELOW THIS LINE TO RUN THIS EXAMPLE
##########################################################################

from clarifai_grpc.channel.clarifai_channel import ClarifaiChannel
from clarifai_grpc.grpc.api import resources_pb2, service_pb2, service_pb2_grpc
from clarifai_grpc.grpc.api.status import status_code_pb2
from google.protobuf.struct_pb2 import Struct

channel = ClarifaiChannel.get_grpc_channel()
stub = service_pb2_grpc.V2Stub(channel)

params = Struct()
params.update({
USER_ID_1: 1,
USER_ID_2: 1,
USER_ID_3: 1
})

metadata = (('authorization', 'Key ' + PAT),)

userDataObject = resources_pb2.UserAppIDSet(user_id=USER_ID, app_id=APP_ID)

post_tasks_response = stub.PostTasks(
service_pb2.PostTasksRequest(
user_app_id=userDataObject, # The userDataObject is created in the overview and is required when using a PAT
tasks=[
resources_pb2.Task(
type='CONCEPTS_CLASSIFICATION',
name='Annotate ' + CONCEPT_ID,
worker=resources_pb2.TaskWorker(
strategy='PARTITIONED',
users=[
resources_pb2.User(id=USER_ID_1),
resources_pb2.User(id=USER_ID_2),
resources_pb2.User(id=USER_ID_3)
],
partitioned_strategy_info=resources_pb2.TaskWorkerPartitionedStrategyInfo(
type='WEIGHTED',
workers_per_input=3,
weights=params
)
),
concepts=[
resources_pb2.TaskConcept(
concept=resources_pb2.Concept(id=CONCEPT_ID)
)
],
input_source=resources_pb2.TaskInputSource(type='ALL_INPUTS'),
sample_ms=1000,
review=resources_pb2.TaskReview(
strategy='CONSENSUS',
consensus_strategy_info=resources_pb2.TaskReviewConsensusStrategyInfo(
approval_threshold=2
),
users=[resources_pb2.User(id=USER_ID_4)]
)
)
]
),
metadata=metadata
)

if post_tasks_response.status.code != status_code_pb2.SUCCESS:
print(post_tasks_response.status)
raise Exception('Post tasks failed, status: ' + post_tasks_response.status.description)

Get

Get Task by ID

You can get the details of a single task by its ID, which is automatically generated upon task creation. You can output the API response to obtain this task ID.

Alternatively, you can also locate the task ID within the Clarifai platform's user interface.

#################################################################################
# In this section, we set the user authentication, app ID, and task ID.
# Change these strings to run your own example.
#################################################################################

USER_ID = 'YOUR_USER_ID_HERE'
# Your PAT (Personal Access Token) can be found in the Account's Security section
PAT = 'YOUR_PAT_HERE'
APP_ID = 'YOUR_APP_ID_HERE'
# Change this to get a task by its ID
TASK_ID = 'c454edb9446c4de58d4fe3a66c286e55'

##########################################################################
# YOU DO NOT NEED TO CHANGE ANYTHING BELOW THIS LINE TO RUN THIS EXAMPLE
##########################################################################

from clarifai_grpc.channel.clarifai_channel import ClarifaiChannel
from clarifai_grpc.grpc.api import resources_pb2, service_pb2, service_pb2_grpc
from clarifai_grpc.grpc.api.status import status_code_pb2

channel = ClarifaiChannel.get_grpc_channel()
stub = service_pb2_grpc.V2Stub(channel)

metadata = (('authorization', 'Key ' + PAT),)

userDataObject = resources_pb2.UserAppIDSet(user_id=USER_ID, app_id=APP_ID)

get_tasks_response = stub.GetTask(
service_pb2.GetTaskRequest(
user_app_id=userDataObject, # The userDataObject is created in the overview and is required when using a PAT
task_id=TASK_ID
),
metadata=metadata
)

if get_tasks_response.status.code != status_code_pb2.SUCCESS:
print(get_tasks_response.status)
raise Exception(f'Get task failed, status: {get_tasks_response.status.description}')

print(get_tasks_response)

List All Tasks

You can get a list of tasks within your app with a GET call. This call supports pagination.

#######################################################################
# In this section, we set the user authentication and app ID.
# Change these strings to run your own example.
#######################################################################

USER_ID = 'YOUR_USER_ID_HERE'
# Your PAT (Personal Access Token) can be found in the Account's Security section
PAT = 'YOUR_PAT_HERE'
APP_ID = 'YOUR_APP_ID_HERE'

##########################################################################
# YOU DO NOT NEED TO CHANGE ANYTHING BELOW THIS LINE TO RUN THIS EXAMPLE
##########################################################################

from clarifai_grpc.channel.clarifai_channel import ClarifaiChannel
from clarifai_grpc.grpc.api import resources_pb2, service_pb2, service_pb2_grpc
from clarifai_grpc.grpc.api.status import status_code_pb2

channel = ClarifaiChannel.get_grpc_channel()
stub = service_pb2_grpc.V2Stub(channel)

metadata = (('authorization', 'Key ' + PAT),)

userDataObject = resources_pb2.UserAppIDSet(user_id=USER_ID, app_id=APP_ID)

list_tasks_response = stub.ListTasks(
service_pb2.ListTasksRequest(
user_app_id=userDataObject, # The userDataObject is created in the overview and is required when using a PAT
),
metadata=metadata
)

if list_tasks_response.status.code != status_code_pb2.SUCCESS:
print(list_tasks_response.status)
raise Exception(f'Get task failed, status: {list_tasks_response.status.description}')

print(list_tasks_response)

List Tasks Assigned to User

Get only the tasks assigned to a specific user for work.

#######################################################################
# In this section, we set the user authentication, app ID, and
# worker user ID. Change these strings to run your own example.
#######################################################################

USER_ID = 'YOUR_USER_ID_HERE'
# Your PAT (Personal Access Token) can be found in the Account's Security section
PAT = 'YOUR_PAT_HERE'
APP_ID = 'YOUR_APP_ID_HERE'
WORKER_USER_ID = 'WORKER_USER_ID_HERE'

##########################################################################
# YOU DO NOT NEED TO CHANGE ANYTHING BELOW THIS LINE TO RUN THIS EXAMPLE
##########################################################################

from clarifai_grpc.channel.clarifai_channel import ClarifaiChannel
from clarifai_grpc.grpc.api import resources_pb2, service_pb2, service_pb2_grpc
from clarifai_grpc.grpc.api.status import status_code_pb2

channel = ClarifaiChannel.get_grpc_channel()
stub = service_pb2_grpc.V2Stub(channel)

metadata = (('authorization', 'Key ' + PAT),)

userDataObject = resources_pb2.UserAppIDSet(user_id=USER_ID, app_id=APP_ID)

list_tasks_response = stub.ListTasks(
service_pb2.ListTasksRequest(
user_app_id=userDataObject, # The userDataObject is created in the overview and is required when using a PAT
worker_user_ids=[WORKER_USER_ID]
),
metadata=metadata
)

if list_tasks_response.status.code != status_code_pb2.SUCCESS:
print(list_tasks_response.status)
raise Exception(f'Get task failed, status: {list_tasks_response.status.description}')

print(list_tasks_response)

List Tasks Assigned to User for Review

Get only the tasks assigned to a specific user for review.

#######################################################################
# In this section, we set the user authentication, app ID, and
# review user ID. Change these strings to run your own example.
#######################################################################

USER_ID = 'YOUR_USER_ID_HERE'
# Your PAT (Personal Access Token) can be found in the Account's Security section
PAT = 'YOUR_PAT_HERE'
APP_ID = 'YOUR_APP_ID_HERE'
REVIEW_USER_ID = 'REVIEW_USER_ID_HERE'

##########################################################################
# YOU DO NOT NEED TO CHANGE ANYTHING BELOW THIS LINE TO RUN THIS EXAMPLE
##########################################################################

from clarifai_grpc.channel.clarifai_channel import ClarifaiChannel
from clarifai_grpc.grpc.api import resources_pb2, service_pb2, service_pb2_grpc
from clarifai_grpc.grpc.api.status import status_code_pb2

channel = ClarifaiChannel.get_grpc_channel()
stub = service_pb2_grpc.V2Stub(channel)

metadata = (('authorization', 'Key ' + PAT),)

userDataObject = resources_pb2.UserAppIDSet(user_id=USER_ID, app_id=APP_ID)

list_tasks_response = stub.ListTasks(
service_pb2.ListTasksRequest(
user_app_id=userDataObject, # The userDataObject is created in the overview and is required when using a PAT
review_user_ids=[REVIEW_USER_ID]
),
metadata=metadata
)

if list_tasks_response.status.code != status_code_pb2.SUCCESS:
print(list_tasks_response.status)
raise Exception(f'Get task failed, status: {list_tasks_response.status.description}')

print(list_tasks_response)

Update

Currently, we only support updating a task by providing all information at once.

Update Task

#################################################################################
# In this section, we set the user authentication, app ID, and details for
# updating a task. Change these strings to run your own example.
#################################################################################

USER_ID = 'YOUR_USER_ID_HERE'
# Your PAT (Personal Access Token) can be found in the Account's Security section
PAT = 'YOUR_PAT_HERE'
APP_ID = 'YOUR_APP_ID_HERE'
# Change these to update your own task
CONCEPT_ID = 'water'
TASK_ID = 'd0f2fa2b61234d1cb6b66983ea021b5b'
USER_ID_1 = 'USER_ID_1_HERE'
USER_ID_2 = 'USER_ID_2_HERE'
USER_ID_3 = 'USER_ID_3_HERE'
USER_ID_4 = 'USER_ID_4_HERE'

##########################################################################
# YOU DO NOT NEED TO CHANGE ANYTHING BELOW THIS LINE TO RUN THIS EXAMPLE
##########################################################################

from clarifai_grpc.channel.clarifai_channel import ClarifaiChannel
from clarifai_grpc.grpc.api import resources_pb2, service_pb2, service_pb2_grpc
from clarifai_grpc.grpc.api.status import status_code_pb2, status_pb2
from google.protobuf.struct_pb2 import Struct

channel = ClarifaiChannel.get_grpc_channel()
stub = service_pb2_grpc.V2Stub(channel)

params = Struct()
params.update({
USER_ID_1: 1,
USER_ID_2: 1,
USER_ID_3: 1
})

metadata = (('authorization', 'Key ' + PAT),)

userDataObject = resources_pb2.UserAppIDSet(user_id=USER_ID, app_id=APP_ID)

patch_tasks_response = stub.PatchTasks(
service_pb2.PatchTasksRequest(
user_app_id=userDataObject, # The userDataObject is created in the overview and is required when using a PAT
action="overwrite",
tasks=[
resources_pb2.Task(
id=TASK_ID,
type='CONCEPTS_CLASSIFICATION',
name='Annotate ' + CONCEPT_ID,
worker=resources_pb2.TaskWorker(
strategy='PARTITIONED',
users=[
resources_pb2.User(id=USER_ID_1),
resources_pb2.User(id=USER_ID_2),
resources_pb2.User(id=USER_ID_3)
],
partitioned_strategy_info=resources_pb2.TaskWorkerPartitionedStrategyInfo(
type='WEIGHTED',
workers_per_input=3,
weights=params
)
),
concepts=[
resources_pb2.TaskConcept(
concept=resources_pb2.Concept(id=CONCEPT_ID)
)
],
input_source=resources_pb2.TaskInputSource(type='ALL_INPUTS'),
sample_ms=1000,
review=resources_pb2.TaskReview(
strategy='CONSENSUS',
consensus_strategy_info=resources_pb2.TaskReviewConsensusStrategyInfo(
approval_threshold=2
),
users=[resources_pb2.User(id=USER_ID_4)]
),
status=status_pb2.Status(
code='TASK_DONE'
)
)
]
),
metadata=metadata
)

if patch_tasks_response.status.code != status_code_pb2.SUCCESS:
print(patch_tasks_response.status)
raise Exception('Patch tasks failed, status: ' + patch_tasks_response.status.description)

Delete

You can delete a task by specifying its ID. You can also delete multiple tasks by specifying a list of their IDs.

#################################################################################
# In this section, we set the user authentication, app ID, and task IDs.
# Change these strings to run your own example.
#################################################################################

USER_ID = 'YOUR_USER_ID_HERE'
# Your PAT (Personal Access Token) can be found in the Account's Security section
PAT = 'YOUR_PAT_HERE'
APP_ID = 'YOUR_APP_ID_HERE'
# Change these to delete your own tasks
TASK_ID_1 = 'd1f2f891da9844ccafed93a18574b027'
TASK_ID_2 = 'd1f2f891da9844ccafed93a18574b027'

##########################################################################
# YOU DO NOT NEED TO CHANGE ANYTHING BELOW THIS LINE TO RUN THIS EXAMPLE
##########################################################################

from clarifai_grpc.channel.clarifai_channel import ClarifaiChannel
from clarifai_grpc.grpc.api import resources_pb2, service_pb2, service_pb2_grpc
from clarifai_grpc.grpc.api.status import status_code_pb2

channel = ClarifaiChannel.get_grpc_channel()
stub = service_pb2_grpc.V2Stub(channel)

metadata = (('authorization', 'Key ' + PAT),)

userDataObject = resources_pb2.UserAppIDSet(user_id=USER_ID, app_id=APP_ID)

delete_tasks_response = stub.DeleteTasks(
service_pb2.DeleteTasksRequest(
user_app_id=userDataObject, # The userDataObject is created in the overview and is required when using a PAT
ids=[TASK_ID_1, TASK_ID_2]

),
metadata=metadata
)

if delete_tasks_response.status.code != status_code_pb2.SUCCESS:
print(delete_tasks_response.status)
raise Exception('Delete tasks failed, status: ' + delete_tasks_response.status.description)