Building Workflow Graphs
Learn how to build workflows using Clarifai SDKs
Unlock the potential of efficient data processing pipelines tailored precisely to your unique requirements with this empowering feature. As you embark on the journey of constructing workflow graphs, you gain unprecedented flexibility to seamlessly connect and orchestrate a diverse array of models, operators, and processing steps. Whether your focus is on integrating sophisticated image classification, precise object detection, or custom operators, our SDK offers an intuitive and user-friendly interface. Experience the freedom to effortlessly create, modify, and optimize workflow graphs, empowering you to design and implement highly specialized data processing pipelines that align seamlessly with your specific needs.
Create Workflow
Designing and implementing workflows is a crucial aspect of building a robust and flexible inference pipeline. Workflows offer a modular architecture that serves as the foundation for both sophisticated machine learning ensemble modeling and seamless integration of business logic. These workflows empower you to streamline your processes, enhance model performance, and effectively handle diverse tasks in your machine learning endeavors. Workflow can be created with the SDK using a YAML specification.
To know more about workflows visit this link.
- Python
from clarifai.client.user import User
# Your PAT (Personal Access Token) can be found in the Account's Security section
# Specify the correct user_id/app_id pairings
# Since you're making inferences outside your app's scope
USER_ID = "user_id"
APP_ID = "app_id"
app = User(user_id=USER_ID, pat="YOUR_PAT").create_app(
app_id=APP_ID, base_workflow="Empty"
)
# create a yaml file specifying the workflow structure
# eg:
"""configs/prompter_llm.yml
workflow:
id: wf-prompter-llm
nodes:
- id: prompter
model:
model_id: prompter
model_type_id: prompter
description: 'Prompter Model'
output_info:
params:
prompt_template: 'Classify sentiment between postive and negative for the text {data.text.raw}'
- id: llm
model:
user_id: mistralai
model_id: mistral-7B-Instruct
app_id: completion
node_inputs:
- node_id: prompter
"""
# create the workflow
prompter_llm_workflow = app.create_workflow(config_filepath="configs/prompter_llm.yml")
Output
Patch Workflow
After creating a workflow, you can perform patch operations on it by merging, removing, or overwriting data. By default, all actions support overwriting, with specific behaviors for lists of objects.
- The
merge
action updates an existingkey:value
pair withkey:new_value
or appends to an existing list. For dictionaries, it merges objects that share a matchingid
field. - The
remove
action is only used to delete the workflow's cover image on the platform UI. - The
overwrite
action fully replaces an existing object with a new one.
Below is an example of performing patch operations on a workflow, where the YAML configuration is updated, and changes are made to its description, notes, and image URL.
- Python
from clarifai.client.app import App
app = App(app_id="YOUR_APP_ID_HERE", user_id="YOUR_USER_ID_HERE", pat="YOUR_PAT_HERE")
# Update an existing workflow to use a different yml configuration like "demographics"
WORKFLOW_ID = general_workflow.kwargs['id']
app.patch_workflow(workflow_id=WORKFLOW_ID, action="merge", config_filepath="configs/demographics.yml")
# Update workflow details, such as description, notes, etc
app.patch_workflow(workflow_id=WORKFLOW_ID, action="merge", description="description", notes="notes", image_url="https://samples.clarifai.com/metro-north.jpg")
# Remove the workflow's image by specifying the 'remove' action
app.patch_workflow(workflow_id=WORKFLOW_ID, action="remove", image_url="https://samples.clarifai.com/metro-north.jpg")
Examples for YAML-based Workflows
ASR Sentiment
Automatic Speech Recognition (ASR) sentiment analysis refers to the process of analyzing the emotional tone or sentiment expressed in spoken language using ASR model.
workflow:
id: asr-sentiment
nodes:
- id: audio-speech-recognition
model:
model_id: asr-wav2vec2-large-robust-ft-swbd-300h-english
user_id: facebook
app_id: asr
- id: text-sentiment-classification
model:
model_id: sentiment-analysis-twitter-roberta-base
user_id: erfan
app_id: text-classification
node_inputs:
- node_id: audio-speech-recognition
Demographics
This is a Multi-model workflow that detects, crops, and recognizes demographic characteristics of faces. Visually classifies age, gender, and multi-culture.
workflow:
id: Demographics
nodes:
- id: detect-concept
model:
model_id: face-detection
model_version_id: 45fb9a671625463fa646c3523a3087d5
- id: image-crop
model:
model_id: margin-110-image-crop
model_version_id: b9987421b40a46649566826ef9325303
node_inputs:
- node_id: detect-concept
- id: demographics-race
model:
model_id: ethnicity-demographics-recognition
model_version_id: b2897edbda314615856039fb0c489796
node_inputs:
- node_id: image-crop
- id: demographics-gender
model:
model_id: gender-demographics-recognition
model_version_id: ff83d5baac004aafbe6b372ffa6f8227
node_inputs:
- node_id: image-crop
- id: demographics-age
model:
model_id: age-demographics-recognition
model_version_id: fb9f10339ac14e23b8e960e74984401b
node_inputs:
- node_id: image-crop
Face Search
A workflow that combines detection, recognition, and embedding to generate face landmarks and enable visual search using detected faces's embeddings.
workflow:
id: Face-Search
nodes:
- id: face-detect
model:
model_id: face-detection
model_version_id: fe995da8cb73490f8556416ecf25cea3
- id: crop
model:
model_id: margin-100-image-crop
model_version_id: 0af5cd8ad40e43ef92154e4f4bc76bef
node_inputs:
- node_id: face-detect
- id: face-landmarks
model:
model_id: face-landmarks
model_version_id: 98ace9ca45e64339be94b06011557e2a
node_inputs:
- node_id: crop
- id: face-alignment
model:
model_id: landmarks-align
model_version_id: 4bc8b83a327247829ec638c78cde5f8b
node_inputs:
- node_id: face-landmarks
- id: face-embed
model:
model_id: face-identification-transfer-learn
model_version_id: fc3b8814fbe54533a3d80a1896dc9884
node_inputs:
- node_id: face-alignment
- id: face-cluster
model:
model_id: face-clustering
model_version_id: 621d74074a5443d7ad9dc1503fba9ff0
node_inputs:
- node_id: face-embed
Face Sentiment
Multi-model workflow that combines face detection and sentiment classification of 7 concepts: anger, disgust, fear, neutral, happiness, sadness, contempt.
workflow:
id: Face-Sentiment
nodes:
- id: face-det
model:
model_id: face-detection
model_version_id: 6dc7e46bc9124c5c8824be4822abe105
- id: margin-110
model:
model_id: margin-110-image-crop
model_version_id: b9987421b40a46649566826ef9325303
node_inputs:
- node_id: face-det
- id: face-sentiment
model:
model_id: face-sentiment-recognition
model_version_id: a5d7776f0c064a41b48c3ce039049f65
node_inputs:
- node_id: margin-110
General
A general image detection workflow that detects a variety of common objects, and enable visual search using general embeddings on detected regions.
workflow:
id: General
nodes:
- id: general-v1.5-concept
model:
model_id: aaa03c23b3724a16a56b629203edc62c
model_version_id: aa7f35c01e0642fda5cf400f543e7c40
- id: general-v1.5-embed
model:
model_id: bbb5f41425b8468d9b7a554ff10f8581
model_version_id: bb186755eda04f9cbb6fe32e816be104
- id: general-v1.5-cluster
model:
model_id: cccbe437d6e54e2bb911c6aa292fb072
model_version_id: cc2074cff6dc4c02b6f4e1b8606dcb54
node_inputs:
- node_id: general-v1.5-embed
Language Aware OCR
A workflows that performs OCR in different languages.
workflow:
id: wf-ocr
nodes:
- id: ocr-workflow
model:
model_id: language-aware-multilingual-ocr-multiplex
- id: text-aggregator
model:
model_id: text-aggregation
model_type_id: text-aggregation-operator
output_info:
params:
avg_word_width_window_factor: 2.0
avg_word_height_window_factor: 1.0
node_inputs:
- node_id: ocr-workflow
- id: language-id-operator
model:
model_id: language-id
model_type_id: language-id-operator
output_info:
params:
library: "fasttext"
topk: 1
threshold: 0.1
lowercase: true
node_inputs:
- node_id: text-aggregator
Prompter LLM
A workflow that uses a prompt template with LLM.
workflow:
id: wf-prompter-llm
nodes:
- id: prompter
model:
model_id: prompter
model_type_id: prompter
description: 'Prompter Model'
output_info:
params:
prompt_template: 'Classify sentiment between postive and negative for the text {data.text.raw}'
- id: llm
model:
user_id: mistralai
model_id: mistral-7B-Instruct
app_id: completion
node_inputs:
- node_id: prompter
RAG Prompter LLM
This workflow uses LLM with a RAG prompter template.
workflow:
id: wf-prompter-llm
nodes:
- id: rag-prompter
model:
model_id: rag-prompter
model_type_id: rag-prompter
description: 'RAG Prompter Model'
- id: llm
model:
user_id: mistralai
model_id: mistral-7B-Instruct
app_id: completion
node_inputs:
- node_id: rag-prompter
Visit this link to view some YAML examples.
List workflows
The Clarifai SDKs provides an API that empowers users to effortlessly retrieve a comprehensive list of all workflows accessible within an app. This functionality enables seamless exploration and management of workflows, facilitating a streamlined experience for developers to interact with and leverage the power of Clarifai's workflow capabilities.
To know more about workflows visit this link.
- Python
from clarifai.client.app import App
# Your PAT (Personal Access Token) can be found in the Account's Security section
app = App(app_id="APP_ID", user_id="USER_ID", pat="YOUR_PAT")
for workflow in app.list_workflows(page_no=1,per_page=7):
print("Workflow ID: ", workflow.id)
Output
Workflow ID: object_track
Workflow ID: video_track
Workflow ID: multimodal_to_text
Workflow ID: text_to_audio
Workflow ID: upscale_workflow
Workflow ID: image_generation
Workflow ID: text_generation
Delete workflows
The Clarifai SDKs empowers users to seamlessly delete workflows through the API by specifying the unique identifier of the workflow. This functionality provides a straightforward method for users to manage and remove workflows as needed within their applications or projects. By utilizing the provided workflow id parameter, developers can efficiently interact with the Clarifai platform to streamline workflow management processes.
To know more about workflows visit this link.
Be certain that you want to delete a particular workflow as the operation cannot be undone.
- Python
from clarifai.client.app import App
# Your PAT (Personal Access Token) can be found in the Account's Security section
app = App(app_id="APP_ID", user_id="USER_ID", pat="YOUR_PAT")
# Delete the workflow within an application
app.delete_workflow(workflow_id="workflow-id")
Output
2024-01-18 16:34:46 INFO clarifai.client.app: app.py:653
Workflow Deleted
code: SUCCESS
description: "Ok"
req_id: "a979f35e9c826bb9046f4d92879c6b7c"
Export workflow
The Clarifai SDKs provides a powerful feature for initiating or swiftly modifying existing Clarifai community workflows through an intuitive YAML configuration. With the export functionality, users can effortlessly download a YAML file representing the entire workflow. This file serves as a local copy, allowing seamless edits and providing the flexibility to create and manage new workflows with ease.
To know more about workflows visit this link.
- Python
from clarifai.client.workflow import Workflow
# Your PAT (Personal Access Token) can be found in the Account's Security section
workflow_url = "https://clarifai.com/clarifai/main/workflows/Demographics"
demographics_workflow = Workflow(
url=workflow_url , pat="YOUR_PAT"
)
demographics_workflow.export("demographics_workflow.yml")
"""
Now the parameters of each model can be changed and new workflow can be easily created by app.create_workflow().
Here we change the margin parameter of the image cropper node to be 1.5
- id: image-crop
model:
model_id: margin-100-image-crop-custom
model_type_id: image-crop
description: Custom crop model
output_info:
params:
margin: 1.5
"""