Text Classifier

Learn how to use transfer learning to create custom text classifier models

---

A text classifier is a machine learning model designed to categorize text data into predefined classes or labels.

Let's demonstrate how you can create a custom text classifier model using the transfer learning technique.

Via the UI

Let's demonstrate how you can create a text classifier model that identifies whether an email message is a spam or not a spam.

Step 1: Create an App

Click here to learn how to create an application on the Clarifai platform.

note

When creating the application, choose the Text/Document option as the primary input type.

Step 2: Create a Dataset

Click here to learn how to create a dataset that will store the text inputs.

Step 3: Add and Annotate Inputs

Next, you'll need to upload data to the app you've created. This input data, labeled with specific concepts, will be used to train your model.

A concept is something that describes the content of your text input, similar to a "tag" or "keyword." The data in these concepts give the model something to "observe" about the keyword, and learn from.

For transfer learning, you can start with a small dataset. We recommend beginning with just 10 text inputs and gradually adding more as needed.

In this example, we'll use 5 examples of spam messages and 5 examples of non-spam messages sourced from this dataset. You can download the dataset and follow along with this documentation.

To upload inputs, select the Inputs option in the collapsible left sidebar. Next, click the Upload inputs button.

The small window that pops up allows you to upload your inputs.

You can use either of the following options to add text inputs to your app:

• Upload from a .csv file
• Add texts directly

Option 1: Upload From a .csv File

You can upload your text directly from a .csv file. This means you can work with your favorite spreadsheet software or text editor when preparing your data for upload.

Click here to learn how to upload your text data from a .csv file.

Option 2: Add Texts Directly

Select the Text option in the input uploader window. Then, use the input field to add each of the text inputs directly to your app.

For this illustration, let's use the second option.

• In the Select or add datasets search box, choose the dataset you previously created for storing your uploaded inputs.

• To label the inputs with the non-spam concept, click the plus (+) icon next to the Select or add concepts search box. Then, type the concept name in the search box. The new name will appear below the search box — click Add new concept to create it. The newly created concept will now be listed below the search box.

• To complete the process, click Upload inputs at the bottom of the pop-up window to upload your annotated input to the selected dataset.

• Similarly, upload the examples of spam messages to the previously created dataset and label them with the spam concept.

note

If you're adding text inputs directly, you'll need to repeat the process for each example message: select the dataset to which you want to add the input, choose the relevant concept, and then click the upload button.

tip

Click here to learn more about labeling inputs.

Step 4: Update Dataset

Next, navigate to your dataset's individual page and create a new version by clicking the New version button. This action bookmarks the current state of your dataset, allowing you to apply a specific version when training your custom text model.

Step 5: Choose a Model Type

Once you've uploaded the example text messages containing the concepts you want to train for, you're ready to create your custom model.

Begin by selecting the Models option from the collapsible left sidebar. On the following page, click the Add Model button in the upper-right corner.

In the pop-up window, select Build a Custom Model and click Continue to proceed.

You'll then be redirected to a page where you can choose the model type. In the Text section, select the Transfer Learn model type.

Step 6: Create a Model

On the ensuing page, provide a unique ID and click the Continue to Configure Model button to create your model.

Step 7: Set Up the Model

Next, provide the details for training your text classification model.

• Dataset selection — Choose the dataset you want to use for training. For this example, let's select the dataset we previously created, along with its specific version.
• Base embedding model — You can select the base model version to use for embeddings, which has to be one of the embed models in the app workflow. This allows you to specify the specific model in case the default workflow of your app has multiple embedding models present in it. For this example, let's use the default option.
• Define concepts — Specify the concepts you want the model to classify. Here, let's use spam and non-spam as the two concepts.
• Set concepts as mutually exclusive — Enable this option to indicate that there is no overlap between the defined concepts.
• Enrich dataset — If set to Automatic, this feature will enhance the model by including additional data from pre-built datasets with negative embeddings, improving accuracy. If set to Disabled, it will exclude these embeddings. Let's use the default Automatic setting for this example.
• Configure inference settings (optional) — Adjust inference settings as needed for your model. This step is optional.

After configuring the settings, click on Train Model to start the training process. Training typically takes a few seconds.

Step 8: Use Your Custom Model

You'll be redirected to the created model's page. Once the text classifier model is trained, you can put it to work, such as for making a prediction.

To test it:

• Click the blue (+) button labeled "Try your own text."
• A window will appear where you can input a text sample.
• The model will display prediction probabilities, showing how it classifies the text input.

Via the API

info

Before using the Python SDK, Node.js SDK, or any of our gRPC clients, ensure they are properly installed on your machine. Refer to their respective installation guides for instructions on how to install and initialize them.

Example 1

Let's demonstrate how you can create a text classifier model using our API.

Step 1: App Creation

Let's start by creating an app.

Python SDK

from clarifai.client.user import User
#replace your "user_id"
client = User(user_id="user_id")
app = client.create_app(app_id="demo_train", base_workflow="Universal")

Step 2: Dataset Upload

Next, let’s upload the dataset that will be used to train the model to the app.

You can find the dataset we used here.

Python SDK

#importing load_module_dataloader for calling the dataloader object in dataset.py in the local data folder
from clarifai.datasets.upload.utils import load_module_dataloader


# Construct the path to the dataset folder
CSV_PATH = os.path.join(os.getcwd().split('/models/model_train')[0],'datasets/upload/data/imdb.csv')


# Create a Clarifai dataset with the specified dataset_id 
dataset = app.create_dataset(dataset_id="text_dataset")
# Upload the dataset using the provided dataloader and get the upload status
dataset.upload_from_csv(csv_path=CSV_PATH,input_type='text',csv_type='raw', labels=True)

Step 3: Model Creation

Let's list all the available trainable model types in the Clarifai platform.

Python SDK

print(app.list_trainable_model_types())

Output

['visual-classifier',
 'visual-detector',
 'visual-segmenter',
 'visual-embedder',
 'clusterer',
 'text-classifier',
 'embedding-classifier',
 'text-to-text']

Next, let's select the embedding-classifier model type and use it to create a model.

Python

MODEL_ID = "model_text_embedder"
MODEL_TYPE_ID = "embedding-classifier"

# Create a model by passing the model name and model type as parameter
model = app.create_model(model_id=MODEL_ID, model_type_id=MODEL_TYPE_ID)

tip

Click here to learn how to patch your model.

Step 4: Set Up Model Parameters

You can customize the model parameters as needed before starting the training process.

Python SDK

# get the model params for default template
model_params = model.get_params()
concepts = [concept.id for concept in app.list_concepts()]
# update the concept field in model_params
model.update_params(dataset_id = 'text_dataset',concepts = ["id-pos","id-neg"])

Output

{'dataset_id': 'text_dataset',
 'dataset_version_id': '',
 'concepts': ['id-pos', 'id-neg'],
 'train_params': {'base_embed_model': None, 'enrich_dataset': 'Automatic'},
 'inference_params': {'min_value': 0.0}}

Step 5: Initiate Model Training

To initiate the model training process, call the model.train() method. The Clarifai API also provides features for monitoring training status and saving training logs to a local file.

note

If the training status code returns MODEL-TRAINED, it means the model has successfully completed training and is ready for use.

Python SDK

import time
#Starting the training
model_version_id = model.train()

#Checking the status of training
while True:
    status = model.training_status(version_id=model_version_id,training_logs=False)
    if status.code == 21106: #MODEL_TRAINING_FAILED
        print(status)
        break
    elif status.code == 21100: #MODEL_TRAINED
        print(status)
        break
    else:
        print("Current Status:",status)
        print("Waiting---")
        time.sleep(120)

Step 6: Model Prediction

After the model is trained and ready to use, you can run some predictions with it.

Python SDK

# Get the predictions
TEXT = b"This is a great place to work"
model_prediction = model.predict_by_bytes(TEXT, input_type="text")

# Get the output
print('Input: ',TEXT)
for concept in model_prediction.outputs[0].data.concepts:
    print(concept.id,':',round(concept.value,2))

Output

Input:  b'This is a great place to work'

id-neg : 0.72

id-pos : 0.28

Step 7: Model Evaluation

Let’s evaluate the model using both the training and test datasets. We’ll start by reviewing the evaluation metrics for the training dataset.

Python SDK

# Evaluate the model using the specified dataset ID 'text_dataset' and evaluation ID 'one'.
model.evaluate(dataset_id='text_dataset',eval_id='one',eval_info={"use_kfold":False})

# Retrieve the evaluation result for the evaluation ID 'one'.
result = model.get_eval_by_id(eval_id="one")

# Print the summary of the evaluation result.
print(result.summary)

Output

macro_avg_roc_auc: 1.0
macro_avg_f1_score: 1.0
macro_avg_precision: 1.0
macro_avg_recall: 1.0

Before evaluating the model on the test dataset, ensure it is uploaded using the data loader. Once uploaded, proceed with the evaluation.

Python SDK

#importing load_module_dataloader for calling the dataloader object in dataset.py in the local data folder
from clarifai.datasets.upload.utils import load_module_dataloader


# Construct the path to the dataset folder
CSV_PATH = os.path.join(os.getcwd().split('/models/model_train')[0],'datasets/upload/data/test_imdb.csv')


# Create a Clarifai dataset with the specified dataset_id
test_dataset = app.create_dataset(dataset_id="test_text_dataset")
# Upload the dataset using the provided dataloader and get the upload status
test_dataset.upload_from_csv(csv_path=CSV_PATH,input_type='text',csv_type='raw', labels=True)

# Evaluate the model using the specified test text dataset identified as 'test_text_dataset'
# and the evaluation identifier 'two'.
model.evaluate(dataset_id='test_text_dataset', eval_id='two')

# Retrieve the evaluation result with the identifier 'two'.
result = model.get_eval_by_id("two")

# Print the summary of the evaluation result.
print(result.summary)

Output

macro_avg_roc_auc: 1.0
macro_avg_f1_score: 1.0
macro_avg_precision: 1.0
macro_avg_recall: 1.0

Finally, to gain deeper insights into the model’s performance, use the EvalResultCompare method to compare results across multiple datasets.

Python SDK

from clarifai.utils.evaluation import EvalResultCompare

# Creating an instance of EvalResultCompare class with specified models and datasets
eval_result = EvalResultCompare(models=[model], datasets=[dataset, test_dataset])

# Printing a detailed summary of the evaluation result
print(eval_result.detailed_summary())

Output

(  Concept  Accuracy (ROC AUC)  Total Labeled  Total Predicted  True Positives  \
0  id-pos                 1.0             80               80              80   
0  id-neg                 1.0            120              120             120   
0  id-pos                 1.0             31               31              31   
0  id-neg                 1.0             40               40              40   

   False Negatives  False Positives  Recall  Precision   F1            Dataset  
0                0                0     1.0        1.0  1.0      text_dataset2  
0                0                0     1.0        1.0  1.0      text_dataset2  
0                0                0     1.0        1.0  1.0  test_text_dataset  
0                0                0     1.0        1.0  1.0  test_text_dataset  ,                Total Concept  Accuracy (ROC AUC)  Total Labeled  \
0      Dataset:text_dataset2                 1.0            200   
0  Dataset:test_text_dataset                 1.0             71   

   Total Predicted  True Positives  False Negatives  False Positives  Recall  \
0              200             200                0                0     1.0   
0               71              71                0                0     1.0   

   Precision   F1  
0        1.0  1.0  
0        1.0  1.0  )

Example 2

Let's demonstrate how you can create a custom text classifier using transfer learning.

Step 1: App Creation

Let's start by creating an app.

Step 2: Add a Batch of Texts

We'll now add several text inputs that we will later use as training data in our custom model. The idea is that we'll create a model which can differentiate between positive and negative sentences (in a grammatical sense).

We'll mark each input with one of the two concepts: positive or negative.

The texts can be added either directly (it's called raw) or from a URL.

Python (gRPC)

##########################################################################
# In this section, we set the user authentication, app ID, and negative 
# and positive texts. Change these strings to run your own example.
##########################################################################

USER_ID = 'YOUR_USER_ID_HERE'
# Your PAT (Personal Access Token) can be found in the Account's Security section
PAT = 'YOUR_PAT_HERE'
APP_ID = 'YOUR_APP_ID_HERE'
# Add your own batch of texts
positive_raw_texts = [
    "Marie is a published author.",
    "In three years, everyone will be happy.",
    "Nora Roberts is the most prolific romance writer the world has ever known.",
    "She has written more than 225 books.",
    "If you walk into Knoxville, you'll find a shop named Rala.",
    "There are more than 850 miles of hiking trails in the Great Smoky Mountains.",
    "Harrison Ford is 6'1\".",
    "According to Reader's Digest, in the original script of Return of The Jedi, Han Solo died.",
    "Kate travels to Doolin, Ireland every year for a writers' conference.",
    "Fort Stevens was decommissioned by the United States military in 1947.",
]
negative_text_urls = [
    "https://samples.clarifai.com/negative_sentence_1.txt",
    "https://samples.clarifai.com/negative_sentence_2.txt",
    "https://samples.clarifai.com/negative_sentence_3.txt",
    "https://samples.clarifai.com/negative_sentence_4.txt",
    "https://samples.clarifai.com/negative_sentence_5.txt",
    "https://samples.clarifai.com/negative_sentence_6.txt",
    "https://samples.clarifai.com/negative_sentence_7.txt",
    "https://samples.clarifai.com/negative_sentence_8.txt",
    "https://samples.clarifai.com/negative_sentence_9.txt",
    "https://samples.clarifai.com/negative_sentence_10.txt",
]

##########################################################################
# YOU DO NOT NEED TO CHANGE ANYTHING BELOW THIS LINE TO RUN THIS EXAMPLE
##########################################################################

from clarifai_grpc.channel.clarifai_channel import ClarifaiChannel
from clarifai_grpc.grpc.api import resources_pb2, service_pb2, service_pb2_grpc
from clarifai_grpc.grpc.api.status import status_code_pb2

channel = ClarifaiChannel.get_grpc_channel()
stub = service_pb2_grpc.V2Stub(channel)

metadata = (('authorization', 'Key ' + PAT),)

userDataObject = resources_pb2.UserAppIDSet(user_id=USER_ID, app_id=APP_ID)

post_inputs_response = stub.PostInputs(
    service_pb2.PostInputsRequest(
        user_app_id=userDataObject,
        inputs=[
            resources_pb2.Input(
                data=resources_pb2.Data(
                    text=resources_pb2.Text(raw=raw_text),
                    concepts=[resources_pb2.Concept(id="positive", value=1)]
                )
            )
            for raw_text in positive_raw_texts
        ] + [
            resources_pb2.Input(
                data=resources_pb2.Data(
                    text=resources_pb2.Text(
                        url=text_url,
                        allow_duplicate_url=True
                    ),
                    concepts=[resources_pb2.Concept(id="negative", value=1)]
                )
            )
            for text_url in negative_text_urls
        ]
    ),
    metadata=metadata
)

if post_inputs_response.status.code != status_code_pb2.SUCCESS:
    print(post_inputs_response.status)
    raise Exception("Failed response, status: " + post_inputs_response.status.description)

# Uncomment this line to see the structure and data of the response
#print(post_inputs_response)

Step 3: Wait for Inputs to Download

Let's now wait for all the inputs to download.

Python (gRPC)

###############################################################
# In this section, we set the user authentication and app ID. 
# Change these strings to run your own example.
###############################################################

USER_ID = 'YOUR_USER_ID_HERE'
# Your PAT (Personal Access Token) can be found in the Account's Security section
PAT = 'YOUR_PAT_HERE'
APP_ID = 'YOUR_APP_ID_HERE'

##########################################################################
# YOU DO NOT NEED TO CHANGE ANYTHING BELOW THIS LINE TO RUN THIS EXAMPLE
##########################################################################

from clarifai_grpc.channel.clarifai_channel import ClarifaiChannel
from clarifai_grpc.grpc.api import resources_pb2, service_pb2, service_pb2_grpc
from clarifai_grpc.grpc.api.status import status_code_pb2
import time

channel = ClarifaiChannel.get_grpc_channel()
stub = service_pb2_grpc.V2Stub(channel)

metadata = (('authorization', 'Key ' + PAT),)

userDataObject = resources_pb2.UserAppIDSet(user_id=USER_ID, app_id=APP_ID)

while True:
    list_inputs_response = stub.ListInputs(
        service_pb2.ListInputsRequest(
            user_app_id=userDataObject,
            page=1,
            per_page=100
        ),
        metadata=metadata
    )
    
    if list_inputs_response.status.code != status_code_pb2.SUCCESS:
        print(list_inputs_response.status)
        raise Exception("Failed response, status: " + list_inputs_response.status.description)

    for the_input in list_inputs_response.inputs:
        input_status_code = the_input.status.code
        if input_status_code == status_code_pb2.INPUT_DOWNLOAD_SUCCESS:
            continue
        elif input_status_code in (status_code_pb2.INPUT_DOWNLOAD_PENDING, status_code_pb2.INPUT_DOWNLOAD_IN_PROGRESS):
            print("Not all inputs have been downloaded yet. Checking again shortly.")
            break
        else:
            error_message = (
                    str(input_status_code) + " " +
                    the_input.status.description + " " +
                    the_input.status.details
            )
            raise Exception(
                f"Expected inputs to download, but got {error_message}. Full response: {list_inputs_response}"
            )
    else:
        # Once all inputs have been successfully downloaded, break the while True loop
        print("All inputs have been successfully downloaded.")
        break
    time.sleep(2)

Step 4: Create a Custom Model

Let's create a custom transfer learning model (also called an "embedding-classifier").

Python (gRPC)

################################################################################
# In this section, we set the user authentication, app ID, and the ID of the 
# model we want to create. Change these strings to run your own example.
################################################################################

USER_ID = 'YOUR_USER_ID_HERE'
# Your PAT (Personal Access Token) can be found in the Account's Security section
PAT = 'YOUR_PAT_HERE'
APP_ID = 'YOUR_APP_ID_HERE'
# Change this to create your own custom model
MODEL_ID = 'text-model-1' 

##########################################################################
# YOU DO NOT NEED TO CHANGE ANYTHING BELOW THIS LINE TO RUN THIS EXAMPLE
##########################################################################

from clarifai_grpc.channel.clarifai_channel import ClarifaiChannel
from clarifai_grpc.grpc.api import resources_pb2, service_pb2, service_pb2_grpc
from clarifai_grpc.grpc.api.status import status_code_pb2

channel = ClarifaiChannel.get_grpc_channel()
stub = service_pb2_grpc.V2Stub(channel)

metadata = (('authorization', 'Key ' + PAT),)

userDataObject = resources_pb2.UserAppIDSet(user_id=USER_ID, app_id=APP_ID)

post_models_response = stub.PostModels(
    service_pb2.PostModelsRequest(
        user_app_id=userDataObject,
        models=[
            resources_pb2.Model(
                id=MODEL_ID
            )
        ]
    ),
    metadata=metadata
)

if post_models_response.status.code != status_code_pb2.SUCCESS:
    print(post_models_response.status)
    raise Exception("Failed response, status: " + post_models_response.status.description)

# Uncomment this line to see the structure and data of the response
#print(post_models_response)

Step 5: Train the Model

Let's train the model using the positive and negative concepts.

All inputs (in our application) associated with these two concepts will be used as training data. This will make the model to learn from these inputs so that we can later predict new text examples.

Python (gRPC)

################################################################################
# In this section, we set the user authentication, app ID, and the ID of the 
# model we want to train. Change these strings to run your own example.
################################################################################

USER_ID = 'YOUR_USER_ID_HERE'
# Your PAT (Personal Access Token) can be found in the Account's Security section
PAT = 'YOUR_PAT_HERE'
APP_ID = 'YOUR_APP_ID_HERE'
# Change these to train your own custom model
MODEL_ID = 'text-model-1' 
CONCEPT_ID_1 = 'positive'
CONCEPT_ID_2 = 'negative'

##########################################################################
# YOU DO NOT NEED TO CHANGE ANYTHING BELOW THIS LINE TO RUN THIS EXAMPLE
##########################################################################

from clarifai_grpc.channel.clarifai_channel import ClarifaiChannel
from clarifai_grpc.grpc.api import resources_pb2, service_pb2, service_pb2_grpc
from clarifai_grpc.grpc.api.status import status_code_pb2

channel = ClarifaiChannel.get_grpc_channel()
stub = service_pb2_grpc.V2Stub(channel)

metadata = (('authorization', 'Key ' + PAT),)

userDataObject = resources_pb2.UserAppIDSet(user_id=USER_ID, app_id=APP_ID)

post_model_versions_response = stub.PostModelVersions(
    service_pb2.PostModelVersionsRequest(
        user_app_id=userDataObject,
        model_id=MODEL_ID,
        model_versions=[
            resources_pb2.ModelVersion(
                output_info=resources_pb2.OutputInfo(
                    data=resources_pb2.Data(
                        concepts=[
                            resources_pb2.Concept(id=CONCEPT_ID_1, value=1), # 1 means true, this concept is present
                            resources_pb2.Concept(id=CONCEPT_ID_2, value=1)
                            ] 
                        ),
                )
            )]
    ),
    metadata=metadata
)

if post_model_versions_response.status.code != status_code_pb2.SUCCESS:
    print(post_model_versions_response.status)
    raise Exception("Failed response, status: " + post_model_versions_response.status.description)

# Uncomment this line to see the structure and data of the response
#print(post_model_versions_response)

Step 6: Wait for Model Training to Complete

Let's wait for the model training to complete.

Each model training produces a new model version. Notice that on the bottom of the following code example, we placed the model version ID into its own variable.

We'll be using it later to specify which specific model version we want to use (since a model can have multiple versions).

Python (gRPC)

###############################################################################
# In this section, we set the user authentication, app ID, and the ID of the 
# model to wait for its training. Change these strings to run your own example.
################################################################################

USER_ID = 'YOUR_USER_ID_HERE'
# Your PAT (Personal Access Token) can be found in the Account's Security section
PAT = 'YOUR_PAT_HERE'
APP_ID = 'YOUR_APP_ID_HERE'
# Change this to wait for your own model's training
MODEL_ID = 'text-model-1' 

##########################################################################
# YOU DO NOT NEED TO CHANGE ANYTHING BELOW THIS LINE TO RUN THIS EXAMPLE
##########################################################################

from clarifai_grpc.channel.clarifai_channel import ClarifaiChannel
from clarifai_grpc.grpc.api import resources_pb2, service_pb2, service_pb2_grpc
from clarifai_grpc.grpc.api.status import status_code_pb2
import time

channel = ClarifaiChannel.get_grpc_channel()
stub = service_pb2_grpc.V2Stub(channel)

metadata = (('authorization', 'Key ' + PAT),)

userDataObject = resources_pb2.UserAppIDSet(user_id=USER_ID, app_id=APP_ID)

while True:
    get_model_response = stub.GetModel(
        service_pb2.GetModelRequest(
             user_app_id=userDataObject,
            model_id=MODEL_ID
        ),
        metadata=metadata
    )
    
    if get_model_response.status.code != status_code_pb2.SUCCESS:
        print(get_model_response.status)
        raise Exception("Failed response, status: " + get_model_response.status.description)

    version_status_code = get_model_response.model.model_version.status.code
    if version_status_code == status_code_pb2.MODEL_TRAINED:
        print("The model has been successfully trained.")
        break
    elif version_status_code in (status_code_pb2.MODEL_QUEUED_FOR_TRAINING, status_code_pb2.MODEL_TRAINING):
        print("The model hasn't been trained yet. Trying again shortly.")
        time.sleep(2)
    else:
        error_message = (
                str(get_model_response.status.code) + " " +
                get_model_response.status.description + " " +
                get_model_response.status.details
        )
        raise Exception(
            f"Expected model to train, but got {error_message}. Full response: {get_model_response}"
        )

model_version_id = get_model_response.model.model_version.id

Step 7: Predict on New Inputs

Let's now use the trained custom model to predict new text examples.

Python (gRPC)

###########################################################################
# In this section, we set the user authentication, app ID, model details, 
# and new input samples. Change these strings to run your own example.
###########################################################################

USER_ID = 'YOUR_USER_ID_HERE'
# Your PAT (Personal Access Token) can be found in the Account's Security section
PAT = 'YOUR_PAT_HERE'
APP_ID = 'YOUR_APP_ID_HERE'
# Change these to make your own predictions
MODEL_ID = 'text-model-1'
MODEL_VERSION_ID = '49219b5968624221ac488303dde55327' 
INPUT_TEXT = 'Butchart Gardens contains over 900 varieties of plants.'
INPUT_URL = 'https://samples.clarifai.com/negative_sentence_12.txt'

##########################################################################
# YOU DO NOT NEED TO CHANGE ANYTHING BELOW THIS LINE TO RUN THIS EXAMPLE
##########################################################################

from clarifai_grpc.channel.clarifai_channel import ClarifaiChannel
from clarifai_grpc.grpc.api import resources_pb2, service_pb2, service_pb2_grpc
from clarifai_grpc.grpc.api.status import status_code_pb2

channel = ClarifaiChannel.get_grpc_channel()
stub = service_pb2_grpc.V2Stub(channel)

metadata = (('authorization', 'Key ' + PAT),)

userDataObject = resources_pb2.UserAppIDSet(user_id=USER_ID, app_id=APP_ID)

post_model_outputs_response = stub.PostModelOutputs(
    service_pb2.PostModelOutputsRequest(
        user_app_id=userDataObject,
        model_id=MODEL_ID,
        # By default, the latest model version will be used, but it doesn't hurt to set it explicitly
        version_id=MODEL_VERSION_ID,
        inputs=[
            resources_pb2.Input(data=resources_pb2.Data(text=resources_pb2.Text(raw=INPUT_TEXT))),
            resources_pb2.Input(data=resources_pb2.Data(text=resources_pb2.Text(url=INPUT_URL))),
        ]
    ),
    metadata=metadata
)

if post_model_outputs_response.status.code != status_code_pb2.SUCCESS:
    print(post_model_outputs_response.status)
    raise Exception("Failed response, status: " + post_model_outputs_response.status.description)

for output in post_model_outputs_response.outputs:
    text_object = output.input.data.text
    val = text_object.raw if text_object.raw else text_object.url

    print(f"The following concepts were predicted for the input `{val}`:")
    for concept in output.data.concepts:
        print(f"\t{concept.name}: {concept.value:.2f}")

Text Output Example

The following concepts were predicted for the input `Butchart Gardens contains over 900 varieties of plants.`:
	positive: 0.83
	negative: 0.17
The following concepts were predicted for the input `https://samples.clarifai.com/negative_sentence_12.txt`:
	negative: 1.00
	positive: 0.00

Step 8: Start Model Evaluation

Let's now test the performance of the model by using model evaluation. Take note of the evaluation_id returned in the response, as you will need it for the next step.

tip

See the Evaluating Models section to learn more.

Python (gRPC)

###########################################################################
# In this section, we set the user authentication, app ID, model ID,
# and model version. Change these strings to run your own example.
###########################################################################

USER_ID = "YOUR_USER_ID_HERE"
# Your PAT (Personal Access Token) can be found in the Account's Security section
PAT = "YOUR_PAT_HERE"
APP_ID = "YOUR_APP_ID_HERE"
# Change these to make your own evaluations
MODEL_ID = "text-model-1"
MODEL_VERSION_ID = "3ad2c152232e46ebb16ed31f67dc54d8"

##########################################################################
# YOU DO NOT NEED TO CHANGE ANYTHING BELOW THIS LINE TO RUN THIS EXAMPLE
##########################################################################

from clarifai_grpc.channel.clarifai_channel import ClarifaiChannel
from clarifai_grpc.grpc.api import resources_pb2, service_pb2, service_pb2_grpc
from clarifai_grpc.grpc.api.status import status_code_pb2

channel = ClarifaiChannel.get_grpc_channel()
stub = service_pb2_grpc.V2Stub(channel)

metadata = (("authorization", "Key " + PAT),)

userDataObject = resources_pb2.UserAppIDSet(user_id=USER_ID, app_id=APP_ID)

post_model_evaluations = stub.PostEvaluations(
    service_pb2.PostEvaluationsRequest(
        user_app_id=userDataObject,
        eval_metrics=[
            resources_pb2.EvalMetrics(
                model=resources_pb2.Model(
                    app_id=APP_ID,
                    user_id=USER_ID,
                    id=MODEL_ID,
                    model_version=resources_pb2.ModelVersion(id=MODEL_VERSION_ID),
                )                
            )
        ],
    ),
    metadata=metadata,
)

if post_model_evaluations.status.code != status_code_pb2.SUCCESS:
    print(post_model_evaluations.status)
    raise Exception("Failed response, status: " + post_model_evaluations.status.description)

print(post_model_evaluations)

Raw Output Example

status {
  code: SUCCESS
  description: "Ok"
  req_id: "cbc9cbf2478ea008c68b74cc07126c05"
}
eval_metrics {
  status {
    code: MODEL_QUEUED_FOR_EVALUATION
    description: "Model is queued for evaluation."
  }
  user_id: "ei2leoz3s3iy"
  app_id: "text-search-app"
  id: "e223fa4ac14b4784b223cd31cc545f34"
}

Step 9: Wait for Model Evaluation Results

Model evaluation takes some time — depending on the amount of data the model has.

Let's wait for it to complete, and print all the results that it gives us.

Python (gRPC)

###########################################################################################
# In this section, we set the user authentication, app ID, and the model evaluation ID.
# Change these strings to run your own example.
##########################################################################################

USER_ID = "YOUR_USER_ID_HERE"
# Your PAT (Personal Access Token) can be found in the Account's Security section
PAT = "YOUR_PAT_HERE"
APP_ID = "text-search-app"
# Change these to wait for your own model's evaluation results
EVALUATION_ID = "e223fa4ac14b4784b223cd31cc545f34"

##########################################################################
# YOU DO NOT NEED TO CHANGE ANYTHING BELOW THIS LINE TO RUN THIS EXAMPLE
##########################################################################

from clarifai_grpc.channel.clarifai_channel import ClarifaiChannel
from clarifai_grpc.grpc.api import resources_pb2, service_pb2, service_pb2_grpc
from clarifai_grpc.grpc.api.status import status_code_pb2
import time

channel = ClarifaiChannel.get_grpc_channel()
stub = service_pb2_grpc.V2Stub(channel)

metadata = (("authorization", "Key " + PAT),)

userDataObject = resources_pb2.UserAppIDSet(user_id=USER_ID, app_id=APP_ID)

while True:
    get_evaluation_response = stub.GetEvaluation(
        service_pb2.GetEvaluationRequest(
            user_app_id=userDataObject,
            evaluation_id=EVALUATION_ID,
            fields=resources_pb2.FieldsValue(
                confusion_matrix=True,
                cooccurrence_matrix=True,
                label_counts=True,
                binary_metrics=True,
                test_set=True,
                metrics_by_area=True,
                metrics_by_class=True,
            ),
        ),
        metadata=metadata,
    )

    if get_evaluation_response.status.code != status_code_pb2.SUCCESS:
        print(get_evaluation_response.status)
        raise Exception("Get model version metrics failed: " + get_evaluation_response.status.description)

    print(get_evaluation_response)
    metrics_status_code = get_evaluation_response.eval_metrics.status.code
    if metrics_status_code == status_code_pb2.MODEL_EVALUATED:
        print("The model has been successfully evaluated.")
        break
    elif metrics_status_code in (
        status_code_pb2.MODEL_NOT_EVALUATED,
        status_code_pb2.MODEL_QUEUED_FOR_EVALUATION,
        status_code_pb2.MODEL_EVALUATING,
    ):
        print("The model hasn't been evaluated yet. Trying again shortly.")
        time.sleep(2)
    else:
        error_message = (
            str(get_evaluation_response.status.code) + " " + 
            get_evaluation_response.status.description + " " + 
            get_evaluation_response.status.details
        )
        raise Exception(
            f"Expected model to evaluate, but got {error_message}. Full response: {get_evaluation_response}"
        )

print("The model metrics response object:")
print(get_evaluation_response)

Raw Output Example

status {
  code: SUCCESS
  description: "Ok"
  req_id: "aa8517cf0d340d6afef484ddae938124"
}
eval_metrics {
  status {
    code: MODEL_EVALUATED
    description: "Model was successfully evaluated."
  }
  summary {
    macro_avg_roc_auc: 1.0
    macro_avg_f1_score: 0.8809523582458496
    macro_std_f1_score: 0.13677529990673065
    macro_avg_precision: 0.9375
    macro_avg_recall: 0.875
  }
  confusion_matrix {
    matrix {
      predicted: "positive"
      actual: "positive"
      value: 0.7497637867927551
      predicted_concept {
        id: "positive"
        name: "positive"
        value: 0.7497637867927551
        app_id: "text-search-app"
      }
      actual_concept {
        id: "positive"
        name: "positive"
        value: 1.0
        app_id: "text-search-app"
      }
    }
    matrix {
      predicted: "negative"
      actual: "positive"
      value: 0.2502362132072449
      predicted_concept {
        id: "negative"
        name: "negative"
        value: 0.2502362132072449
        app_id: "text-search-app"
      }
      actual_concept {
        id: "positive"
        name: "positive"
        value: 1.0
        app_id: "text-search-app"
      }
    }
    matrix {
      predicted: "positive"
      actual: "negative"
      value: 3.033356961168465e-07
      predicted_concept {
        id: "positive"
        name: "positive"
        value: 3.033356961168465e-07
        app_id: "text-search-app"
      }
      actual_concept {
        id: "negative"
        name: "negative"
        value: 1.0
        app_id: "text-search-app"
      }
    }
    matrix {
      predicted: "negative"
      actual: "negative"
      value: 0.9999997019767761
      predicted_concept {
        id: "negative"
        name: "negative"
        value: 0.9999997019767761
        app_id: "text-search-app"
      }
      actual_concept {
        id: "negative"
        name: "negative"
        value: 1.0
        app_id: "text-search-app"
      }
    }
    concept_ids: "positive"
    concept_ids: "negative"
  }
  cooccurrence_matrix {
    matrix {
      row: "positive"
      col: "positive"
      count: 10
    }
    matrix {
      row: "negative"
      col: "negative"
      count: 11
    }
    concept_ids: "positive"
    concept_ids: "negative"
  }
  label_counts {
    positive_label_counts {
      concept_name: "positive"
      count: 10
      concept {
        id: "positive"
        name: "positive"
        value: 1.0
        app_id: "text-search-app"
      }
    }
    positive_label_counts {
      concept_name: "negative"
      count: 11
      concept {
        id: "negative"
        name: "negative"
        value: 1.0
        app_id: "text-search-app"
      }
    }
  }
  binary_metrics {
    num_pos: 2
    num_neg: 2
    num_tot: 4
    roc_auc: 1.0
    f1: 0.8333333730697632
    concept {
      id: "positive"
      name: "positive"
      value: 1.0
      app_id: "text-search-app"
    }
    roc_curve {
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 1.0
      tpr: 0.0
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 1.0
      thresholds: 1.0
      thresholds: 0.9900000095367432
      thresholds: 0.9800000190734863
      thresholds: 0.9700000286102295
      thresholds: 0.9599999785423279
      thresholds: 0.949999988079071
      thresholds: 0.9399999976158142
      thresholds: 0.9300000071525574
      thresholds: 0.9200000166893005
      thresholds: 0.9100000262260437
      thresholds: 0.8999999761581421
      thresholds: 0.8899999856948853
      thresholds: 0.8799999952316284
      thresholds: 0.8700000047683716
      thresholds: 0.8600000143051147
      thresholds: 0.8500000238418579
      thresholds: 0.8399999737739563
      thresholds: 0.8299999833106995
      thresholds: 0.8199999928474426
      thresholds: 0.8100000023841858
      thresholds: 0.800000011920929
      thresholds: 0.7900000214576721
      thresholds: 0.7799999713897705
      thresholds: 0.7699999809265137
      thresholds: 0.7599999904632568
      thresholds: 0.75
      thresholds: 0.7400000095367432
      thresholds: 0.7300000190734863
      thresholds: 0.7200000286102295
      thresholds: 0.7099999785423279
      thresholds: 0.699999988079071
      thresholds: 0.6899999976158142
      thresholds: 0.6800000071525574
      thresholds: 0.6700000166893005
      thresholds: 0.6600000262260437
      thresholds: 0.6499999761581421
      thresholds: 0.6399999856948853
      thresholds: 0.6299999952316284
      thresholds: 0.6200000047683716
      thresholds: 0.6100000143051147
      thresholds: 0.6000000238418579
      thresholds: 0.5899999737739563
      thresholds: 0.5799999833106995
      thresholds: 0.5699999928474426
      thresholds: 0.5600000023841858
      thresholds: 0.550000011920929
      thresholds: 0.5400000214576721
      thresholds: 0.5299999713897705
      thresholds: 0.5199999809265137
      thresholds: 0.5099999904632568
      thresholds: 0.5
      thresholds: 0.49000000953674316
      thresholds: 0.47999998927116394
      thresholds: 0.4699999988079071
      thresholds: 0.46000000834465027
      thresholds: 0.44999998807907104
      thresholds: 0.4399999976158142
      thresholds: 0.4300000071525574
      thresholds: 0.41999998688697815
      thresholds: 0.4099999964237213
      thresholds: 0.4000000059604645
      thresholds: 0.38999998569488525
      thresholds: 0.3799999952316284
      thresholds: 0.3700000047683716
      thresholds: 0.36000001430511475
      thresholds: 0.3499999940395355
      thresholds: 0.3400000035762787
      thresholds: 0.33000001311302185
      thresholds: 0.3199999928474426
      thresholds: 0.3100000023841858
      thresholds: 0.30000001192092896
      thresholds: 0.28999999165534973
      thresholds: 0.2800000011920929
      thresholds: 0.27000001072883606
      thresholds: 0.25999999046325684
      thresholds: 0.25
      thresholds: 0.23999999463558197
      thresholds: 0.23000000417232513
      thresholds: 0.2199999988079071
      thresholds: 0.20999999344348907
      thresholds: 0.20000000298023224
      thresholds: 0.1899999976158142
      thresholds: 0.18000000715255737
      thresholds: 0.17000000178813934
      thresholds: 0.1599999964237213
      thresholds: 0.15000000596046448
      thresholds: 0.14000000059604645
      thresholds: 0.12999999523162842
      thresholds: 0.11999999731779099
      thresholds: 0.10999999940395355
      thresholds: 0.10000000149011612
      thresholds: 0.09000000357627869
      thresholds: 0.07999999821186066
      thresholds: 0.07000000029802322
      thresholds: 0.05999999865889549
      thresholds: 0.05000000074505806
      thresholds: 0.03999999910593033
      thresholds: 0.029999999329447746
      thresholds: 0.019999999552965164
      thresholds: 0.009999999776482582
      thresholds: 0.0
    }
    precision_recall_curve {
      recall: 1.0
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.0
      precision: 0.5
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      thresholds: 0.0
      thresholds: 0.009999999776482582
      thresholds: 0.019999999552965164
      thresholds: 0.029999999329447746
      thresholds: 0.03999999910593033
      thresholds: 0.05000000074505806
      thresholds: 0.05999999865889549
      thresholds: 0.07000000029802322
      thresholds: 0.07999999821186066
      thresholds: 0.09000000357627869
      thresholds: 0.10000000149011612
      thresholds: 0.10999999940395355
      thresholds: 0.11999999731779099
      thresholds: 0.12999999523162842
      thresholds: 0.14000000059604645
      thresholds: 0.15000000596046448
      thresholds: 0.1599999964237213
      thresholds: 0.17000000178813934
      thresholds: 0.18000000715255737
      thresholds: 0.1899999976158142
      thresholds: 0.20000000298023224
      thresholds: 0.20999999344348907
      thresholds: 0.2199999988079071
      thresholds: 0.23000000417232513
      thresholds: 0.23999999463558197
      thresholds: 0.25
      thresholds: 0.25999999046325684
      thresholds: 0.27000001072883606
      thresholds: 0.2800000011920929
      thresholds: 0.28999999165534973
      thresholds: 0.30000001192092896
      thresholds: 0.3100000023841858
      thresholds: 0.3199999928474426
      thresholds: 0.33000001311302185
      thresholds: 0.3400000035762787
      thresholds: 0.3499999940395355
      thresholds: 0.36000001430511475
      thresholds: 0.3700000047683716
      thresholds: 0.3799999952316284
      thresholds: 0.38999998569488525
      thresholds: 0.4000000059604645
      thresholds: 0.4099999964237213
      thresholds: 0.41999998688697815
      thresholds: 0.4300000071525574
      thresholds: 0.4399999976158142
      thresholds: 0.44999998807907104
      thresholds: 0.46000000834465027
      thresholds: 0.4699999988079071
      thresholds: 0.47999998927116394
      thresholds: 0.49000000953674316
      thresholds: 0.5
      thresholds: 0.5099999904632568
      thresholds: 0.5199999809265137
      thresholds: 0.5299999713897705
      thresholds: 0.5400000214576721
      thresholds: 0.550000011920929
      thresholds: 0.5600000023841858
      thresholds: 0.5699999928474426
      thresholds: 0.5799999833106995
      thresholds: 0.5899999737739563
      thresholds: 0.6000000238418579
      thresholds: 0.6100000143051147
      thresholds: 0.6200000047683716
      thresholds: 0.6299999952316284
      thresholds: 0.6399999856948853
      thresholds: 0.6499999761581421
      thresholds: 0.6600000262260437
      thresholds: 0.6700000166893005
      thresholds: 0.6800000071525574
      thresholds: 0.6899999976158142
      thresholds: 0.699999988079071
      thresholds: 0.7099999785423279
      thresholds: 0.7200000286102295
      thresholds: 0.7300000190734863
      thresholds: 0.7400000095367432
      thresholds: 0.75
      thresholds: 0.7599999904632568
      thresholds: 0.7699999809265137
      thresholds: 0.7799999713897705
      thresholds: 0.7900000214576721
      thresholds: 0.800000011920929
      thresholds: 0.8100000023841858
      thresholds: 0.8199999928474426
      thresholds: 0.8299999833106995
      thresholds: 0.8399999737739563
      thresholds: 0.8500000238418579
      thresholds: 0.8600000143051147
      thresholds: 0.8700000047683716
      thresholds: 0.8799999952316284
      thresholds: 0.8899999856948853
      thresholds: 0.8999999761581421
      thresholds: 0.9100000262260437
      thresholds: 0.9200000166893005
      thresholds: 0.9300000071525574
      thresholds: 0.9399999976158142
      thresholds: 0.949999988079071
      thresholds: 0.9599999785423279
      thresholds: 0.9700000286102295
      thresholds: 0.9800000190734863
      thresholds: 0.9900000095367432
      thresholds: 1.0
    }
  }
  binary_metrics {
    num_pos: 2
    num_neg: 2
    num_tot: 4
    roc_auc: 1.0
    f1: 0.9285714626312256
    concept {
      id: "negative"
      name: "negative"
      value: 1.0
      app_id: "text-search-app"
    }
    roc_curve {
      fpr: 0.0
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 1.0
      tpr: 0.5
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      thresholds: 1.0
      thresholds: 0.9900000095367432
      thresholds: 0.9800000190734863
      thresholds: 0.9700000286102295
      thresholds: 0.9599999785423279
      thresholds: 0.949999988079071
      thresholds: 0.9399999976158142
      thresholds: 0.9300000071525574
      thresholds: 0.9200000166893005
      thresholds: 0.9100000262260437
      thresholds: 0.8999999761581421
      thresholds: 0.8899999856948853
      thresholds: 0.8799999952316284
      thresholds: 0.8700000047683716
      thresholds: 0.8600000143051147
      thresholds: 0.8500000238418579
      thresholds: 0.8399999737739563
      thresholds: 0.8299999833106995
      thresholds: 0.8199999928474426
      thresholds: 0.8100000023841858
      thresholds: 0.800000011920929
      thresholds: 0.7900000214576721
      thresholds: 0.7799999713897705
      thresholds: 0.7699999809265137
      thresholds: 0.7599999904632568
      thresholds: 0.75
      thresholds: 0.7400000095367432
      thresholds: 0.7300000190734863
      thresholds: 0.7200000286102295
      thresholds: 0.7099999785423279
      thresholds: 0.699999988079071
      thresholds: 0.6899999976158142
      thresholds: 0.6800000071525574
      thresholds: 0.6700000166893005
      thresholds: 0.6600000262260437
      thresholds: 0.6499999761581421
      thresholds: 0.6399999856948853
      thresholds: 0.6299999952316284
      thresholds: 0.6200000047683716
      thresholds: 0.6100000143051147
      thresholds: 0.6000000238418579
      thresholds: 0.5899999737739563
      thresholds: 0.5799999833106995
      thresholds: 0.5699999928474426
      thresholds: 0.5600000023841858
      thresholds: 0.550000011920929
      thresholds: 0.5400000214576721
      thresholds: 0.5299999713897705
      thresholds: 0.5199999809265137
      thresholds: 0.5099999904632568
      thresholds: 0.5
      thresholds: 0.49000000953674316
      thresholds: 0.47999998927116394
      thresholds: 0.4699999988079071
      thresholds: 0.46000000834465027
      thresholds: 0.44999998807907104
      thresholds: 0.4399999976158142
      thresholds: 0.4300000071525574
      thresholds: 0.41999998688697815
      thresholds: 0.4099999964237213
      thresholds: 0.4000000059604645
      thresholds: 0.38999998569488525
      thresholds: 0.3799999952316284
      thresholds: 0.3700000047683716
      thresholds: 0.36000001430511475
      thresholds: 0.3499999940395355
      thresholds: 0.3400000035762787
      thresholds: 0.33000001311302185
      thresholds: 0.3199999928474426
      thresholds: 0.3100000023841858
      thresholds: 0.30000001192092896
      thresholds: 0.28999999165534973
      thresholds: 0.2800000011920929
      thresholds: 0.27000001072883606
      thresholds: 0.25999999046325684
      thresholds: 0.25
      thresholds: 0.23999999463558197
      thresholds: 0.23000000417232513
      thresholds: 0.2199999988079071
      thresholds: 0.20999999344348907
      thresholds: 0.20000000298023224
      thresholds: 0.1899999976158142
      thresholds: 0.18000000715255737
      thresholds: 0.17000000178813934
      thresholds: 0.1599999964237213
      thresholds: 0.15000000596046448
      thresholds: 0.14000000059604645
      thresholds: 0.12999999523162842
      thresholds: 0.11999999731779099
      thresholds: 0.10999999940395355
      thresholds: 0.10000000149011612
      thresholds: 0.09000000357627869
      thresholds: 0.07999999821186066
      thresholds: 0.07000000029802322
      thresholds: 0.05999999865889549
      thresholds: 0.05000000074505806
      thresholds: 0.03999999910593033
      thresholds: 0.029999999329447746
      thresholds: 0.019999999552965164
      thresholds: 0.009999999776482582
      thresholds: 0.0
    }
    precision_recall_curve {
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 0.5
      precision: 0.5
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 1.0
      thresholds: 0.0
      thresholds: 0.009999999776482582
      thresholds: 0.019999999552965164
      thresholds: 0.029999999329447746
      thresholds: 0.03999999910593033
      thresholds: 0.05000000074505806
      thresholds: 0.05999999865889549
      thresholds: 0.07000000029802322
      thresholds: 0.07999999821186066
      thresholds: 0.09000000357627869
      thresholds: 0.10000000149011612
      thresholds: 0.10999999940395355
      thresholds: 0.11999999731779099
      thresholds: 0.12999999523162842
      thresholds: 0.14000000059604645
      thresholds: 0.15000000596046448
      thresholds: 0.1599999964237213
      thresholds: 0.17000000178813934
      thresholds: 0.18000000715255737
      thresholds: 0.1899999976158142
      thresholds: 0.20000000298023224
      thresholds: 0.20999999344348907
      thresholds: 0.2199999988079071
      thresholds: 0.23000000417232513
      thresholds: 0.23999999463558197
      thresholds: 0.25
      thresholds: 0.25999999046325684
      thresholds: 0.27000001072883606
      thresholds: 0.2800000011920929
      thresholds: 0.28999999165534973
      thresholds: 0.30000001192092896
      thresholds: 0.3100000023841858
      thresholds: 0.3199999928474426
      thresholds: 0.33000001311302185
      thresholds: 0.3400000035762787
      thresholds: 0.3499999940395355
      thresholds: 0.36000001430511475
      thresholds: 0.3700000047683716
      thresholds: 0.3799999952316284
      thresholds: 0.38999998569488525
      thresholds: 0.4000000059604645
      thresholds: 0.4099999964237213
      thresholds: 0.41999998688697815
      thresholds: 0.4300000071525574
      thresholds: 0.4399999976158142
      thresholds: 0.44999998807907104
      thresholds: 0.46000000834465027
      thresholds: 0.4699999988079071
      thresholds: 0.47999998927116394
      thresholds: 0.49000000953674316
      thresholds: 0.5
      thresholds: 0.5099999904632568
      thresholds: 0.5199999809265137
      thresholds: 0.5299999713897705
      thresholds: 0.5400000214576721
      thresholds: 0.550000011920929
      thresholds: 0.5600000023841858
      thresholds: 0.5699999928474426
      thresholds: 0.5799999833106995
      thresholds: 0.5899999737739563
      thresholds: 0.6000000238418579
      thresholds: 0.6100000143051147
      thresholds: 0.6200000047683716
      thresholds: 0.6299999952316284
      thresholds: 0.6399999856948853
      thresholds: 0.6499999761581421
      thresholds: 0.6600000262260437
      thresholds: 0.6700000166893005
      thresholds: 0.6800000071525574
      thresholds: 0.6899999976158142
      thresholds: 0.699999988079071
      thresholds: 0.7099999785423279
      thresholds: 0.7200000286102295
      thresholds: 0.7300000190734863
      thresholds: 0.7400000095367432
      thresholds: 0.75
      thresholds: 0.7599999904632568
      thresholds: 0.7699999809265137
      thresholds: 0.7799999713897705
      thresholds: 0.7900000214576721
      thresholds: 0.800000011920929
      thresholds: 0.8100000023841858
      thresholds: 0.8199999928474426
      thresholds: 0.8299999833106995
      thresholds: 0.8399999737739563
      thresholds: 0.8500000238418579
      thresholds: 0.8600000143051147
      thresholds: 0.8700000047683716
      thresholds: 0.8799999952316284
      thresholds: 0.8899999856948853
      thresholds: 0.8999999761581421
      thresholds: 0.9100000262260437
      thresholds: 0.9200000166893005
      thresholds: 0.9300000071525574
      thresholds: 0.9399999976158142
      thresholds: 0.949999988079071
      thresholds: 0.9599999785423279
      thresholds: 0.9700000286102295
      thresholds: 0.9800000190734863
      thresholds: 0.9900000095367432
      thresholds: 1.0
    }
  }
  test_set {
    predicted_concepts {
      id: "positive"
      name: "positive"
      value: 0.9999601244926453
      app_id: "text-search-app"
    }
    predicted_concepts {
      id: "negative"
      name: "negative"
      value: 3.9902104617794976e-05
      app_id: "text-search-app"
    }
    ground_truth_concepts {
      id: "positive"
      name: "positive"
      value: 1.0
      app_id: "text-search-app"
    }
    input {
      id: "UyZACEDqN6WhAQOO"
      data {
        text {
          url: "https://data.clarifai.com/orig/users/alfrick/apps/text-search-app/inputs/text/e9dc85e0585a7a6d32da791e6cfa2c52"
          hosted {
            prefix: "https://data.clarifai.com"
            suffix: "users/alfrick/apps/text-search-app/inputs/text/e9dc85e0585a7a6d32da791e6cfa2c52"
            sizes: "orig"
            crossorigin: "use-credentials"
          }
          text_info {
            char_count: 507
            encoding: "UTF8"
          }
        }
      }
      created_at {
        seconds: 1687435506
        nanos: 969900000
      }
      modified_at {
        seconds: 1690525794
        nanos: 876378000
      }
      status {
        code: INPUT_DOWNLOAD_SUCCESS
        description: "Download complete"
      }
    }
  }
  test_set {
    predicted_concepts {
      id: "negative"
      name: "negative"
      value: 0.9999999403953552
      app_id: "text-search-app"
    }
    predicted_concepts {
      id: "positive"
      name: "positive"
      value: 6.925120743517255e-08
      app_id: "text-search-app"
    }
    ground_truth_concepts {
      id: "negative"
      name: "negative"
      value: 1.0
      app_id: "text-search-app"
    }
    input {
      id: "A2SsbMJrHqiAUmnr"
      data {
        text {
          url: "https://data.clarifai.com/orig/users/alfrick/apps/text-search-app/inputs/text/77fe0c1ff92bcb6d876ec8e551e9268f"
          hosted {
            prefix: "https://data.clarifai.com"
            suffix: "users/alfrick/apps/text-search-app/inputs/text/77fe0c1ff92bcb6d876ec8e551e9268f"
            sizes: "orig"
            crossorigin: "use-credentials"
          }
          text_info {
            char_count: 440
            encoding: "UTF8"
          }
        }
      }
      created_at {
        seconds: 1687436288
        nanos: 370201000
      }
      modified_at {
        seconds: 1690525794
        nanos: 876378000
      }
      status {
        code: INPUT_DOWNLOAD_SUCCESS
        description: "Download complete"
      }
    }
  }
  test_set {
    predicted_concepts {
      id: "negative"
      name: "negative"
      value: 0.9999988675117493
      app_id: "text-search-app"
    }
    predicted_concepts {
      id: "positive"
      name: "positive"
      value: 1.1424209560573217e-06
      app_id: "text-search-app"
    }
    ground_truth_concepts {
      id: "negative"
      name: "negative"
      value: 1.0
      app_id: "text-search-app"
    }
    input {
      id: "NnIBViXRFrXAGKkd"
      data {
        text {
          url: "https://data.clarifai.com/orig/users/alfrick/apps/text-search-app/inputs/text/65d33e371bbdd16cbcf7a2826ab0bcc1"
          hosted {
            prefix: "https://data.clarifai.com"
            suffix: "users/alfrick/apps/text-search-app/inputs/text/65d33e371bbdd16cbcf7a2826ab0bcc1"
            sizes: "orig"
            crossorigin: "use-credentials"
          }
          text_info {
            char_count: 359
            encoding: "UTF8"
          }
        }
      }
      created_at {
        seconds: 1687435598
        nanos: 758239000
      }
      modified_at {
        seconds: 1690525794
        nanos: 876378000
      }
      status {
        code: INPUT_DOWNLOAD_SUCCESS
        description: "Download complete"
      }
    }
  }
  test_set {
    predicted_concepts {
      id: "positive"
      name: "positive"
      value: 0.9999937415122986
      app_id: "text-search-app"
    }
    predicted_concepts {
      id: "negative"
      name: "negative"
      value: 6.254503659874899e-06
      app_id: "text-search-app"
    }
    ground_truth_concepts {
      id: "positive"
      name: "positive"
      value: 1.0
      app_id: "text-search-app"
    }
    input {
      id: "OK2cJALgpFQYafF2"
      data {
        text {
          url: "https://data.clarifai.com/orig/users/alfrick/apps/text-search-app/inputs/text/90fd9b5c2f9af536520add1a2da13db8"
          hosted {
            prefix: "https://data.clarifai.com"
            suffix: "users/alfrick/apps/text-search-app/inputs/text/90fd9b5c2f9af536520add1a2da13db8"
            sizes: "orig"
            crossorigin: "use-credentials"
          }
          text_info {
            char_count: 399
            encoding: "UTF8"
          }
        }
      }
      created_at {
        seconds: 1687435419
        nanos: 661211000
      }
      modified_at {
        seconds: 1690525794
        nanos: 810009000
      }
      status {
        code: INPUT_DOWNLOAD_SUCCESS
        description: "Download complete"
      }
    }
  }
  test_set {
    predicted_concepts {
      id: "positive"
      name: "positive"
      value: 0.9985570907592773
      app_id: "text-search-app"
    }
    predicted_concepts {
      id: "negative"
      name: "negative"
      value: 0.001442914130166173
      app_id: "text-search-app"
    }
    ground_truth_concepts {
      id: "positive"
      name: "positive"
      value: 1.0
      app_id: "text-search-app"
    }
    input {
      id: "YViTPPBzt3pTlmgY"
      data {
        text {
          url: "https://data.clarifai.com/orig/users/alfrick/apps/text-search-app/inputs/text/5bccdbc8c9f8fb9a673d61dc2e32e40b"
          hosted {
            prefix: "https://data.clarifai.com"
            suffix: "users/alfrick/apps/text-search-app/inputs/text/5bccdbc8c9f8fb9a673d61dc2e32e40b"
            sizes: "orig"
            crossorigin: "use-credentials"
          }
          text_info {
            char_count: 413
            encoding: "UTF8"
          }
        }
      }
      created_at {
        seconds: 1687436129
        nanos: 772735000
      }
      modified_at {
        seconds: 1690525794
        nanos: 876378000
      }
      status {
        code: INPUT_DOWNLOAD_SUCCESS
        description: "Download complete"
      }
    }
  }
  id: "e223fa4ac14b4784b223cd31cc545f34"
  eval_info {
    params {
      fields {
        key: "dataset_id"
        value {
          string_value: ""
        }
      }
      fields {
        key: "dataset_version_id"
        value {
          string_value: ""
        }
      }
      fields {
        key: "use_kfold"
        value {
          bool_value: true
        }
      }
    }
  }
  model {
    id: "text-model-1"
    app_id: "text-search-app"
    model_version {
      id: "3ad2c152232e46ebb16ed31f67dc54d8"
      created_at {
        seconds: 1693564041
        nanos: 515456000
      }
      status {
        code: MODEL_TRAINED
        description: "Model is trained and ready"
      }
      active_concept_count: 2
      metrics {
        status {
          code: MODEL_EVALUATED
          description: "Model was successfully evaluated."
        }
        summary {
          macro_avg_roc_auc: 1.0
          macro_avg_f1_score: 0.8809523582458496
          macro_std_f1_score: 0.13677529990673065
          macro_avg_precision: 0.9375
          macro_avg_recall: 0.875
        }
      }
      total_input_count: 21
      completed_at {
        seconds: 1693564044
        nanos: 915680000
      }
      visibility {
        gettable: PRIVATE
      }
      app_id: "text-search-app"
      user_id: "alfrick"
      metadata {
      }
      output_info {
        output_config {
        }
        message: "Show output_info with: GET /models/{model_id}/output_info"
        params {
          fields {
            key: "max_concepts"
            value {
              number_value: 20.0
            }
          }
          fields {
            key: "min_value"
            value {
              number_value: 0.0
            }
          }
          fields {
            key: "select_concepts"
            value {
              list_value {
              }
            }
          }
        }
      }
      input_info {
        base_embed_model {
          id: "multilingual-text-embedding"
          app_id: "main"
          model_version {
            id: "9b33adf15280465b857163ddaaacdcb1"
          }
          user_id: "clarifai"
          model_type_id: "text-embedder"
        }
      }
      train_info {
        params {
          fields {
            key: "dataset_id"
            value {
              string_value: ""
            }
          }
          fields {
            key: "dataset_version_id"
            value {
              string_value: ""
            }
          }
          fields {
            key: "enrich_dataset"
            value {
              string_value: "Automatic"
            }
          }
        }
      }
      import_info {
      }
    }
    user_id: "alfrick"
    model_type_id: "embedding-classifier"
  }
  user_id: "alfrick"
  app_id: "text-search-app"
}

The model has been successfully evaluated.
The model metrics response object:
status {
  code: SUCCESS
  description: "Ok"
  req_id: "aa8517cf0d340d6afef484ddae938124"
}
eval_metrics {
  status {
    code: MODEL_EVALUATED
    description: "Model was successfully evaluated."
  }
  summary {
    macro_avg_roc_auc: 1.0
    macro_avg_f1_score: 0.8809523582458496
    macro_std_f1_score: 0.13677529990673065
    macro_avg_precision: 0.9375
    macro_avg_recall: 0.875
  }
  confusion_matrix {
    matrix {
      predicted: "positive"
      actual: "positive"
      value: 0.7497637867927551
      predicted_concept {
        id: "positive"
        name: "positive"
        value: 0.7497637867927551
        app_id: "text-search-app"
      }
      actual_concept {
        id: "positive"
        name: "positive"
        value: 1.0
        app_id: "text-search-app"
      }
    }
    matrix {
      predicted: "negative"
      actual: "positive"
      value: 0.2502362132072449
      predicted_concept {
        id: "negative"
        name: "negative"
        value: 0.2502362132072449
        app_id: "text-search-app"
      }
      actual_concept {
        id: "positive"
        name: "positive"
        value: 1.0
        app_id: "text-search-app"
      }
    }
    matrix {
      predicted: "positive"
      actual: "negative"
      value: 3.033356961168465e-07
      predicted_concept {
        id: "positive"
        name: "positive"
        value: 3.033356961168465e-07
        app_id: "text-search-app"
      }
      actual_concept {
        id: "negative"
        name: "negative"
        value: 1.0
        app_id: "text-search-app"
      }
    }
    matrix {
      predicted: "negative"
      actual: "negative"
      value: 0.9999997019767761
      predicted_concept {
        id: "negative"
        name: "negative"
        value: 0.9999997019767761
        app_id: "text-search-app"
      }
      actual_concept {
        id: "negative"
        name: "negative"
        value: 1.0
        app_id: "text-search-app"
      }
    }
    concept_ids: "positive"
    concept_ids: "negative"
  }
  cooccurrence_matrix {
    matrix {
      row: "positive"
      col: "positive"
      count: 10
    }
    matrix {
      row: "negative"
      col: "negative"
      count: 11
    }
    concept_ids: "positive"
    concept_ids: "negative"
  }
  label_counts {
    positive_label_counts {
      concept_name: "positive"
      count: 10
      concept {
        id: "positive"
        name: "positive"
        value: 1.0
        app_id: "text-search-app"
      }
    }
    positive_label_counts {
      concept_name: "negative"
      count: 11
      concept {
        id: "negative"
        name: "negative"
        value: 1.0
        app_id: "text-search-app"
      }
    }
  }
  binary_metrics {
    num_pos: 2
    num_neg: 2
    num_tot: 4
    roc_auc: 1.0
    f1: 0.8333333730697632
    concept {
      id: "positive"
      name: "positive"
      value: 1.0
      app_id: "text-search-app"
    }
    roc_curve {
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 0.0
      fpr: 1.0
      tpr: 0.0
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 0.75
      tpr: 1.0
      thresholds: 1.0
      thresholds: 0.9900000095367432
      thresholds: 0.9800000190734863
      thresholds: 0.9700000286102295
      thresholds: 0.9599999785423279
      thresholds: 0.949999988079071
      thresholds: 0.9399999976158142
      thresholds: 0.9300000071525574
      thresholds: 0.9200000166893005
      thresholds: 0.9100000262260437
      thresholds: 0.8999999761581421
      thresholds: 0.8899999856948853
      thresholds: 0.8799999952316284
      thresholds: 0.8700000047683716
      thresholds: 0.8600000143051147
      thresholds: 0.8500000238418579
      thresholds: 0.8399999737739563
      thresholds: 0.8299999833106995
      thresholds: 0.8199999928474426
      thresholds: 0.8100000023841858
      thresholds: 0.800000011920929
      thresholds: 0.7900000214576721
      thresholds: 0.7799999713897705
      thresholds: 0.7699999809265137
      thresholds: 0.7599999904632568
      thresholds: 0.75
      thresholds: 0.7400000095367432
      thresholds: 0.7300000190734863
      thresholds: 0.7200000286102295
      thresholds: 0.7099999785423279
      thresholds: 0.699999988079071
      thresholds: 0.6899999976158142
      thresholds: 0.6800000071525574
      thresholds: 0.6700000166893005
      thresholds: 0.6600000262260437
      thresholds: 0.6499999761581421
      thresholds: 0.6399999856948853
      thresholds: 0.6299999952316284
      thresholds: 0.6200000047683716
      thresholds: 0.6100000143051147
      thresholds: 0.6000000238418579
      thresholds: 0.5899999737739563
      thresholds: 0.5799999833106995
      thresholds: 0.5699999928474426
      thresholds: 0.5600000023841858
      thresholds: 0.550000011920929
      thresholds: 0.5400000214576721
      thresholds: 0.5299999713897705
      thresholds: 0.5199999809265137
      thresholds: 0.5099999904632568
      thresholds: 0.5
      thresholds: 0.49000000953674316
      thresholds: 0.47999998927116394
      thresholds: 0.4699999988079071
      thresholds: 0.46000000834465027
      thresholds: 0.44999998807907104
      thresholds: 0.4399999976158142
      thresholds: 0.4300000071525574
      thresholds: 0.41999998688697815
      thresholds: 0.4099999964237213
      thresholds: 0.4000000059604645
      thresholds: 0.38999998569488525
      thresholds: 0.3799999952316284
      thresholds: 0.3700000047683716
      thresholds: 0.36000001430511475
      thresholds: 0.3499999940395355
      thresholds: 0.3400000035762787
      thresholds: 0.33000001311302185
      thresholds: 0.3199999928474426
      thresholds: 0.3100000023841858
      thresholds: 0.30000001192092896
      thresholds: 0.28999999165534973
      thresholds: 0.2800000011920929
      thresholds: 0.27000001072883606
      thresholds: 0.25999999046325684
      thresholds: 0.25
      thresholds: 0.23999999463558197
      thresholds: 0.23000000417232513
      thresholds: 0.2199999988079071
      thresholds: 0.20999999344348907
      thresholds: 0.20000000298023224
      thresholds: 0.1899999976158142
      thresholds: 0.18000000715255737
      thresholds: 0.17000000178813934
      thresholds: 0.1599999964237213
      thresholds: 0.15000000596046448
      thresholds: 0.14000000059604645
      thresholds: 0.12999999523162842
      thresholds: 0.11999999731779099
      thresholds: 0.10999999940395355
      thresholds: 0.10000000149011612
      thresholds: 0.09000000357627869
      thresholds: 0.07999999821186066
      thresholds: 0.07000000029802322
      thresholds: 0.05999999865889549
      thresholds: 0.05000000074505806
      thresholds: 0.03999999910593033
      thresholds: 0.029999999329447746
      thresholds: 0.019999999552965164
      thresholds: 0.009999999776482582
      thresholds: 0.0
    }
    precision_recall_curve {
      recall: 1.0
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.75
      recall: 0.0
      precision: 0.5
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      precision: 1.0
      thresholds: 0.0
      thresholds: 0.009999999776482582
      thresholds: 0.019999999552965164
      thresholds: 0.029999999329447746
      thresholds: 0.03999999910593033
      thresholds: 0.05000000074505806
      thresholds: 0.05999999865889549
      thresholds: 0.07000000029802322
      thresholds: 0.07999999821186066
      thresholds: 0.09000000357627869
      thresholds: 0.10000000149011612
      thresholds: 0.10999999940395355
      thresholds: 0.11999999731779099
      thresholds: 0.12999999523162842
      thresholds: 0.14000000059604645
      thresholds: 0.15000000596046448
      thresholds: 0.1599999964237213
      thresholds: 0.17000000178813934
      thresholds: 0.18000000715255737
      thresholds: 0.1899999976158142
      thresholds: 0.20000000298023224
      thresholds: 0.20999999344348907
      thresholds: 0.2199999988079071
      thresholds: 0.23000000417232513
      thresholds: 0.23999999463558197
      thresholds: 0.25
      thresholds: 0.25999999046325684
      thresholds: 0.27000001072883606
      thresholds: 0.2800000011920929
      thresholds: 0.28999999165534973
      thresholds: 0.30000001192092896
      thresholds: 0.3100000023841858
      thresholds: 0.3199999928474426
      thresholds: 0.33000001311302185
      thresholds: 0.3400000035762787
      thresholds: 0.3499999940395355
      thresholds: 0.36000001430511475
      thresholds: 0.3700000047683716
      thresholds: 0.3799999952316284
      thresholds: 0.38999998569488525
      thresholds: 0.4000000059604645
      thresholds: 0.4099999964237213
      thresholds: 0.41999998688697815
      thresholds: 0.4300000071525574
      thresholds: 0.4399999976158142
      thresholds: 0.44999998807907104
      thresholds: 0.46000000834465027
      thresholds: 0.4699999988079071
      thresholds: 0.47999998927116394
      thresholds: 0.49000000953674316
      thresholds: 0.5
      thresholds: 0.5099999904632568
      thresholds: 0.5199999809265137
      thresholds: 0.5299999713897705
      thresholds: 0.5400000214576721
      thresholds: 0.550000011920929
      thresholds: 0.5600000023841858
      thresholds: 0.5699999928474426
      thresholds: 0.5799999833106995
      thresholds: 0.5899999737739563
      thresholds: 0.6000000238418579
      thresholds: 0.6100000143051147
      thresholds: 0.6200000047683716
      thresholds: 0.6299999952316284
      thresholds: 0.6399999856948853
      thresholds: 0.6499999761581421
      thresholds: 0.6600000262260437
      thresholds: 0.6700000166893005
      thresholds: 0.6800000071525574
      thresholds: 0.6899999976158142
      thresholds: 0.699999988079071
      thresholds: 0.7099999785423279
      thresholds: 0.7200000286102295
      thresholds: 0.7300000190734863
      thresholds: 0.7400000095367432
      thresholds: 0.75
      thresholds: 0.7599999904632568
      thresholds: 0.7699999809265137
      thresholds: 0.7799999713897705
      thresholds: 0.7900000214576721
      thresholds: 0.800000011920929
      thresholds: 0.8100000023841858
      thresholds: 0.8199999928474426
      thresholds: 0.8299999833106995
      thresholds: 0.8399999737739563
      thresholds: 0.8500000238418579
      thresholds: 0.8600000143051147
      thresholds: 0.8700000047683716
      thresholds: 0.8799999952316284
      thresholds: 0.8899999856948853
      thresholds: 0.8999999761581421
      thresholds: 0.9100000262260437
      thresholds: 0.9200000166893005
      thresholds: 0.9300000071525574
      thresholds: 0.9399999976158142
      thresholds: 0.949999988079071
      thresholds: 0.9599999785423279
      thresholds: 0.9700000286102295
      thresholds: 0.9800000190734863
      thresholds: 0.9900000095367432
      thresholds: 1.0
    }
  }
  binary_metrics {
    num_pos: 2
    num_neg: 2
    num_tot: 4
    roc_auc: 1.0
    f1: 0.9285714626312256
    concept {
      id: "negative"
      name: "negative"
      value: 1.0
      app_id: "text-search-app"
    }
    roc_curve {
      fpr: 0.0
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 0.25
      fpr: 1.0
      tpr: 0.5
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      tpr: 1.0
      thresholds: 1.0
      thresholds: 0.9900000095367432
      thresholds: 0.9800000190734863
      thresholds: 0.9700000286102295
      thresholds: 0.9599999785423279
      thresholds: 0.949999988079071
      thresholds: 0.9399999976158142
      thresholds: 0.9300000071525574
      thresholds: 0.9200000166893005
      thresholds: 0.9100000262260437
      thresholds: 0.8999999761581421
      thresholds: 0.8899999856948853
      thresholds: 0.8799999952316284
      thresholds: 0.8700000047683716
      thresholds: 0.8600000143051147
      thresholds: 0.8500000238418579
      thresholds: 0.8399999737739563
      thresholds: 0.8299999833106995
      thresholds: 0.8199999928474426
      thresholds: 0.8100000023841858
      thresholds: 0.800000011920929
      thresholds: 0.7900000214576721
      thresholds: 0.7799999713897705
      thresholds: 0.7699999809265137
      thresholds: 0.7599999904632568
      thresholds: 0.75
      thresholds: 0.7400000095367432
      thresholds: 0.7300000190734863
      thresholds: 0.7200000286102295
      thresholds: 0.7099999785423279
      thresholds: 0.699999988079071
      thresholds: 0.6899999976158142
      thresholds: 0.6800000071525574
      thresholds: 0.6700000166893005
      thresholds: 0.6600000262260437
      thresholds: 0.6499999761581421
      thresholds: 0.6399999856948853
      thresholds: 0.6299999952316284
      thresholds: 0.6200000047683716
      thresholds: 0.6100000143051147
      thresholds: 0.6000000238418579
      thresholds: 0.5899999737739563
      thresholds: 0.5799999833106995
      thresholds: 0.5699999928474426
      thresholds: 0.5600000023841858
      thresholds: 0.550000011920929
      thresholds: 0.5400000214576721
      thresholds: 0.5299999713897705
      thresholds: 0.5199999809265137
      thresholds: 0.5099999904632568
      thresholds: 0.5
      thresholds: 0.49000000953674316
      thresholds: 0.47999998927116394
      thresholds: 0.4699999988079071
      thresholds: 0.46000000834465027
      thresholds: 0.44999998807907104
      thresholds: 0.4399999976158142
      thresholds: 0.4300000071525574
      thresholds: 0.41999998688697815
      thresholds: 0.4099999964237213
      thresholds: 0.4000000059604645
      thresholds: 0.38999998569488525
      thresholds: 0.3799999952316284
      thresholds: 0.3700000047683716
      thresholds: 0.36000001430511475
      thresholds: 0.3499999940395355
      thresholds: 0.3400000035762787
      thresholds: 0.33000001311302185
      thresholds: 0.3199999928474426
      thresholds: 0.3100000023841858
      thresholds: 0.30000001192092896
      thresholds: 0.28999999165534973
      thresholds: 0.2800000011920929
      thresholds: 0.27000001072883606
      thresholds: 0.25999999046325684
      thresholds: 0.25
      thresholds: 0.23999999463558197
      thresholds: 0.23000000417232513
      thresholds: 0.2199999988079071
      thresholds: 0.20999999344348907
      thresholds: 0.20000000298023224
      thresholds: 0.1899999976158142
      thresholds: 0.18000000715255737
      thresholds: 0.17000000178813934
      thresholds: 0.1599999964237213
      thresholds: 0.15000000596046448
      thresholds: 0.14000000059604645
      thresholds: 0.12999999523162842
      thresholds: 0.11999999731779099
      thresholds: 0.10999999940395355
      thresholds: 0.10000000149011612
      thresholds: 0.09000000357627869
      thresholds: 0.07999999821186066
      thresholds: 0.07000000029802322
      thresholds: 0.05999999865889549
      thresholds: 0.05000000074505806
      thresholds: 0.03999999910593033
      thresholds: 0.029999999329447746
      thresholds: 0.019999999552965164
      thresholds: 0.009999999776482582
      thresholds: 0.0
    }
    precision_recall_curve {
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 1.0
      recall: 0.5
      precision: 0.5
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 0.875
      precision: 1.0
      thresholds: 0.0
      thresholds: 0.009999999776482582
      thresholds: 0.019999999552965164
      thresholds: 0.029999999329447746
      thresholds: 0.03999999910593033
      thresholds: 0.05000000074505806
      thresholds: 0.05999999865889549
      thresholds: 0.07000000029802322
      thresholds: 0.07999999821186066
      thresholds: 0.09000000357627869
      thresholds: 0.10000000149011612
      thresholds: 0.10999999940395355
      thresholds: 0.11999999731779099
      thresholds: 0.12999999523162842
      thresholds: 0.14000000059604645
      thresholds: 0.15000000596046448
      thresholds: 0.1599999964237213
      thresholds: 0.17000000178813934
      thresholds: 0.18000000715255737
      thresholds: 0.1899999976158142
      thresholds: 0.20000000298023224
      thresholds: 0.20999999344348907
      thresholds: 0.2199999988079071
      thresholds: 0.23000000417232513
      thresholds: 0.23999999463558197
      thresholds: 0.25
      thresholds: 0.25999999046325684
      thresholds: 0.27000001072883606
      thresholds: 0.2800000011920929
      thresholds: 0.28999999165534973
      thresholds: 0.30000001192092896
      thresholds: 0.3100000023841858
      thresholds: 0.3199999928474426
      thresholds: 0.33000001311302185
      thresholds: 0.3400000035762787
      thresholds: 0.3499999940395355
      thresholds: 0.36000001430511475
      thresholds: 0.3700000047683716
      thresholds: 0.3799999952316284
      thresholds: 0.38999998569488525
      thresholds: 0.4000000059604645
      thresholds: 0.4099999964237213
      thresholds: 0.41999998688697815
      thresholds: 0.4300000071525574
      thresholds: 0.4399999976158142
      thresholds: 0.44999998807907104
      thresholds: 0.46000000834465027
      thresholds: 0.4699999988079071
      thresholds: 0.47999998927116394
      thresholds: 0.49000000953674316
      thresholds: 0.5
      thresholds: 0.5099999904632568
      thresholds: 0.5199999809265137
      thresholds: 0.5299999713897705
      thresholds: 0.5400000214576721
      thresholds: 0.550000011920929
      thresholds: 0.5600000023841858
      thresholds: 0.5699999928474426
      thresholds: 0.5799999833106995
      thresholds: 0.5899999737739563
      thresholds: 0.6000000238418579
      thresholds: 0.6100000143051147
      thresholds: 0.6200000047683716
      thresholds: 0.6299999952316284
      thresholds: 0.6399999856948853
      thresholds: 0.6499999761581421
      thresholds: 0.6600000262260437
      thresholds: 0.6700000166893005
      thresholds: 0.6800000071525574
      thresholds: 0.6899999976158142
      thresholds: 0.699999988079071
      thresholds: 0.7099999785423279
      thresholds: 0.7200000286102295
      thresholds: 0.7300000190734863
      thresholds: 0.7400000095367432
      thresholds: 0.75
      thresholds: 0.7599999904632568
      thresholds: 0.7699999809265137
      thresholds: 0.7799999713897705
      thresholds: 0.7900000214576721
      thresholds: 0.800000011920929
      thresholds: 0.8100000023841858
      thresholds: 0.8199999928474426
      thresholds: 0.8299999833106995
      thresholds: 0.8399999737739563
      thresholds: 0.8500000238418579
      thresholds: 0.8600000143051147
      thresholds: 0.8700000047683716
      thresholds: 0.8799999952316284
      thresholds: 0.8899999856948853
      thresholds: 0.8999999761581421
      thresholds: 0.9100000262260437
      thresholds: 0.9200000166893005
      thresholds: 0.9300000071525574
      thresholds: 0.9399999976158142
      thresholds: 0.949999988079071
      thresholds: 0.9599999785423279
      thresholds: 0.9700000286102295
      thresholds: 0.9800000190734863
      thresholds: 0.9900000095367432
      thresholds: 1.0
    }
  }
  test_set {
    predicted_concepts {
      id: "positive"
      name: "positive"
      value: 0.9999601244926453
      app_id: "text-search-app"
    }
    predicted_concepts {
      id: "negative"
      name: "negative"
      value: 3.9902104617794976e-05
      app_id: "text-search-app"
    }
    ground_truth_concepts {
      id: "positive"
      name: "positive"
      value: 1.0
      app_id: "text-search-app"
    }
    input {
      id: "UyZACEDqN6WhAQOO"
      data {
        text {
          url: "https://data.clarifai.com/orig/users/alfrick/apps/text-search-app/inputs/text/e9dc85e0585a7a6d32da791e6cfa2c52"
          hosted {
            prefix: "https://data.clarifai.com"
            suffix: "users/alfrick/apps/text-search-app/inputs/text/e9dc85e0585a7a6d32da791e6cfa2c52"
            sizes: "orig"
            crossorigin: "use-credentials"
          }
          text_info {
            char_count: 507
            encoding: "UTF8"
          }
        }
      }
      created_at {
        seconds: 1687435506
        nanos: 969900000
      }
      modified_at {
        seconds: 1690525794
        nanos: 876378000
      }
      status {
        code: INPUT_DOWNLOAD_SUCCESS
        description: "Download complete"
      }
    }
  }
  test_set {
    predicted_concepts {
      id: "negative"
      name: "negative"
      value: 0.9999999403953552
      app_id: "text-search-app"
    }
    predicted_concepts {
      id: "positive"
      name: "positive"
      value: 6.925120743517255e-08
      app_id: "text-search-app"
    }
    ground_truth_concepts {
      id: "negative"
      name: "negative"
      value: 1.0
      app_id: "text-search-app"
    }
    input {
      id: "A2SsbMJrHqiAUmnr"
      data {
        text {
          url: "https://data.clarifai.com/orig/users/alfrick/apps/text-search-app/inputs/text/77fe0c1ff92bcb6d876ec8e551e9268f"
          hosted {
            prefix: "https://data.clarifai.com"
            suffix: "users/alfrick/apps/text-search-app/inputs/text/77fe0c1ff92bcb6d876ec8e551e9268f"
            sizes: "orig"
            crossorigin: "use-credentials"
          }
          text_info {
            char_count: 440
            encoding: "UTF8"
          }
        }
      }
      created_at {
        seconds: 1687436288
        nanos: 370201000
      }
      modified_at {
        seconds: 1690525794
        nanos: 876378000
      }
      status {
        code: INPUT_DOWNLOAD_SUCCESS
        description: "Download complete"
      }
    }
  }
  test_set {
    predicted_concepts {
      id: "negative"
      name: "negative"
      value: 0.9999988675117493
      app_id: "text-search-app"
    }
    predicted_concepts {
      id: "positive"
      name: "positive"
      value: 1.1424209560573217e-06
      app_id: "text-search-app"
    }
    ground_truth_concepts {
      id: "negative"
      name: "negative"
      value: 1.0
      app_id: "text-search-app"
    }
    input {
      id: "NnIBViXRFrXAGKkd"
      data {
        text {
          url: "https://data.clarifai.com/orig/users/alfrick/apps/text-search-app/inputs/text/65d33e371bbdd16cbcf7a2826ab0bcc1"
          hosted {
            prefix: "https://data.clarifai.com"
            suffix: "users/alfrick/apps/text-search-app/inputs/text/65d33e371bbdd16cbcf7a2826ab0bcc1"
            sizes: "orig"
            crossorigin: "use-credentials"
          }
          text_info {
            char_count: 359
            encoding: "UTF8"
          }
        }
      }
      created_at {
        seconds: 1687435598
        nanos: 758239000
      }
      modified_at {
        seconds: 1690525794
        nanos: 876378000
      }
      status {
        code: INPUT_DOWNLOAD_SUCCESS
        description: "Download complete"
      }
    }
  }
  test_set {
    predicted_concepts {
      id: "positive"
      name: "positive"
      value: 0.9999937415122986
      app_id: "text-search-app"
    }
    predicted_concepts {
      id: "negative"
      name: "negative"
      value: 6.254503659874899e-06
      app_id: "text-search-app"
    }
    ground_truth_concepts {
      id: "positive"
      name: "positive"
      value: 1.0
      app_id: "text-search-app"
    }
    input {
      id: "OK2cJALgpFQYafF2"
      data {
        text {
          url: "https://data.clarifai.com/orig/users/alfrick/apps/text-search-app/inputs/text/90fd9b5c2f9af536520add1a2da13db8"
          hosted {
            prefix: "https://data.clarifai.com"
            suffix: "users/alfrick/apps/text-search-app/inputs/text/90fd9b5c2f9af536520add1a2da13db8"
            sizes: "orig"
            crossorigin: "use-credentials"
          }
          text_info {
            char_count: 399
            encoding: "UTF8"
          }
        }
      }
      created_at {
        seconds: 1687435419
        nanos: 661211000
      }
      modified_at {
        seconds: 1690525794
        nanos: 810009000
      }
      status {
        code: INPUT_DOWNLOAD_SUCCESS
        description: "Download complete"
      }
    }
  }
  test_set {
    predicted_concepts {
      id: "positive"
      name: "positive"
      value: 0.9985570907592773
      app_id: "text-search-app"
    }
    predicted_concepts {
      id: "negative"
      name: "negative"
      value: 0.001442914130166173
      app_id: "text-search-app"
    }
    ground_truth_concepts {
      id: "positive"
      name: "positive"
      value: 1.0
      app_id: "text-search-app"
    }
    input {
      id: "YViTPPBzt3pTlmgY"
      data {
        text {
          url: "https://data.clarifai.com/orig/users/alfrick/apps/text-search-app/inputs/text/5bccdbc8c9f8fb9a673d61dc2e32e40b"
          hosted {
            prefix: "https://data.clarifai.com"
            suffix: "users/alfrick/apps/text-search-app/inputs/text/5bccdbc8c9f8fb9a673d61dc2e32e40b"
            sizes: "orig"
            crossorigin: "use-credentials"
          }
          text_info {
            char_count: 413
            encoding: "UTF8"
          }
        }
      }
      created_at {
        seconds: 1687436129
        nanos: 772735000
      }
      modified_at {
        seconds: 1690525794
        nanos: 876378000
      }
      status {
        code: INPUT_DOWNLOAD_SUCCESS
        description: "Download complete"
      }
    }
  }
  id: "e223fa4ac14b4784b223cd31cc545f34"
  eval_info {
    params {
      fields {
        key: "dataset_id"
        value {
          string_value: ""
        }
      }
      fields {
        key: "dataset_version_id"
        value {
          string_value: ""
        }
      }
      fields {
        key: "use_kfold"
        value {
          bool_value: true
        }
      }
    }
  }
  model {
    id: "text-model-1"
    app_id: "text-search-app"
    model_version {
      id: "3ad2c152232e46ebb16ed31f67dc54d8"
      created_at {
        seconds: 1693564041
        nanos: 515456000
      }
      status {
        code: MODEL_TRAINED
        description: "Model is trained and ready"
      }
      active_concept_count: 2
      metrics {
        status {
          code: MODEL_EVALUATED
          description: "Model was successfully evaluated."
        }
        summary {
          macro_avg_roc_auc: 1.0
          macro_avg_f1_score: 0.8809523582458496
          macro_std_f1_score: 0.13677529990673065
          macro_avg_precision: 0.9375
          macro_avg_recall: 0.875
        }
      }
      total_input_count: 21
      completed_at {
        seconds: 1693564044
        nanos: 915680000
      }
      visibility {
        gettable: PRIVATE
      }
      app_id: "text-search-app"
      user_id: "alfrick"
      metadata {
      }
      output_info {
        output_config {
        }
        message: "Show output_info with: GET /models/{model_id}/output_info"
        params {
          fields {
            key: "max_concepts"
            value {
              number_value: 20.0
            }
          }
          fields {
            key: "min_value"
            value {
              number_value: 0.0
            }
          }
          fields {
            key: "select_concepts"
            value {
              list_value {
              }
            }
          }
        }
      }
      input_info {
        base_embed_model {
          id: "multilingual-text-embedding"
          app_id: "main"
          model_version {
            id: "9b33adf15280465b857163ddaaacdcb1"
          }
          user_id: "clarifai"
          model_type_id: "text-embedder"
        }
      }
      train_info {
        params {
          fields {
            key: "dataset_id"
            value {
              string_value: ""
            }
          }
          fields {
            key: "dataset_version_id"
            value {
              string_value: ""
            }
          }
          fields {
            key: "enrich_dataset"
            value {
              string_value: "Automatic"
            }
          }
        }
      }
      import_info {
      }
    }
    user_id: "ei2leoz3s3iy"
    model_type_id: "embedding-classifier"
  }
  user_id: "ei2leoz3s3iy"
  app_id: "text-search-app"
}