Prompt Scoring

On this page you'll find the following:

• Get Started
• Rankings, Scores, and Weighted Scores
• Scoring Report
• Auto-Improve Score
• Score Prompt with Different Criteria
• Score Prompt with Different Models

Get Started

Scoring prompts provides an objective way to measure prompt quality based on a specific set of criteria. To perform prompt scoring, go to a version you want to score and click its Score tab. You'll see the Default list of criteria is preselected with all options on.

Tip: Toggle off any piece of criteria you don't need or want and they'll be excluded from the scoring.

Click the purple Score prompt button to get started. This will bring up the Scoring Prompt modal box. After a few seconds the modal box will go away and the scoring results will be displayed.

The scoring results shows you the overall prompt score, its ranking, and the scores and weighted scores for each of the criteria used. You'll also see how many tokens were used and the how long it took to perform the analysis in the bottom-right corner of the results.

Rankings, Scores, and Weighted Scores

The overall prompt score rankings are as follows:

Poor	Scores from 0 - 49%
Fair	Scores from 50 - 74%
Good	Scores from 75 - 89%
Excellent	Scores from 90 - 100%

Each criteria used in the scoring analysis is given a raw score based on how well it's addressed by the prompt. The score for each criteria is defined accordingly:

0 = The criteria is not addressed at all by the prompt.

1 = The criteria is poorly addressed by the prompt.

2 = The criteria is addressed by the prompt but needs improvement.

3 = The criteria is partially addressed by the prompt.

4 = The criteria is well addressed by the prompt.

5 = The criteria is fully addressed by the prompt.

The weighted score for each criteria in the list is its contribution to the overall score. The sum of the weighted scores will equal the overall prompt score.

Scoring Report

With the prompt scoring results, a green View report button will appear next to the purple Score prompt button. Clicking View report will open a flyout panel with detailed analysis of the prompt. Here you will find a summary of the scoring analysis, an overview of the prompt's strengths, areas for improvement, and feedback for each piece of criteria used in the analysis.

Auto-Improve Score

For prompts that score less than 90%, a purple Improve button will appear at the top-right of the report flyout panel. Clicking this button will use the analysis to fix any parts of the prompt that scored 0, 1, 2, or 3 (shown with red or yellow circles) and create a new prompt version with those changes in place.

The Improve button does not display for prompts that score 90% or greater.

When the prompt improvement is complete, you'll be taken to the prompt Compare screen so you can see the changes that were made side-by-side with the previous version

Score Prompt with Different Criteria

Starting a new prompt score from scratch uses the Default system criteria provided by Zatomic. This includes about a dozen pieces of criteria that apply to just about every prompt. However, if you've created your own scoring criteria, you can score prompts using that criteria instead of the system default.

To do this, click the Prompt Criteria link at the top of the criteria list.

This will bring up the Scoring Criteria modal box. Select your custom criteria from the dropdown list and click the green Select criteria button.

This will reload the screen with the selected criteria and will clear out all previous scoring. You can then click the purple Score prompt button again to get a new score based on the updated criteria.

Score Prompt with Different Models

To change which AI model is used to score the prompt, click the Settings link near the top-right of the screen.

This will open a side panel where you can choose different models to use for the scoring. Make your model selection, click the green Apply button, then re-run the prompt scoring again.