Do Now

Terminology and Concepts to look out for

criterion · subjective · consistent

Relevance to Me

Almost nothing you would actually use one of these tools for has a right answer you can look up. Today's prompt does not have one either, and you still have to rank the three responses and defend it.

Board Question

You will score three responses against four criteria. One model will finish first. Out of the four criteria, how many will that model win? Write a number from 0 to 4 in your notebook and commit to it.

Same stations. Same groups. The four criteria are posted before the prompt runs.

When There Is Nothing To Check

Yesterday's prompt had answers you could look up. Today's does not.

Today's prompt asks for advice. There is no source anywhere that settles whether one piece of advice is better than another. You cannot verify it. And you still have to rank the three responses, and you still have to be able to defend the ranking to somebody who ranked them differently.

This is the situation you are actually in most of the time. When people use these tools they are usually asking for help writing something, or explaining something, or deciding something. Almost none of that has an answer key. So if correctness is the only judgment you know how to make, you can barely judge anything.

There is a word for a judgment that depends on the person making it. That word is subjective. A subjective judgment is not the same as a worthless one, and it is not the same as an arbitrary one. Arbitrary means no reason at all. Subjective means the reason comes from a standard that somebody chose, rather than from a fact sitting out in the world.

The thing that keeps a subjective judgment from sliding into an arbitrary one is a criterion. A criterion is a stated standard that you apply to every response the same way. You say what you are looking for before you look. Then you look for exactly that and nothing else.

You are not inventing criteria today. You are given four, and every group in every period uses the same four.

Specific. The response names actions a person could actually take, rather than describing how they should feel.

Usable this week. Following the advice does not require money, equipment, or another person's cooperation.

Complete. The response addresses before, during, and after, not just one of the three.

Honest. The response does not promise the problem will go away.

Score each response on each criterion. Yes or no. There is no half credit and no sort of, because sort of is where a standard quietly turns back into a preference.

The hardest part of today is consistency. Being consistent means applying a criterion the same way to a response you like and a response you do not. If a vague sentence costs the first response a yes, the same vague sentence costs the third response a yes. Groups fail this without noticing, which is why your form asks you to name the one criterion you found hardest to apply the same way twice.

Twelve judgments. Three responses, four criteria. The model with the most yes marks ranks first.

Today's Lab Overview

Today's prompt

This is exactly what gets sent, to all three models, at the same moment.

A tenth grade student has to give a five minute presentation in front of the class
next week and is nervous about it. Give them advice.

The instructions all three are given

You are answering a question for a high school class in New Orleans.
Answer the question directly and completely.
Do not ask the student a question back.
PaneModel
1Ministral 3B
2Ministral 14B
3Mistral Large

Pane order never changes. Refer to responses by pane number all week. What differs between them is size.

Today's Lab Overview

The four criteria

Posted before the prompt runs. Identical for every group in every period. Score every pane against all four.

  • Specific. Names actions a person could take, rather than how they should feel.
  • Usable this week. Does not require money, equipment, or another person's cooperation.
  • Complete. Addresses before, during, and after.
  • Honest. Does not promise the nervousness will go away.

These stay reachable all period — the button in the bar below.

I Do · pick an answer, then Check
Question 1notebook cue: criterion
A criterion is
Question 2notebook cue: subjective
A judgment is subjective when
Question 3notebook cue: consistent
Your group marks a vague sentence against pane 1 and lets the same kind of sentence pass in pane 3. Which word names what your group failed to be?
Question 4worked aloud
Arbitrary and subjective are not the same thing. What is the difference?
Question 5worked aloud
A response says "just try to relax and be confident." Which criterion does it most clearly fail?
Question 6worked aloud
A response says "the nervousness will disappear once you start talking." Which criterion does it most clearly fail?
Lab 1, Day 3 — Wednesday, September 2 · How to tell which one is better, and how to say why.

The four criteria