We Do · Lab 1 Close-out, Day 3 — Friday, September 11

How this activity works

This week's classwork is turning last week's observations into a complete lab report.

On the next slide, you will see a left pane called My Observations from Last Week with tabs for your Day 2, Day 3, and Day 4 observation forms from last week, and a right pane called My Lab Report with three tabs — Intro, Body, Conclusion. Click back and forth between them to finish all three.

Today is structured work time. Intro, then Body, then Conclusion. Finish each section that is not yet done.

Submit and reopen: every form on this page except the exit ticket can be submitted whenever you have something worth showing me. Submitting lets me see your work in the responses sheet. Once you submit, the confirmation screen shows an Edit your response link — click it to reopen the same form and keep working. Submit again when you make it stronger; I see the newer version.

Catch up: to look back at what the models sent your station last week, open MuggsOfPrompts and sign in as your station.

NO EXIT TICKET TODAY.

4th period is out for a school-wide activity, and exit tickets are not made up. The report forms stay open past today.

My Observations from Last Week

View

Friday, September 11

Station view is read-only. It is the backup for a form that will not load. Switch to My view to answer for a grade.

Lab Report 1 · Intro

Lab 1: Three Answers, One Question. Report, part 2 of 2: the introduction, and the reflection and conclusion.

Introduction

You are writing a paper. Complete sentences, for a reader who was not in this room.

An introduction tells a reader what you were trying to find out and how it was set up, before you tell them what happened. Someone who was not in this room for those four days should be able to read this and understand the lab.

1.What question was this lab trying to answer?

Discuss what you were trying to learn by looking at the performance of these 3 different models over last week's lab? What were you trying to determine about these models' generated responses?

2.What were the three models, and what was different about them? *

Name the 3 models. What is the name of the model that generated responses in Pane 1? What is the name of the model that generated responses in Pane 2? What is the name of the model that generated responses in Pane 3?

What is the main difference between the models?

3.What was held the same every single day?

Did we change the instructions we gave each model each day? Did we change the prompts each day? Did each model get the same instructions each day? Did they get the same prompts? Did you use the same criteria each day, or did criteria change?

4.What changed from one day to the next?

If there were changes day-to-day, what were they? And did the same changes happen to each model?

5.Before you started, what did you expect to find?

Example, "I expected Mistral Large to out perform the other 2 models (Mistral 3b and Mistral 14b), because..."

6.Anything a reader needs to know before they read the body.

Would it help the reader to know what company, or companies, made these models? Would it help them to know what a parameter is, more or less?

Lab Report 1 · Body

Lab 1: Three Answers, One Question. Report, part 1 of 2: the body.

Tuesday, September 1 — Correctness

You are writing a paper. Complete sentences, for a reader who was not in this room.

This is exactly what was sent, to all three models, at the same moment:

Answer all three questions.

In what year did the Superdome in New Orleans open?
Who is the current head coach of the New Orleans Saints?
What is the population of the city of New Orleans?

The instructions all three were given:

You are answering a question for a high school class in New Orleans. Answer the question directly and completely. Do not ask the student a question back.

Your Day 2 observation form is open on the left. Everything you need is in it. Name the pane before you use a shortcut. Give every number with what it is out of. Say what happened before you say what it means.

1.The sources. For each of the three questions, where did you check, and what did you find there? *

Could a reader open the same source and find the same answer? Is there a date on the population figure?

2.The score. How many of the three did each pane get right, and which question did each one miss? If a pane refused or gave no answer, say so. *

Is every count written as a number out of three? Does what you wrote here match the marks on your Day 2 form?

3.Confidence. How did the wrong answers sound next to the right ones? Did any pane say it was unsure? *

From your description alone, could a reader tell that a confident answer was wrong?

4.Tuesday's ranking, and what the word better meant on Tuesday. *

Could a reader rebuild your ranking from the counts above? If two panes tied, did you say so?

Wednesday, September 2 — Criteria

You are writing a paper. Complete sentences, for a reader who was not in this room.

This is exactly what was sent, to all three models, at the same moment:

A tenth grade student has to give a five minute presentation in front of the class next week and is nervous about it. Give them advice.

The instructions all three were given:

You are answering a question for a high school class in New Orleans. Answer the question directly and completely. Do not ask the student a question back.

Your Day 3 observation form is open on the left. There was no answer key on Wednesday. What there was instead is the subject of this section.

5.The four criteria. What did each one ask of a response? *

Would a reader who never saw the criteria sheet understand what usable this week was testing?

6.The tally. How many of the four criteria did each pane meet, and which ones did it miss? *

Is every count out of four? Does it match the yes and no marks on your Day 3 form?

7.The sentences that decided it. For the pane that ranked first and the pane that ranked last, which sentence settled a criterion, and which criterion did it settle? *

Is the sentence quoted exactly, from the copy button, not from memory? Does the pane number come before the quote?

8.The criterion that was hardest to apply the same way twice. What did your group decide, and did that decision change any pane's count? *

If your group disagreed, what was the disagreement actually about?

9.Wednesday's ranking, and what the word better meant on Wednesday. *

How is Wednesday's meaning of better different from Tuesday's? Can a reader see that difference in your two rankings?

Thursday, September 3 — The Length Limit

You are writing a paper. Complete sentences, for a reader who was not in this room.

This is exactly what was sent, to all three models, at the same moment:

Is going to college worth taking out student loans for?

Answer in no more than 120 words.

The instructions all three were given:

You are answering a question for a high school class in New Orleans. Answer the question directly and completely. Do not ask the student a question back.

Your Day 4 observation form is open on the left. This was Monday's question with one new instruction. The numbers under each pane were read off the screen, not estimated.

10.The numbers. For each pane: words, seconds, and tokens per second. *

Which pane finished first? Which pane wrote fastest? Are those the same pane, and if not, why not?

11.The limit. Which panes stayed at or under 120 words, and by how much did the others go over? *

Is over written as a number of words, or as a feeling?

12.Specificity. Did each pane commit to amounts, names, or times, or stay general? One example from each pane. *

Would a reader know which pane you are quoting? Is the example a quote or a summary?

13.Hedging. How many hedges did each pane use? Quote one exactly and say which pane it came from. If no pane hedged, say that plainly. *

Is zero written down as zero? If three models of three different sizes all made the same choice, what might that tell a reader?

14.Structure, and what all three had in common. How was each response laid out, and apart from layout, what did all three share? *

Could a reader picture each response from your description without seeing the screen?

15.Monday's question, with a limit. What did the 120-word limit change, and what did it not change? Point at a number.

Look at your Day 1 form. Which of Monday's numbers moved, and which stayed put?

Lab Report 1 · Conclusion

Lab 1: Three Answers, One Question. Report: the reflection and conclusion.

Reflection and Conclusion

You are writing a paper. Complete sentences, for a reader who was not in this room.

A conclusion is a claim plus the evidence that holds it up. The evidence is already written, in your body. Your job here is to say what it adds up to and to point at the specific numbers that support it.

A conclusion that says a negative result honestly is worth more than one that overstates a finding.

7.Your claim. In two or three sentences, what did this lab show about these three models? *

Could this claim be wrong? If it could not be wrong, is it a claim?

8.Which day supports your claim most strongly? Point at the actual numbers in it. *

Could a reader open your body, find those numbers, and check them? Is each number out of something?

9.Name one thing you cannot say from this lab, even though you might believe it. What evidence would you need? *

How many prompts did you run? How many runs of each? Who wrote the questions? What does each of those limit?

10.Across the four days, which changed more: the three models, or what you meant by the word better? *

When, if ever, did the models change? What did better mean on Tuesday, and what did it mean on Thursday?

11.If you ran this lab again with one more week, what would you change, and what would you expect to find? *

Would your change fix the limit you named above, or a different one?

12.One sentence. What is the single thing you would tell someone who uses these tools and has never thought about which one they are using?

Intro, Body, Conclusion — click a tab to switch forms. All three stay open past today.

My Lab Report