1
Audit the input
Record token splits and the complete context budget for three representative prompts.
Required evidence: Tokenizer version, token table, and one demonstrated boundary case.
0/80 minimum characters
Foundation mini-project·2-3 hours
LLM Behaviour Lab
Build a small, reproducible report showing how tokenization, context, decoding, prompting, and evidence change language-model behavior.
Scenario
A product team has five impressive model screenshots but no repeatable evidence. You must replace the demo claims with controlled cases, visible intermediate state, and bounded conclusions.
You will demonstrate
- Inspect the mechanisms that shape one LLM response.
- Run controlled comparisons instead of cherry-picked demos.
- Separate fluent continuation from grounded support.
Project evidence
Show the work, not a checked box.
Each response is stored in this browser as you type. Include metrics, test output, or a decision rationale wherever the deliverable asks for it.
2
Compare decoding
Hold the prompt fixed and compare two decoding policies over repeated runs.
Required evidence: Probability or output traces with a validity and diversity observation.
0/80 minimum characters
3
Test a prompt contract
Define labels or a response schema and run ordinary, ambiguous, and adversarial inputs.
Required evidence: Versioned prompt, expected behavior, and a pass/fail table.
0/80 minimum characters
4
Test support
Create answerable and unanswerable evidence cases and map accepted claims to sources.
Required evidence: Claim-support table plus an explicit abstention result.
0/80 minimum characters
5
Write the behaviour brief
State what changed, what remained uncertain, and which control belongs outside the model.
Required evidence: A concise recommendation tied to the four experiments.
0/80 minimum characters
Runnable Python lab
Complete a grounded-output scorecard
Measure valid schemas and supported answers across fixed cases.
Project defense
What makes two LLM outputs a useful decoding comparison?
Rubric self-review
Rate the evidence, not your effort: 0 missing, 1 weak, 2 adequate, 3 strong. All criteria must be reviewed, but a low honest score does not get hidden.
Every comparison holds unrelated variables fixed.
Intermediate token or probability evidence is visible.
Machine-consumed output is validated.
Unsupported claims are rejected or abstained.
Conclusions stay within the fixed cases and name a next test.
Useful references
Project completion gate
Completion is controlled by stored evidence, deterministic tests, a decision defense, rubric review, and the artifact when required.
Evidence pendingCode pendingDefense pendingRubric pending
Optional cloud portfolio
Submit evidence across devices.
An account is required. Submit only when the local completion gate passes. AI review is advisory and separate from deterministic completion.