Interview Preparation · NAB Melbourne · AI Scientist

The Prep Pack

Five pages, one job: walk into the room able to answer any question on evaluation, guardrails, memory, orchestration and intent recognition — at architecture level, at implementation level, and with a real project behind every claim.

The Five Pages

Read them in this order the first time. After that, jump straight to whichever one matches the question you just failed to answer out loud.

Master Prep

Memory · Evaluation · Guardrails, three levels each

The backbone. Each of the three themes is answered three times over: L1 architecture (how you'd frame it for a room of leaders), L2 implementation (how you'd actually build it), L3 in this project (what you personally did in Mines Financial). If you only have time for one page, it's this one.

1 · Memory — what an agent should remember, where it lives, how it expires
2 · Evaluation — offline harness, judges, online experiments, drift
3 · Guardrails — layered runtime controls, groundedness, audit trails
Numbers to remember + glossary

Open →

Use it forAny open-ended "tell me how you'd approach…" question

Reading time~45 min, dense

DepthLeader → senior → hands-on

STAR Stories

Design decisions from Mines Financial, in STAR form

Evidence. Every architectural claim on page 01 needs a story with a situation, a task, an action and a measured result — that's what STAR means, and it's how behavioural interviews are scored. Sixteen real decisions from this project, each written so you can tell it in ninety seconds, including the ones that went wrong.

1 · Evaluation — the harness, mock vs real runs, the BM25 tokeniser experiment
2 · Guardrails — groundedness gating, table units, contamination
3 · Memory and 4 · Retrieval, ingest, orchestration
Weakest points, prepared — the questions you'd rather not be asked

Open →

Use it for"Tell me about a time when…" and every follow-up drill-down

Reading time~40 min

DepthConcrete, with numbers

Evals & Guardrails

The two topics the job description leans on hardest, as Q&A

A focused drill on the two subjects that appear in almost every line of the NAB job ad. Twenty-five questions, from "how would you set an evaluation strategy for a whole AI function" down to "what is a golden dataset" — each answered problem-first, then naive approach, then the real method, which is also a good way to talk in the room.

JD map — each JD line paired with what they're likely probing
Evaluations Q1–14 · Guardrails Q15–25
The one table to remember — evals vs guardrails, side by side
Rapid-fire follow-ups with model answers

Open →

Use it forLast-minute revision; the 30-second synthesis answer

Reading time~25 min

DepthLeader → senior → junior

Core Concepts

Memory, context management, orchestration — and where they differ

Foundations, assuming nothing. These three terms get used interchangeably and they are not the same thing; being able to draw the boundary cleanly is itself a senior signal. Concepts first, then exactly how each one shows up in this project, then one request traced end to end.

Telling the three apart — the boundary question
1 · Memory systems — the four kinds, and which ones you built
2 · Context management — the five jobs, every call's context budget
3 · Orchestration + one request, end to end

Open →

Use it forDefinition questions; whiteboard explanations

Reading time~30 min

DepthGround-up, zero assumed knowledge

Intent Recognition

Two subpages: the discipline, then the code

The hinge between understanding a request and acting on it — and the one place where a misunderstanding becomes a wrong action rather than a wrong sentence. A section of its own, because it has two halves that need reading in order: the general design and measurement of intent recognition, then exactly how it is implemented in this project, gaps included.

1 · The concept — designing intents, granularity, three implementations, unknown and ambiguous, multi-intent as planning
1 · Measuring it — why accuracy lies, confusion matrices, four evaluation layers, wrong action rate
2 · In this project — four flags instead of a label, regex slots, the clarification interrupt, capped retries
2 · Owned weaknesses — no routing confidence, no clarify loop counter, an eval that partly grades itself

Open →

Use it for"How does your agent decide what to do?"

Reading time~35 min across both

DepthConcept → implementation → gaps

Eval Deep Dive

Twelve concepts in four groups of three

Evaluation on its own, from the ground up. Four groups of three, ordered so the structure itself is recallable under pressure. Every concept carries a plain explanation with a NAB example, a concrete failure scenario if it is missing, and a six-row STAR block including the key points to memorise and the literal questions it answers.

A · What you measure — golden set, component vs end-to-end, per-stage metrics
B · How you measure — deterministic vs judge, judge validation, rubrics
C · When you measure — offline, CI tiering, shadow / canary / A-B
D · How you trust it — variance, sample size, drift and baselines

Open →

Use it forAny "how do you know it works?" question

Reading time~50 min

DepthConcept → failure → STAR, per concept

Guardrails Deep Dive

Four boundaries, three controls at each

Guardrails on their own, organised by the boundary a request crosses rather than by technique — which is both easier to remember and closer to how you would defend the design in a review. Same three-part treatment per concept, with the regulated-advice boundary and the agentic-action gap called out for a bank.

A · Input — scope and advice boundary, ambiguity, untrusted input and personal data
B · Evidence — relevance filtering, coverage checking, provenance
C · Generation — groundedness, claim classification, bounded retry
D · Output and runtime — the deterministic gate, fail-open direction, auditability

Open →

Use it for"How do you keep it safe?"; Responsible AI questions

Reading time~55 min

DepthConcept → failure → STAR, per concept

If They Ask This, Go There

The mapping to run in your head the night before.

The question	Page	What to reach for
"How would you evaluate an agent?"	03 then 01	End-to-end vs trajectory scoring; then your own harness as proof
"How do you keep an LLM safe in a bank?"	03	Defence in depth: input, action, output, oversight layers
"Tell me about a hard technical trade-off."	02	Any STAR entry — lead with the measured result
"What's the difference between context and memory?"	04	The boundary section; draw it, don't just say it
"Walk me through your architecture."	04 then 02	One request end to end, then the decisions behind it
"How do you know it actually got better?"	03	Confidence intervals, paired comparisons, why one run proves nothing
"What went wrong, and what did you do?"	02	Weakest points, prepared — answer it before they find it
"How does the agent decide what to do?"	05	Four flags rather than one label — and why that gives multi-intent for free
"What if the user's request is ambiguous?"	05	Confidence plus a clarification policy; here, a real graph interrupt

The frame that ties it together NAB is a regulated bank: APRA sets prudential expectations, ASIC governs conduct. So every answer lands better when it ends where the risk ends — trust, auditability, and the cost of a failure reaching a customer. Technique first, then the sentence that shows you know who has to sign off on it.

How To Use This Pack

Reading is not preparing. The pages are written to be spoken.

Four passes, in order

Pass 1 — read. Straight through, pages 04 → 01 → 03 → 02. Don't take notes; you'll re-read the parts that didn't stick.
Pass 2 — speak. Cover the answer, read only the question, say your answer out loud in under ninety seconds. Anything you can't say out loud, you don't know yet.
Pass 3 — numbers. Memorise the measured results on page 02 and the "numbers to remember" block on page 01. Specific numbers are the single strongest credibility signal in a technical interview.
Pass 4 — the night before. Only the route table above, the one comparison table on page 03, and the 30-second synthesis. Nothing new.

Every answer has the same shape What problem exists → the obvious approach and why it fails → what you actually do → how you'd know it worked. Use it aloud. It makes you sound like someone who has shipped the thing, not read about it.