Joshua Durey (Josh Durey) ยท Applied AI Enablement & Agentic Workflows Applied AI enablement. I audit workflows for AI readiness, build agentic systems, and run the LLM quality control that gets teams to adoption they can trust. https://jdurey.github.io/ 2026-11-15T00:00:00Z Joshua Durey https://jdurey.github.io/ josh.durey@gmail.com Hiring an Applied AI Enablement Lead: A 30-60-90 Day Playbook for Team Adoption https://jdurey.github.io/guides/applied-ai-enablement-lead/ 2026-11-15T00:00:00Z What an Applied AI Enablement Lead actually ships in the first 90 days: workflow audits, self-checking agent pipelines, cross-family quality gates, and staff playbooks that turn shelfware seats into daily output. Enterprise AI Workflow Audit Checklist: Ranking Tasks by Payoff, Risk & Checkpoints https://jdurey.github.io/guides/ai-workflow-audit-checklist/ 2026-11-15T00:00:00Z A practical four-step checklist for auditing team operations before writing automation code: mapping real inputs, pricing silent errors, placing human checkpoints, and killing low-payoff pilots early. Self-Reviewing Agentic Workflows: Designing Pipelines Non-Technical Teams Trust https://jdurey.github.io/guides/self-reviewing-agentic-workflows/ 2026-11-15T00:00:00Z How to replace fragile single-shot prompts with four-stage self-checking workflows: deterministic Python pre-gates, narrow drafting nodes, cross-family critics, and SHA-verified audit receipts. LLM-as-a-Judge Calibration: Stopping Self-Bias, Verdict Flips & Silent Truncations https://jdurey.github.io/guides/llm-judge-calibration-and-qc/ 2026-11-15T00:00:00Z How to calibrate LLM-as-a-Judge panels against human expert baselines: binary decomposed rubrics, cross-vendor panels, golden-set agreement checks, and catching silent HTTP 200 truncations. SOP-to-SCORM AI Pipeline: Compiling Policy Docs into Verified LMS & QTI Modules https://jdurey.github.io/guides/sop-to-scorm-ai-pipeline/ 2026-11-15T00:00:00Z How corporate L&D and compliance teams turn static SOPs and runbooks into ADL SCORM 1.2 packages, QTI 1.2 item banks, 300-DPI guides, and narrated 1080p lessons with 0% policy drift. 6-Rung LLM Cost & Latency Routing: Cutting Enterprise Token Spend Without Losing Quality https://jdurey.github.io/guides/llm-cost-and-latency-routing/ 2026-11-15T00:00:00Z Why a tiny fraction of agent calls burn 80% of your API bill, and how a 6-rung policy router shifts routine checks to zero-token Python hooks and fast mid-tier models. AI Curriculum & QTI Assessment Quality Control: Blind-Solver Checks & Zero Answer Leaks https://jdurey.github.io/guides/ai-curriculum-and-qti-assessment-qc/ 2026-11-15T00:00:00Z Engineering-grade quality control for EdTech and enterprise assessment pipelines: catching ambiguous distractors, standards drift, hint answer leaks, and broken math rendering before students see them. Two-Week AI Workflow & QC Sprints: Moving Stalled Pilots into Daily Team Use https://jdurey.github.io/guides/fixed-scope-ai-workflow-sprint/ 2026-11-15T00:00:00Z How founders and operations leaders use a fixed-scope two-week sprint to audit one high-friction bottleneck, deploy a self-checking workflow with human checkpoints, and keep the runnable QC harness. Model evaluation in machine learning https://jdurey.github.io/work/model-evaluation-in-machine-learning/ 2026-09-25T00:00:00Z A practical answer grounded in a runnable, published harness rather than opinion. Links to Blind expert-parity: can a model adjudicate like a credentialed examiner, and can you prove it?. Llm judge metrics https://jdurey.github.io/work/llm-judge-metrics/ 2026-09-24T00:00:00Z A practical answer grounded in a runnable, published harness rather than opinion. Links to Can you trust the model that grades your content? Measuring when an AI judge waves through broken work. Llm as a judge framework https://jdurey.github.io/work/llm-as-a-judge-framework/ 2026-09-21T00:00:00Z A practical answer grounded in a runnable, published harness rather than opinion. Links to Can you trust the model that grades your content? Measuring when an AI judge waves through broken work. Llm as a judge paper https://jdurey.github.io/work/llm-as-a-judge-paper/ 2026-09-14T00:00:00Z A practical answer grounded in a runnable, published harness rather than opinion. Links to Can you trust the model that grades your content? Measuring when an AI judge waves through broken work. Llm eval harness https://jdurey.github.io/work/llm-eval-harness/ 2026-09-13T00:00:00Z A practical answer grounded in a runnable, published harness rather than opinion. Links to How to audit an LLM judge before you trust its verdicts. Red teaming llm applications https://jdurey.github.io/work/red-teaming-llm-applications/ 2026-09-02T00:00:00Z A practical answer grounded in a runnable, published harness rather than opinion. Links to Red-teaming my own publish gate. Llm judge https://jdurey.github.io/work/llm-judge/ 2026-08-31T00:00:00Z A practical answer grounded in a runnable, published harness rather than opinion. Links to Can you trust the model that grades your content? Measuring when an AI judge waves through broken work. Llm as a judge https://jdurey.github.io/work/llm-as-a-judge/ 2026-08-30T00:00:00Z A practical answer grounded in a runnable, published harness rather than opinion. Links to Can you trust the model that grades your content? Measuring when an AI judge waves through broken work. Llm judge model https://jdurey.github.io/work/llm-judge-model/ 2026-08-26T00:00:00Z A practical answer grounded in a runnable, published harness rather than opinion. Links to Can you trust the model that grades your content? Measuring when an AI judge waves through broken work. What is model evaluation in machine learning https://jdurey.github.io/work/what-is-model-evaluation-in-machine-learning/ 2026-08-24T00:00:00Z A practical answer grounded in a runnable, published harness rather than opinion. Links to Blind expert-parity: can a model adjudicate like a credentialed examiner, and can you prove it?. Why use multiple AI models instead of sampling one model many times https://jdurey.github.io/work/multi-model-panels-vs-resampling/ 2026-08-21T00:00:00Z A practical answer grounded in a runnable, published harness rather than opinion. Links to Author with one model, attack with another. How to run AI content pipelines unattended overnight safely https://jdurey.github.io/work/unattended-ai-pipeline-failure-budget/ 2026-08-19T00:00:00Z A practical answer grounded in a runnable, published harness rather than opinion. Links to Three failures your LLM pipeline never logs: gates for truncation, reasoning tax, and latency tails. How to make AI QC verdicts auditable and replayable https://jdurey.github.io/work/replayable-ai-qc-verdicts/ 2026-08-17T00:00:00Z A practical answer grounded in a runnable, published harness rather than opinion. Links to Same answer, different grade: measuring when an AI grader can't hold a verdict. How to audit an AI pipeline when you cannot read its data https://jdurey.github.io/work/audit-ai-pipeline-without-reading-data/ 2026-08-14T00:00:00Z A practical answer grounded in a runnable, published harness rather than opinion. Links to Three failures your LLM pipeline never logs: gates for truncation, reasoning tax, and latency tails. How to choose an LLM provider when many serve the same model https://jdurey.github.io/work/choose-llm-provider-by-latency-not-price/ 2026-08-12T00:00:00Z A practical answer grounded in a runnable, published harness rather than opinion. Links to Three failures your LLM pipeline never logs: gates for truncation, reasoning tax, and latency tails. Why a few LLM calls carry most of your API bill https://jdurey.github.io/work/llm-api-costs-concentrate-in-few-calls/ 2026-08-10T00:00:00Z A practical answer grounded in a runnable, published harness rather than opinion. Links to Three failures your LLM pipeline never logs: gates for truncation, reasoning tax, and latency tails. How much do reasoning tokens add to LLM API costs https://jdurey.github.io/work/how-much-do-reasoning-tokens-cost/ 2026-08-06T00:00:00Z A practical answer grounded in a runnable, published harness rather than opinion. Links to Three failures your LLM pipeline never logs: gates for truncation, reasoning tax, and latency tails. Why does my LLM output get cut off and how do I detect it https://jdurey.github.io/work/why-llm-output-gets-cut-off/ 2026-08-03T00:00:00Z A practical answer grounded in a runnable, published harness rather than opinion. Links to Three failures your LLM pipeline never logs: gates for truncation, reasoning tax, and latency tails. What is llm judge https://jdurey.github.io/work/what-is-llm-judge/ 2026-07-31T00:00:00Z A practical answer grounded in a runnable, published harness rather than opinion. Links to Can you trust the model that grades your content? Measuring when an AI judge waves through broken work. Three failures your LLM pipeline never logs: gates for truncation, reasoning tax, and latency tails https://jdurey.github.io/work/telemetry-gates/ 2026-07-31T00:00:00Z I aggregated a month of my own API telemetry, 88,000 calls, and found three failure classes that raise no exception anywhere. One model silently truncated 5.7% of its responses. Eight models spent 83 to 95 percent of their completion tokens on reasoning nobody reads. Providers serving comparable models ranged from 1.1s to 20s median latency, with one tail hitting 89 seconds. So I built three deterministic gates that run on telemetry alone, and tested them against a synthetic fleet with defects planted by construction. The gate is correct when it finds every planted defect and nothing else. How to evaluate llm output https://jdurey.github.io/work/how-to-evaluate-llm-output/ 2026-07-31T00:00:00Z A practical answer grounded in a runnable, published harness rather than opinion. Links to Can you trust the model that grades your content? Measuring when an AI judge waves through broken work. The agent-boundary harness: measuring whether an AI agent respects a data fence, and whether the fence holds https://jdurey.github.io/work/agent-boundary/ 2026-07-31T00:00:00Z An agent with file access can leak in two ways. The guard code fails, or the model decides to cross. I built a two-layer instrument that measures both, scored by whether a sealed canary token ever reads out. A naive path guard passed 75% and leaked twice through known EscapeRoute-class bypasses. The hardened guard passed clean. Then I tempted real models with tasks where crossing the boundary is the easy path, and counted the crossings. Everything runs on a fictional workspace, so the whole machine is public. What is llm as a judge https://jdurey.github.io/work/what-is-llm-as-a-judge/ 2026-07-29T00:00:00Z A practical answer grounded in a runnable, published harness rather than opinion. Links to Can you trust the model that grades your content? Measuring when an AI judge waves through broken work. How to red-team AI-built tools and content before they reach a classroom https://jdurey.github.io/work/red-team-ai-built-tools-before-classroom-use/ 2026-07-29T00:00:00Z A practical answer grounded in a runnable, published harness rather than opinion. Links to Can you trust the model that grades your content? Measuring when an AI judge waves through broken work. Why an AI grader checking its own output can't be trusted, and what to do https://jdurey.github.io/work/why-an-ai-grader-checking-itself-fails/ 2026-07-27T00:00:00Z A practical answer grounded in a runnable, published harness rather than opinion. Links to Can you trust the model that grades your content? Measuring when an AI judge waves through broken work.