AI Lab Dashboard

Professional AI experimentation workspace

Build AI skills the way real AI teams work.

Design prompts, version them, compare alternatives, test on datasets, define structured outputs and tools, build agent workflows, evaluate quality, document findings, and turn experiments into portfolio-ready AI applications.

Prompt VersioningA/B ComparisonDataset EvalsStructured OutputFunction CallingAgent DesignProjects
Prompt versions
0

Saved, rollback-ready prompt versions.

Experiments
0

Playground runs and comparisons.

Eval cases
0

Reusable test inputs with references.

Projects
0

AI app and workflow prototypes.

Tools
0

Reusable function/tool definitions.

Notebook
0

Findings, reflections and research notes.

Professional workflow

1 · Design & version a prompt

Separate reusable instructions from variables and task inputs.

PromptOps
2 · Test with representative cases

Use a dataset rather than trusting one good-looking response.

Evals
3 · Compare alternatives

Run A/B or pairwise reviews and record why one version wins.

Compare
4 · Define reliable interfaces

Use structured output schemas and tool contracts for predictable application behavior.

Build
5 · Prototype, evaluate and document

Record risks, metrics, limitations, test results and deployment assumptions.

Ship

Quick start

Starter blueprints

Prompt composer

Prompt quality

Clear task—
Context—
Constraints—
Output contract—
Variables—
Scores are local heuristics for learning—not a substitute for real model evaluation.

Rendered prompt preview

Compose a prompt and choose Preview.

Experiment output

Local simulation mode. Configure a secure backend in Settings for real provider calls.
No experiment run yet.

Run inspector

Estimated input tokens0
Target output tokens0
LatencyLocal
ProviderLocal Lab

Run history

Backend-ready request

Use a server-side endpoint so provider secrets stay off the client.

Prompt payload

Prompt Library & Version History

Publish reusable prompts as versions, compare changes and roll back when needed.

Saved versions

Selected version

Select a saved version.

Side-by-side Prompt Comparison

Compare prompt variants before publishing. Paste prompts or load recent versions.

No analysis yet.
No analysis yet.
—A structure
—B structure
—Suggested winner
0Recorded reviews
—Score gap

Create evaluation case

Evaluation methods

Deterministic checks

Required words, JSON validity, length and format rules.

Rubric scoring

Relevance, correctness, clarity, safety and completeness.

Pairwise review

Compare two versions and record the preferred output.

Current eval dataset

#InputReferenceCategoryRiskWeight

Batch experiment

Run your current prompt against every eval case. In local mode Ethan AI Lab scores prompt/test alignment heuristically; with a backend it can store real outputs for review.

Experiment summary

Cases0
Pass rate—
Average score—
High-risk cases0

Case results

InputScoreStatusNotes

Experiment history

Dataset Manager

Build reusable test sets for prompt regression checks and AI quality experiments.

Imported files are read locally in your browser. Nothing is uploaded by this standalone PWA.

Dataset health

Total cases0
With references0
Categories0
High-risk0

Cases

InputReferenceCategoryRisk

JSON Schema Studio

Output Validator

Validation report

Add a schema and candidate output.

Function / Tool Builder

Design checklist

Clear name

Describe one action, not a whole workflow.

Strong typing

Use explicit parameter types and required fields.

Validation

Validate arguments before executing real actions.

Least privilege

Only expose tools an agent actually needs.

Tool catalog

Agent Blueprint Studio

Agent readiness

Goal clarity—
Tool scope—
Guardrails—
Human oversight—

Agent blueprint

Define an agent and choose Build Blueprint.

Saved agents

AI App / Workflow Canvas

Build stages

1. Problem

Define user and measurable need.

2. Prototype

Prompt + tools + interface.

3. Evaluate

Dataset + rubrics + regressions.

4. Safeguard

Privacy, failures and oversight.

5. Ship

Backend, monitoring and iteration.

Project blueprint

Complete the canvas to generate a blueprint.

Saved projects

AI Lab Notebook

Good experiment notes

Record what changed, why you changed it, what you expected, what actually happened and what you will test next.

Entries

Responsible AI is part of professional AI engineering.

Good systems combine model capability with verification, privacy, bias testing, security, transparent limitations and appropriate human oversight.

Verify

Important factual claims need independent checking against reliable sources or reference data.

Protect privacy

Do not expose passwords, secrets, confidential records or unnecessary personal data to model providers.

Evaluate bias

Test representative cases and inspect whether outputs unfairly disadvantage people or groups.

Human oversight

High-impact decisions should have meaningful human review and clear escalation paths.

Secure tools

Validate tool arguments, restrict permissions and require confirmation for consequential actions.

Document limitations

State what the system can and cannot reliably do, and monitor failures after deployment.

Pre-deployment checklist

Representative eval setPrompt version pinnedSchema validatedTool permissions reviewedPrivacy reviewedFailure cases documentedHuman escalation definedMonitoring plan

Live Model Connector

Do not paste provider API keys into this PWA. Configure a secure server/serverless endpoint that owns the provider credentials and accepts the request payload below.

Workspace

Integration contract

POST /api/ai Content-Type: application/json { "modelProfile": "Balanced", "mode": "Chat", "system": "...", "prompt": "...", "format": "Markdown", "metadata": { "experimentId": "..." } } Expected response: { "text": "model output", "usage": {"inputTokens": 0, "outputTokens": 0}, "model": "provider model name" }