Business dataData preparation workspace
Client name private// what we built
We built a workspace that imports inconsistent files, fills in requested information from approved sources, checks each result, and exports data ready for review.
- Product
- Data preparation workspace
- Field
- Business data
- Technical layers
- 04
- Safeguards
- 06
The challenge
What the team needed to solve
Useful business data is often spread across spreadsheets, documents, and online sources. Teams need a repeatable way to complete those records without losing track of where new information came from, who can access it, or what the work costs.
What we built
How the product works
The workspace cleans and profiles imported data, helps define the fields to complete, and researches each record using available workspace and web sources. It checks the shape of every result before export and keeps progress, source, usage, and audit information along the way.
Architecture inspection
The model is one participant in a typed workflow. These notes mark where responsibility, authority, and recovery live.
- sourceCSV / XLSXrows + columns
- sourceDocs / Sheetsworkspace context
- logicWorkspaceowned context
- logicWebpublic evidence
- logicSandboxparse + calculate
01 Normalize and profile
- Responsibility
- Import CSV, XLSX, and selected workspace documents into a normalized dataset with inferred field profiles.
- Boundary
- Workspace authorization, supported source types, and parseable tabular structure are required before profiling.
- Failure path
- Reject an unsupported or malformed source before enrichment rather than silently inventing its schema.
02 Typed enrichment plan
- Responsibility
- Suggest candidate fields and turn the operator's selection into requested keys and value-type contracts.
- Boundary
- The selected field names, types, source scope, and available tool configuration remain authoritative.
- Failure path
- Do not start row execution until the requested output shape is valid and the operator has chosen the fields.
03 Evidence tool boundary
- Responsibility
- Combine workspace context with configured search, scraping, or isolated script tools for each row.
- Boundary
- Every tool is workspace-scoped, explicitly available, and separated from the model by a typed contract.
- Failure path
- Record an unavailable tool or missing source as evidence failure instead of manufacturing a value.
04 Validate and retain lineage
- Responsibility
- Check each result against its requested keys and types while retaining job, evidence, usage, and audit state.
- Boundary
- Only validated row results can move into CSV, document, or spreadsheet export paths.
- Failure path
- Keep an invalid result out of export, record its validation error, and leave the job recoverable.
05 Recoverable job rail
- Responsibility
- Stream progress, honor cancellation, checkpoint row work, and claim jobs left stale by an interrupted process.
- Boundary
- Persisted job state and checkpoints, not an in-memory progress stream, determine what may resume.
- Failure path
- Resume eligible stale work from durable state or retain a terminal cancelled or failed result for review.
Decisions and authority
The important choices are explicit: what the system optimizes, what it refuses to improvise, and where people retain control.
- 01
Constraint
Files and workspace documents arrive with inconsistent headings, types, gaps, and shapes.
Decision
Normalize and profile every selected source before suggesting or executing enrichment fields.
A shared dataset profile gives field selection and row execution an explicit input contract.
Trade-off
Profiling adds an up-front stage and cannot resolve an ambiguous source without operator review.
- 02
Constraint
Free-form generated values are difficult to merge safely into a structured dataset.
Decision
Require requested keys and value types for every per-row enrichment result.
Runtime validation can reject incomplete or mistyped output before it reaches an export.
Trade-off
Potentially useful prose is withheld when it does not satisfy the chosen schema.
- 03
Constraint
Search, scraping, and code execution have different availability and trust boundaries.
Decision
Expose them as optional configured tools and run scripts inside an isolated sandbox.
The workflow can use only capabilities that are present while preserving a trace of how evidence was obtained.
Trade-off
Results vary with provider configuration, and unavailable tools can leave fields unresolved.
- 04
Constraint
Per-row enrichment can be long-running, cancellable, and billable across process restarts.
Decision
Persist checkpoints, job transitions, AI-credit entries, and audit traces around each run.
Recovery and accounting rely on durable events rather than a single in-process request.
Trade-off
The job model and ledger add storage and reconciliation work to the enrichment path.
Capability and stack
Capabilities describe the work the system can perform. The stack notes explain why each technical layer is present.
Enabled capabilities
- CSV and XLSX imports plus selected Google Docs and Google Sheets sources
- Dataset normalization, profiling, field suggestions, and typed per-row enrichment
- Workspace and web evidence with optional search, scraping, and isolated script tools
- Read-only Ask Data tools across workspace datasets
- SSE progress, cancellation, recovery, stale-job claiming, and checkpoints
- CSV, Google Sheets, and Google Docs exports with AI-credit and audit ledgers
- Workspace scoping and encrypted OAuth refresh-token storage
Stack rationale
Runtime
Expose typed import, dataset, enrichment-job, Ask Data, and export contracts.
TypeScript · Bun · Hono · Zod/OpenAPI
Orchestration
Checkpoint per-row workflows and persist normalized data, jobs, usage, and audit state.
LangGraph · Cloudflare AI Gateway · PostgreSQL · Drizzle
Data interfaces
Bring selected tabular and workspace sources into a common profile and export reviewed results.
CSV · XLSX · Google Docs · Google Sheets
Tools and operations
Gather configured evidence, isolate transformations, stream progress, and account for AI use.
web search · web scraping · isolated script sandbox · SSE · AI usage ledger
Safeguards and recovery
Reliable behavior includes the unhappy path: how faults are detected, contained, retried, or handed back to an operator.
Failure runbook
- 01
- Condition
- A worker stops while an enrichment job is still active.
- Detection
- Job claiming compares persisted status and activity timestamps with the stale threshold.
- Response
- Claim eligible stale work and continue from durable checkpoint state without treating streamed progress as the source of truth.
- 02
- Condition
- A row result omits a requested key or returns a value of the wrong type.
- Detection
- Runtime output validation compares the result with the selected field contract.
- Response
- Record the validation error and prevent the invalid row result from entering an export.
- 03
- Condition
- The operator cancels an active enrichment run.
- Detection
- The worker observes the persisted cancellation state between checkpointed units of work.
- Response
- Stop remaining row work, persist the terminal cancellation, and retain completed state for inspection.
- 04
- Condition
- A configured evidence provider or optional tool is unavailable.
- Detection
- The typed tool boundary records a provider, search, scrape, or sandbox failure.
- Response
- Retain the failure in the audit trail and leave unsupported fields unresolved rather than fabricating evidence.
Principles in practice
Start with the workflow
A typed graph separates import, profiling, field selection, per-row evidence work, validation, and export.
Show the sources
Workspace and web evidence remains associated with row enrichment and its audit trace.
Put rules in code
Code owns requested keys, output types, tool configuration, sandbox isolation, and AI-credit accounting.
Keep people in charge
Operators choose sources and fields, can cancel work, review results, and decide what to export.
Limit access by default
Workspace scoping, read-only data tools, selected document access, and encrypted refresh tokens limit access.
Plan for things to go wrong
Durable checkpoints, stale-job claiming, cancellation state, SSE progress, and audit entries expose and recover long work.
// validation signal
What was checked—and what was not.
Validation checks every requested output key and type before results move to export.
Search, scraping, sandbox, and scheduling behavior remains configuration- and provider-dependent; no production-scale claim is made.
Method
Typed runtime validation checks every row-output contract while persisted job and audit records expose how the result was produced.
Checks performed
- Every requested enrichment key is present before a row is accepted
- Every returned value matches the selected field type
- Tool use, evidence, usage, and job transitions remain attributable in the audit path
- Workspace scope and selected-source boundaries are enforced across import, Ask Data, and export
Outside this claim
- The available evidence does not establish production throughput or the accuracy and availability of external providers.
- Search, scraping, sandbox, and in-process scheduling behavior remains configuration-dependent.
// from difficult workflow to working product
Have a workflow like this?
We can help define the product, decide where AI adds value, and take it through a tested implementation.