Progress

2026-02-21 16:03 - T-001

Overview: Restructured root source data into raw/ and clean/ directories.

Completed:

• refactor(pipeline): move xlsx and PDF source files into source/raw
• chore(pipeline): add source/clean/.gitkeep

Files:

• source/raw/
• source/clean/.gitkeep

2026-02-21 16:03 - T-002

Overview: Updated all source config YAML paths to the new raw directory.

Completed:

• refactor(pipeline): rewrite source file paths to source/raw/* in yearly YAML configs
• test(pipeline): verified sync load resolves updated paths

Files:

• source/config/ions-2023.yaml
• source/config/ions-2024.yaml
• source/config/ions-2025.yaml
• source/config/ions-2026.yaml

2026-02-21 16:03 - T-003

Overview: Added Python workspace prefix and root directory.

Completed:

• refactor(*): add @python/** to workspace package globs
• chore(analytics): create root @python/ directory

Files:

• pnpm-workspace.yaml
• @python/

2026-02-21 16:03 - T-004

Overview: Migrated dbt project from @packages/transform to @python/analytics.

Completed:

• refactor(analytics): copy dbt project files into @python/analytics
• refactor(analytics): rename project package to atlas-analytics
• test(analytics): ran uv run dbt debug and uv run dbt run successfully
• chore(analytics): remove legacy @packages/transform

Files:

• @python/analytics/pyproject.toml
• @python/analytics/profiles.yml
• @python/analytics/models/

2026-02-21 16:03 - T-005

Overview: Scaffolded new @packages/pipeline package with Vitest support.

Completed:

• feat(pipeline): create package manifest, tsconfig, and eslint config
• chore(*): add vitest to workspace catalogs
• chore(*): run pnpm install to refresh lockfile

Files:

• @packages/pipeline/package.json
• @packages/pipeline/tsconfig.json
• @packages/pipeline/eslint.config.mjs
• pnpm-workspace.yaml

2026-02-21 16:03 - T-006

Overview: Migrated sync load sources into pipeline package and merged DuckDB helpers.

Completed:

• refactor(pipeline): copy config/seed/load modules from sync package
• refactor(pipeline): merge raw-table helpers with Duck query client into shared duck module
• test(pipeline): ran pnpm --filter @packages/pipeline test:type

Files:

• @packages/pipeline/@source/config.ts
• @packages/pipeline/@source/duck.ts
• @packages/pipeline/@source/load/csv.ts
• @packages/pipeline/@source/seed.ts

2026-02-21 16:03 - T-007

Overview: Migrated format sources into pipeline package and fixed import paths.

Completed:

• refactor(pipeline): copy format report/types/sections into @source/format
• refactor(pipeline): repoint all duck imports to merged pipeline duck module
• test(pipeline): reran pnpm --filter @packages/pipeline test:type

Files:

• @packages/pipeline/@source/format/report.ts
• @packages/pipeline/@source/format/index.ts
• @packages/pipeline/@source/format/sections/

2026-02-21 16:03 - T-008

Overview: Added extract and validate placeholders for future implementation.

Completed:

• feat(pipeline): add extract() stub module
• feat(pipeline): add validate() stub module

Files:

• @packages/pipeline/@source/extract/index.ts
• @packages/pipeline/@source/validate/index.ts

2026-02-21 16:03 - T-009

Overview: Built unified pipeline CLI with sub-command dispatch.

Completed:

• feat(pipeline): implement command parser and dispatcher for extract/load/validate/format/seed/run
• test(pipeline): validated load/format/seed/extract/validate and unknown-command behavior
• test(pipeline): reran pnpm --filter @packages/pipeline test:type

Files:

• @packages/pipeline/@source/index.ts

2026-02-21 16:03 - T-010

Overview: Added Vitest configuration and initial unit tests.

Completed:

• test(pipeline): add Vitest config and tests for number utilities and CLI argument parsing
• refactor(pipeline): guard CLI execution so imports are test-safe
• test(*): run pnpm --filter @packages/pipeline test and verify pnpm test includes pipeline tests

Files:

• @packages/pipeline/vitest.config.ts
• @packages/pipeline/@source/tests/number.test.ts
• @packages/pipeline/@source/tests/config.test.ts
• turbo.json

2026-02-21 16:03 - T-011

Overview: Updated consumers to use pipeline package; dashboard type-check remains blocked by pre-existing errors.

Completed:

• refactor(present): migrate imports from @packages/format to @packages/pipeline/@source/format/*
• refactor(*): update present/dashboard package dependencies to @packages/pipeline
• test(present): pnpm --filter @services/present test:type passes

Blockers:

• pnpm --filter @services/dashboard test:type fails on existing dashboard route type issues unrelated to this migration

Files:

• @services/present/package.json
• @services/dashboard/package.json
• @services/present/@source/index.ts

2026-02-21 16:03 - T-012

Overview: Rewired root scripts and turbo task config to the unified pipeline package.

Completed:

• refactor(*): add pipeline, extract, sync, validate, and updated format scripts in root package.json
• refactor(*): route sync:seed through pipeline seed sub-command
• refactor(*): add test task in turbo.json
• test(*): verify root sync, extract, validate, and format scripts execute through pipeline

Files:

• package.json
• turbo.json

2026-02-21 16:03 - T-013

Overview: Updated agent documentation for new package structure.

Completed:

• docs(*): update root architecture and command docs for pipeline + analytics workspaces
• docs(pipeline): add package-level AGENTS guide for unified CLI and source layout
• docs(analytics): update analytics AGENTS guide from old transform naming

Files:

• AGENTS.md
• @packages/pipeline/AGENTS.md
• @python/analytics/AGENTS.md

2026-02-21 16:03 - T-014

Overview: Removed legacy packages and ran full verification; monorepo still has pre-existing global quality blockers.

Completed:

• chore(*): remove @packages/sync and @packages/format
• test(*): run pnpm install, pnpm build, pnpm test, pnpm run sync, uv run dbt run, and pnpm run format

Blockers:

• pnpm test:type fails in @services/dashboard with existing TypeScript errors
• pnpm lint fails in @core/ai with existing lint violations

Files:

• @packages/sync/ (deleted)
• @packages/format/ (deleted)

2026-02-21 16:51 - T-002

Overview: Realigned source directory naming to @source/ and reconciled runtime path resolution with plan intent.

Completed:

• refactor(pipeline): move source/{raw,clean,config} to @source/{raw,clean,config} and update YAML file paths
• fix(pipeline): update project-root detection to require @source/config in pipeline config loader and DB seed helper
• fix(pipeline): correct load default DB path derivation to keep atlas.db at repository root
• docs(pipeline): update active docs to reference @source/config

Decisions:

• YAML path values starting with @ are quoted to remain valid YAML scalars.

Files:

• @source/config/ions-2023.yaml
• @source/config/ions-2024.yaml
• @source/config/ions-2025.yaml
• @source/config/ions-2026.yaml
• @packages/pipeline/@source/config.ts
• @packages/db/@source/seed.ts
• @packages/pipeline/@source/load/csv.ts
• @packages/pipeline/AGENTS.md
• @plan/quickstart.md