Skip to content

ML Eng Lab — notebooks, systems, and reproducibility

1 · ML ENG LAB

Local notebooks. Remote Atlas execution. Explicit infrastructure contracts.

Core ML
Python Jupyter NumPy pandas PyTorch PyTorch Geometric scikit-learn

NLP and graphs
spaCy NLTK NetworkX

Runtime
Atlas Docker VS Code GitHub Codespaces

Engineering
NNx Papermill pytest Ruff GitHub Actions

ml-eng-lab is a portfolio of self-contained machine-learning notebook experiments built for local editing in VS Code and recommended remote execution through JupyterHub on Atlas's ML Engineering track. Unlike a loose notebook collection, each task declares its runtime needs in a checked infrastructure contract, keeping notebook dependencies explicit as the lab expands beyond JupyterHub. Narrative experiments, reproducible execution tiers, exact dependency pins, validation gates, and the reusable thekaveh-nnx toolkit evolve together.

Contributors can use Browser JupyterLab for mounted-workspace tasks or choose a local virtual environment, Docker, or GitHub Codespaces when Atlas is not the right fit. Host-native Ollama is the only approved Ollama source whenever a future task needs it; containerized Ollama is intentionally excluded. This makes the lab both a practical portfolio and a controlled environment for growing machine-learning systems without hiding operational assumptions inside notebooks.

1.1 Repository map

  • notebooks/ contains twenty-one active task directories and twenty-nine active notebooks.
  • notebooks/archive/ contains preserved Aug-2023 CodeXGLUE summarization experiments.
  • scripts/verify_repo.py is the fast structural, documentation, and notebook-surface verifier.
  • scripts/docs/ owns the three-surface documentation pipeline (manifest, transforms, renderers, checker).
  • Makefile owns notebook execution tiers and local validation targets.
  • infra/ pins Atlas; its ml-eng JupyterHub runtime is the default remote notebook kernel.
  • docs/ holds the documentation-tree portion of the manifest-declared canonical source set; root governance Markdown such as SECURITY.md completes that set.
  • .github/workflows/ contains CI and documentation publishing workflows.

The root README.md is the day-to-day entry point for contributors — it carries the task index, quick-start paths, and the standard make targets. Its opener is hand-authored and parity-guarded against this landing page; it is not a manifest-generated page. This documentation collection is the focused reference surface that complements the README.

1.2 Documentation surfaces

The lab maintains three synchronized documentation surfaces from a manifest-declared canonical source set so the three never drift. That set includes Markdown under docs/ and direct-root governance Markdown such as SECURITY.md:

Surface Source Rendered by Audience
Repository Manifest sources under docs/ plus root governance Markdown GitHub markdown rendering Contributors browsing the repo
Site generated/site/ MkDocs Material (mkdocs build) Public readers of the published site
Wiki generated/wiki/ GitHub wiki rendering Readers who prefer the wiki navigation

The manifest at docs/manifest.yaml is the single source of truth for the hierarchy, numbering, and page set. scripts/docs/build_docs.py consumes the manifest and emits both generated surfaces; scripts/docs/check_docs.py gates CI on self-containment, completeness, placeholders, and determinism. Each manifest page is written once; the site and wiki are transforms of that source, while the repository renders the source directly.

  • System & context view for the repository context, the system diagram, and the three-surface pipeline.
  • Atlas pin-bump and service-admission runbook for the infrastructure ownership boundary, native Ollama rule, and future service workflow.
  • Tabular classification — Iris MLP for the exemplar comprehensive deep-dive — the canonical walk-through of one notebook end to end (problem, math, architecture, code, results, pitfalls, extensions).

All twenty-one active task deep-dives are available under section 8. The three links above are a recommended starting path through that complete catalog.