Advanced Neural Network Engineering Plan

Phase 0: Setup — "Own Your Training Stack"

Goal: Be able to quickly train models on real datasets, with flexible architectures.

Resources:

• FastAI Course Part 1 — only the part about training pipelines (skip beginner lectures).
• Exercise:
  • Build a small training pipeline that takes a dataset, builds a PyTorch model, trains and logs experiments (even if it's just CSV + Tensorboard).
• Deliverable:
  • You can spin up any dataset + model + optimizer + scheduler in <30 minutes.

Phase 1: Foundation — "Architectural Breadth"

Goal: Know the building blocks of real architectures (not toy CNNs), know when/why to modify them.

Resources:

• Dive into Deep Learning (D2L) — Chapters 5–10 only. (skip chapters before 5 if you're strong on basics)
• Keras Code Examples — especially under "Computer Vision" and "Structured Data" sections.

Exercises:

• From D2L: Reimplement the examples at the end of each chapter.
• From Keras:
  • Pick 5 examples.
  • Modify architectures (e.g., double the depth, change activations, add skip connections) and re-train.
  • Write 3-paragraph reports explaining whether and why performance improved or not.

Deliverable:

• You know what happens if you deepen, widen, or modify blocks of a NN.
• You can explain residual connections, attention, depthwise convolutions, etc.