Learnings
The first step is to not be intimidated
There’s an abundance of resources (papers, blog posts, frameworks) online and sometimes you just want a single source of truth
Choose uni lecture courses to see if your level matches (Stanford, Cambridge)
The heart of DL is mathematics, but you need to be able to build it with computational tools. Try to get to the bottom of it
Textbooks are good, until they go too deep
“What I cannot create, I do not understand”– Richard Feynman.
Define a simple task, get the base dataset, and do it.
Focus on understanding the concept and then implementing a working version of it
Find infra tools that help you (weights and biases)
ALWAYS validate on something that a human can intuitively understand. The loss, accuracy metrics can be deceptive
Talking through with others help a lot
Resources and inspirations
Andrej Kaparthy’s Recipe for Training Neural Networks
Become one with the data
Set up the end-to-end training/ evaluation skeleton + visualise EVERYTHING + get dumb baselines. Even better if you deploy it (especially if you want to show people)
Overfit on the training set
Last step: Regularise, tune, ensemble