I worked so damn hard but "IT JUST DOESN'T WORK!" How can I unblock myself quickly and make good progress toward the goals?
Below I compiled a list of tips that I found useful.
Forget about all the technical difficulties for a moment. Imagine you finish your project successfully, would you find the outcome exciting?
If not, drop the project. Yep, just drop it. Free up your time to work on important problems.
drop.mp4
Say your project involves three steps: A -> B -> C.
First, assume that you have perfect output of B and work on the step C.
Next, assume that you have perfect output A and work on the step B and so on.
In the end, you will have a fully working method.
Okay, this is weird. WHY?
Because you get to
- see final outcome early
- measure the performance upper bound
- focus on each task with perfect inputs without distraction
- figure what are needed to achieve the desire results.
backward.mp4
Design toy examples that capture the essence of your problem. They are sufficiently simple so you can focus on the core problem.
It's also often helpful to construct/synthesize such toy examples so that you have access to all the ground truth in all the steps.
toy-example.mp4
Don't know where to start? Start with trying out baseline methods on your problem.
It helps identify limitations of the state-of-the-art. If they work perfectly well, why do you need to work on this problem?
Finding specific gap helps motivate your work.
baseline-first.mp4
If your method does not work on simple/trivial cases, how could you expect it to work on unconstrained, real-world cases?
keep-it-simple.mp4
When doing experiments, change exactly ONE thing at a time. This helps you understand what the results mean.
one-thing-at-a-time.mp4
Do not use full-scale experiments (that may take weeks to complete) as the only way to validate your ideas. Run smaller-scale/simpler experiments with short turnaround time so you get to iteratively refine your ideas a lot faster.
proxy.mp4
If you find that you need to do the same task twice, write a script for that.
Your future self will thank you.
automate-everything.mp4
You cannot debug what you cannot see. Investing time in visualizing your inputs/intermediate steps/outputs is definitely worthwhile!
visualize-everything.mp4
Instead of always eyeballing a few results on your own, identify a couple of quantitative metrics for your problem and let them guide your exploration.
quantify-success.mp4
Plan your experiments so that your machines still work for you while you are not working.