guidelines

My guidelines on Fundamental empirical understanding-based deep learning research.

I do not believe in rigid rules, yet, I do have a set of guiding principles. My collected research guidelines contain several parts about doing research. They involve: research organization, writing, the storyline, meetings, presenting, posters, reviews/rebuttals. Please read them, if you are doing research with me. Individual sections are available here:

• Research organization. About organizing the research.
• Writing. Common writing pitfals. Please check these writing guidelines before asking for my feedback .
• The storyline. The beating heart of empirical understanding-based deep learning research. See also this storyline visualization.
• Research meetings. How to organize research meetings and make the most of your time with your advisor.
• Presentations. Guidelines and common presentation pitfals.
• Posters. Guidelines and common presentation pitfals.
• Reviewing and rebuttals; on reviewing and writing a rebuttal/response answer to a review. Here are some examples of such a rebuttal: Example 1, Example 2, Example 3.

Research paper template. 

Fill in this paper_skeleton when writing a research paper. For everybody who is writing a paper or thesis with me. This template helps to organize our thoughts and adds structure to the writing process. Here is an example skeleton for one of my papers.

Grant proposal writing.

My view on individual grant proposals as a presentation called Fairytales.. ..and successful individual grant proposals.

Other research guidelines

• Reasons to do a PhD. A PhD is a significant commitment that can offer unique opportunities for intellectual growth, research training, and career development - but it’s not for everyone and shouldn’t be. Here’s a couple of models of when a PhD can be valuable

• Paper writing. The Why and How of paper writing: How to Write Papers

• Write/review/rebut.  Excellent advice on Writing a paper, on reviewing, and on how to rebut reviews.

• Deep Learning.  Are you training deep networks? Here is a diy teaching library. Hyper parameter tuning is then essential but can be a time sink and a source of frustration. Here is one great resource: Deep Learning Tuning Playbook, and another one is A Recipe for Training Neural Networks.

• Experiment tracking.  The Weights and Biases tool for organizing and tracking network training is the key to retain your sanity while running deep learning experiments.

• Papers with code.  Reproducing published research from scratch can be quite difficult and also time consuming. Here is a great list of papers and benchmarks that have code available: Papers with code.

• Science in deep learning.  Is deep learning science? The answer might depends on what you do. One indicator for doing science is that science aims to advance our knowledge, which makes it a core property to identfiy which parts already existed, and which part is advanced by your work (ie: how and what is “Related Work”). See The differences between tinkering and research.

• Scientific virtues.  Science depends on tools, methods, math, programming, etc. Yet, for doing science, the attitude is perhaps equally important: The Scientific Virtues argues for: Arrogance, Laziness, Carefreeness, Beauty, Rebellion, Humor.

• Do good research, run a great lab.  Excellent advice about what is important in doing research.

• Exploring literature.  The ResearchRabbit tool helps for literature search. Referral code: vKNw521

• What is a good paper?  Prof. Bill Freeman explains the mores in our field in How to write a CVPR paper (PDF).

• Writing for ML.  The importance of writing through Heuristics for Scientific Writing (a Machine Learning Perspective).

• Rebuttals.  How to write a rebuttal to paper reviews: How we write rebuttals.

• What is a paper?  Papers have two parts: 1. A technical contribution and 2. a motivation why the paper is important: Technical paper rebuttals aren’t just for factual errors.

• Related work.  Excellent advice on How to write the Related Work section of a scientific article.

• Deconstruct your research ideas (NeurIPS talk).  Strive to understand; clear insight make things simpler: The Importance of Deconstruction.

• Stress in research.  A series by Charles Sutton about research stress.

• Stupidity.  In research, by definition, we don’t know what we are doing: If we knew the answer we needn’t do the research. This is quite a different mindset from ‘reproducing’ knowledge, as often measured by course grades. See The importance of stupidity in scientific research.

• You and your research.  Richard Hamming (from the Hamming window) has good advice on You and your research, there is also a video available.

• Research results.  Obtaining research results can be quite frustrating: What I Wish I Knew as a Graduate Student.

• What is important in academia.  An academic should give, educate and inspire: Looking back: What was important?.

Tools

• Figures.  Try to script all graphs/figures that you create. Yes: All. Your adviser may ask for a completely different version of a figure, and automating it prevents lots of manual re-doing. I prefer Matplotlib; it can output high-quality PDF figures and graphs that can directly be included in pdflatex.

• Manual figures and posters.  If you really cannot automate the figure, and you really think you just need 1 version, then I recommend Inkscape for high quality vector graphics (which also export to PDF).

• Version control.  Use git for version control of your latex documents but also of your code