Here I should first clarify that "systems" here refers to computer science topics such as operating systems, database systems, network systems which tend to be more software implementation and empirical evaluation oriented in general.I concur with your emphasis on the importance of mathematical analysis:
- "you must learn how to add rigor to your work through mathematical analyses for your work to be respectable for graduate-level researchers."
This was my response to him:
I think a good starting point for learning theory is learning how to mathematical model real-world problems:
* Check out this very basic book on mathematical modeling (aimed at HS/undergrads, I believe) as a starting point.
* One article that guided my early efforts at doing some mathematical modeling was Hal Varian's "How to Build an Economic Model in your Spare Time"
* I attempted once to write a short "tutorial" on how to apply mathematical modeling to wireless sensor networks that you might find useful as a starting point in thinking about modeling:
* To get a bit deeper, you do need to learn to construct proofs. Polya's "How to Solve it" .. is indeed a good starting reference.
* I took two courses at Cornell that really taught me to prove things:
1. A course on real analysis in the math department (something like this MIT Course on Real Analysis; a good book for it is Strichartz's "The Way of Analysis": )
2. A course on analysis of algorithms taught by Jon Kleinberg; the notes for the class I took got turned into a great book called "Algorithm Design."
Like with everything, the trick to learn how to do more theoretical research is to start small and practice - build some simple models or prove some simple known things first then work your way towards something more substantial...I then added afterwards in a follow-up note:
The other thought that occurred to me is that in my own work, often my students and I base our approach on the analysis in a similar paper.
Reading the classic papers on analysis of CSMA, TCP fluid modeling, Network Utility Maximization, etc. can be a starting point. Ask yourself if you could modify the analysis or assumptions a little to treat a slightly different problem. This will also give you more practice in doing analytical modeling and proofs.
Last but not the least, it could also help to collaborate and discuss with others that are more theoretically oriented to gain insights on your own problem...