To be non materialistic does'nt mean rejecting in a block society pretexting that it is all too materialistic. Is it what the rimpoche means? Taking philosophy as a pretext to be lazy and casual in what one does? I do not see that much intelligence is at play in that sort of posture. And who could be such a fool as to believe the material aspects of existence need not be taken care of. The fact that one is lazy or one is a workaholic needs not and should not be connected to any kind of philosophy.
But the way one relates to what one does and how one approaches one's work seems to me very important. However seemingly important or unimportant the work , one's work has an impact on other people, an impact on nature, it fits into a much bigger whole.