So true. As someone who struggled with the driver when I first started I would suggest that being a demon on the green is irrelevant if you are 3 / 5 off the tee, duffing it 20yds etc.Someone will be along in a minute to say you only hit 14 drivers but 30odd putts. Therefore putting is more important.
If you're spraying it everywhere off the tee and regularly taking 3 off the tee then most definitely sort that out.
However if you can drive relatively ok with a regular shape even only 200-225 and you're not putting yourself in the deep stuff several times a round then putting becomes something more to work on.
Edit to say...
If its a case of being good at one and bad at the other, chose to be a good driver. If its a case of being good at one and average at the other, chose to be a good putter