Sounds like a jobsworth and a prat. If no one is behind you then where is the harm? If you are taking chunks out of the course that is another matter but your description does not follow that. I played on my own last night for 9 holes. I was the only person on those 9 holes. If I hit a shot badly I would drop another ball and take it again. I kept checking that I was not holding anyone up but there was not a person in site. The bloke was a muppet but you were right to walk away. Nothing to be gained trying to have a conversation with people like that.