I appreciate par has nothing to do with handicapping (except for nett double bogey adjustments), and Course Rating is key.
However, to simply say "Par has virtually nothing to do with difficulty" can cause misunderstanding. Length of a hole almost certainly factors into difficulty, thus it is significant in course rating. A player arrives at a 460 yard par 4, I can almost certainly guess that they'll determine this as a difficult hole. More precisely, a difficult par 4. If the same hole was a par 5, there is a good chance they'll call it an easy par 5. Maybe even get excited about an opportunity to give themselves an eagle putt. Same hole, but different perception on difficulty, where par has a direct impact on that perception.
Courses in which CR is a lot higher than Par would generally be considered very tough and difficult courses (absolute difficulty at least, not necessarily relative difficulty between low and high handicappers). Courses were CR is much lower than Par would generally be considered easier courses. Difficulty is subjective to the person thinking something is difficult or not. Furthermore, Par isn't an arbitrary number, as it is clearly not based on random choice or personal whim. There are still general guidelines as to how par is set, and you would hope the group of people setting the par for each hole are competent and able enough to determine reasonable Pars on each hole. I have yet to play a 120 yard par 5 or a 600 yard par 3.