I use to go back when I could drink and it was a tough session (even harder than international rugby matches) but I always booked the following day off as a precaution. I like Gold Cup day although it does become manic with the sheer numbers but to be fair the bars are well organised and staffed and so getting a refreshment isn't too bad. I wouldn't eat in Cheltenham after the racing as it is over crowded, slow service, rushed and poor quality. I normally look to drive home (well the designated driver) and sto at a decent pub at least an hour away
Harder than an International day out? Not a chance!
Just come back from a weekend in Edinburgh for the Calcutta Cup match and a trip to Cheltenham is a walk in the park compared to that.
If you want a real big weekend, try the Hong Kong 7s. I went once and survived four 19 hours days on the beer - barely. I'd never go back. I'm too frightened.