Colin L
Tour Winner
Agree, that is what the rule says. In reality, it is not very easy to accidentally move your ball during the downswing in a way that gets the 9.4b penalty but Stroke/1 mentions one possibility - clubhead separates from the club during downswing, player successfully stops the downswing short of the ball, but the clubhead moves the ball. This is no penalty in the teeing area, but is if the ball was in play - this is the old D14/4 for the historians.
As well as Colin's putting green reference, there is also an exception for accidentally moving your ball in play in the teeing area (when ball has managed to return to the teeing area).
I also have another interesting ruling in this space. Player hits way fat, missing the ball, but the divot moves the ball. Ruling: stroke counts but no penalty and play as lies.
At the time of writing I couldn't actually think how you could possibly move your ball accidentally during the downswing but all of a sudden there comes to mind one of my friends who, because of back trouble takes a Happy Gilmore step into his hitting the ball. As a former hockey player he is rather good at this. I suppose there could be a situation in which that step forward could dislodge a ball in longish grass. Not exactly a common occurrence, though.
The divot moving the ball seems similar to a good bunker shot where the clubhead does not strike the ball or hitting a fence in order to move a ball on the other side or hitting a loose impediment behind your ball.