It does seem slightly odd that when a block of time is completely empty that they would add on to your slot. In a similar situation I think most people would take an empty slot. They can always verbally ask you on the 1st if you would like to join them or carry on on your own.