Pocket milling climb and conventional

Think about how the tool is rotating and which direction the tool is pushed by the cutting force when it is cutting.
If the tool is cutting away from you (Y+), the cutting edge on the 'front' (the edge cutting fresh material) is moving left to right, trying to push the material right.
Thus the tool experiences a reaction and is being pushed left.

so, if there is any play in the system cutting clockwise around a part will tend to leave it a little large, and cutting anticlockwise will tend to leave it a little small.
If you cut a square and it comes out different dimensions in X and Y it means you have more play in one direction than the other.

your finding of coming out on-size it probably because your tool is not quite the size you put into the CAM and the offset happens to compensate for the backlash.

more explanation at

 

oops, this is the video i meant to link to