It doesn't seem to matter whether I round to 3 decimal places or if I just let the whole number rock.
For simplicity, I'll use an example with Ms. Fortune doing 5LP~LP > 5LK > 5MK > 5HK at x1.00 damage with only using up to 3 decimal points for the scaling itself, and this is what I get in excel:
Code:
Scaling - Base DMG - Calculated Dmg (Rounded Down) - Calculated Dmg (no rounding) - Actual Damage
0.875 270 236 236.25 236
0.766 360 275 275.76 276
0.670 270 180 180.90 180
0.586 315 184 184.59 185
0.513 540 277 277.02 277
I have no idea why it rounds down on the 180.9, but rounds up on the 184.59 and the 275.76.