If the punch time doesn't fall "near" the scheduled times (as defined by the "grace" and "rounding" boundaries), you could either leave them as they are (i.e. use the raw, unrounded time for computation), or apply some sort of rounding toward specific-sized increments (e.g. on 6, 10, 20, or 30-minute increments, with a "threshold" value specifying point at which values are rounded back/forward). Whatever the case may be, you should always record the original times and perform the time computation with the rounded values. Often the rounding rules for IN punches may end up being different from those applied to OUT punches. I've seen configurations where punches tend to round toward each other (i.e. rounding thresholds for IN and OUT are set so that IN tends to round forward, and OUT tends to round backwards), or away from each other (i.e. the rounding threshold for IN and OUT are set so that IN tends to round backwards and OUT tends to round forwards).
Additional rules may be required if you have shift/department/job changes being registered (e.g. punch OUT from one job and punch IN to a new one) -- in which case the OUT followed by a subsequent IN should be handled as a single punch (i.e. if the OUT and IN should move "together" -- neither "away" nor "towards" each other). Similar sort of rules might be involved when it comes to lunch or break punches.