I think you can do it as three different UPDATE statement. I don't have SQL Server at hand to work things out. The sticker will probably be Process #2 which wants a record number. But you might be able to change the schema to get a replacement.
This is a off-the-cuff attempt at process #1 but I'm not sure the math is right.
UPDATE the_table
SET cum = (
SELECT CASE COUNT(*)
WHEN 1 THEN ren
ELSE SUM( ((1 + Cumprec) * (1 + Ren)) - 1) END
FROM the_table t
WHERE t.year <= the_table.year AND t.month < the_table.month )
-Mike