..
* Set the printer to the new options
DocumentProperties(hWindow, hPrinter, cPrinter, @cModifiedDevmode, @cModifiedDevmode, DM_IN_BUFFER+DM_OUT_BUFFER)

* try changing the above to:
DocumentProperties(hWindow, hPrinter, cPrinter, @cModifiedDevmode, @cOrigDevmode, DM_IN_BUFFER+DM_OUT_BUFFER)

i think the devmodes structures have to be unequal so the printer gets updated ..

* Showing printer settings dialog again to see if the settings took
* This dialog comes up with the tray settings I chose previously
DocumentProperties(hWindow, hPrinter, cPrinter, @cModifiedDevmode, @cModifiedDevmode, DM_IN_PROMPT+DM_IN_BUFFER+DM_OUT_BUFFER)

*DocumentProperties dialog takes the settings from the passed in DEVMODE,
*that's why the tray is the same as the one you selected previously.
*To verify that options are persisted to the printer settings try this
*instead
DocumentProperties(hWindow, hPrinter, cPrinter,0,0,DM_IN_PROMPT)