Here are some answers, more to come.

>- Does any of this devices have financial software (just a simple personnal account manager)?

The new Pocket PCs (released on April 19th) have Pocket Money.
Palms come with a simple expense application that that interfaces with MS Excel.
You can find other financial apps at www.palmgear.com but they will most likely cost.

>- Windows CE have what? Word? Excel? Powerpoint? I don't think so...

The current handheld (with keyboard) CE devices have Pocket Office, but the palm-sized don't. This changes on Apil 19th with the release of Pocket PCs from Microsoft. They have Pocket Office which includes (Pocket Word, Excel, Outlook, IE, Pocket Money). Although I can't find a reference, I swear it also included Pocket Powerpoint. For screen shots see
http://www.wiredguy.com/reviews/sr/pocketapps/index.htm

>Final question: what's your opinion about the best device to use? (i'm a VFP programmer and work also with SQL Server, i don't know if that can change your opinion...)

To leverage your existing skills, going with a Pocket PC might be best since you can use VB with SQL Server or Pocket Access. Palm is also building connections to databases like Oracle as well.
http://msdn.microsoft.com/cetools/