Hi all of you VFP die-hard developers,
I have been a foxpro traditionalist. Using it from 1985 as Foxbase onwards. The Fox never let me down over the years. But for sure I have been looking for a replacement over the last four years or so. I am now getting into the process of rewriting my last application. Last app? Yep, getting older, though pension time has not arrived for me!
I was looking to replace VFP9 (of course supplemented by a host of components, most of them COM-based). I found nothing that REALLY matches my requirements. Why:
1- I am looking for an interpreter-based readable language. My experience path: APL, then fox+spreadsheets then Smalltalk, later python and a few others "scripting languages" as they said... Except for plain K&R C, I was never comfy with compilers engines except at the margin i-e fox flls,
2- I want insanely fast workstation-based data mangling functionality without resorting to C/C++ coding, in order to replace the good ol' rushmore engine,
In view of what happened to the Fox and a couple of other other tools,
3- I'll accept no "proprietary lockup" à la MS, Oracle and others, except for a few decently replaceable components.
Enters thee new "open source" rushmore : the current version of duckdb used in a "python+pyarrow+numpy" combo. A combination of technologies that is certainly targeted more to ML-AI people than towards VFP developpers.... But it can really fit the bill of a VFP-er who would want crazy levels of performances at workstation levels (and not only for gaming..). I mean using the processing power of your CPUs on a massively parallel scale.
Won't say a lot. It is still early beta. But it is a beast to compare to if you are looking for what I was searching as well: sheer data management speed on workstations.
What I am using it for: complex data mangling on tenths to hundreds of thousands of records loaded immediately from file resources (no csv files, no oldish dbfs, à la mode column-oriented data storage formats such as parquet and feather). Could certainly handle much heavier data loads.
No requirement for SQL services, everything is workstation-based. And it shines from python in the way fox did really shine during the late 90s.
Why does this "python+duckdb+pyarrow+numpy combo" perform so well?
1- use arrow technology for dataload/datasave ie feather/parquet format (more on the arrow website),
2- feed the database engine in-memory from this arrow datastorage,
3- handle the data mangling in a two fold way, using the duckdb engine for raw sql and supplementing what is missing with adequate numpy+cython massaging when needed.
What is so special about this combination and is a source of performance, these tools are able to share large amount of organized "memory" - columnar data format and vectorized query execution engine - in a very smart way. That's especially the case between arrow formats (parquet and feather) and duckdb.
https://duckdb.org/https://duckdb.org/docs/why_duckdb.html#duckdbisfasthttps://arrow.apache.org/docs/index.htmlhttps://arrow.apache.org/docs/python/index.html It indeed makes it possible to load massive quantities of data in an "in memory" db in a fraction of a second is great. IMHO you will never be able to achieve this level of speed from a traditional record-orientated relational db engine. For obvious technical reasons.
Of course, you will certainly miss the plain simple UI support that VFP did bring us over the years. I expect rewriting VFP forms as PYQT/Pyside ones will be a nasty and long experience. I'll missing the easy deployment (using a VFP+boxedapp combination that produce fully standalone apps). But that's time for a change. At last!
To-day python is covering pretty every area of software effectively. I am very enthusiastic about it. This python big data ecosystem is IMHO definitely great new workstation-centric tech for the 20s (2020 ones)... Duckdb is indeed worth a try in 2022!
Daniel
