>>Thinking of Duckdb as some sort of modern day VFP rushmore and parquet as equally to-day's dbfs ain't stupid. Good news: the size limits are nowhere to be found. And, even without high-end index management, performance levels can be impressive on 'columnar workloads' (ie "accross the board"). We are, here as well, a generation forward especially in terms of ability to put parallelism at work.
So am I correct to imagine Duckdb as a temporary repository - like a VFP cursor- for data munging, rather than a data store? Except available on Linux and other ecosystems as well as Windows?
FWIW, Mr Chen's latest effort extends maximum dbf (and cursor) size into the thousands of terabyte range- bigger than any currently available hard drive. But still insecure, unfortunately.
"... They ne'er cared for us
yet: suffer us to famish, and their store-houses
crammed with grain; make edicts for usury, to
support usurers; repeal daily any wholesome act
established against the rich, and provide more
piercing statutes daily, to chain up and restrain
the poor. If the wars eat us not up, they will; and
there's all the love they bear us."
-- Shakespeare: Coriolanus, Act 1, scene 1
