Hi all,
Should you have some interest on what kind of SQL-based performance you can reach from a recent engine such as duckdb, possibly what would be the most recent equivalent of
a modern version of an in-memory in-process engine à la foxpro in 2023:
The load time of a 720MB CSV file containing the lineitem table from the TPC-H benchmark is below 0.6 seconds for a 600 million lines table. Yep, below one second to load a beasty dataset straight from a csv!
That's possibly a recent smart Apple machine. But the performance is still mind-boggling on plain vanilla win10 machines...
More information on this muscular duck:
https://duckdb.org/More information on the kind of performance level you can reach:
https://duckdb.org/2022/11/14/announcing-duckdb-060.htmlThat's a game changer in terms of what you can reach as a SQL data mangler on a sheer workstation. Beware that you will extract performance from your multi-core cpu in the a way no pack-office package would. It will possibly get hot!
Daniel
