Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Dealing with Large Amounts of Data
Message
From
10/07/2008 14:55:20
 
 
To
10/07/2008 13:37:19
General information
Forum:
ASP.NET
Category:
Class design
Environment versions
Environment:
C# 3.0
OS:
Windows XP SP2
Network:
Windows 2003 Server
Database:
MS SQL Server
Miscellaneous
Thread ID:
01330296
Message ID:
01330328
Views:
18
This message has been marked as a message which has helped to the initial question of the thread.
Hi,

>I do a data conversion work and I frequently have to work with and view large data files. Millions of records and the tables can be a few gigabytes.
>
>I need something like a VFP browse window so that I can get an overall feel for what's in the table. Paged views just don't give you the same feel for what you are working with.
>
>Everything I've seen so far in .NET that would do this appears to want to want read all of the data into memory in order to display it in a grid. But because these files can be so large I would prefer to cache and fetch as needed.

Maybe Skip() and Take() in Linq to SQL ?

>
>I am currently writing my own workaround for this, but before I go too far I'm wondering if I missed anything and there is an easy way to do this that I did not see?

Best,
Viv
Previous
Reply
Map
View

Click here to load this message in the networking platform