>>My application hasn’t been designed yet. This data problem is my first >hurdle. It will have to be distributed on a single CD, though, so I need to >get the data down to no more than 650 MB. If need be I can write a dll with >something else like VB or C# if that’s what it takes to get the data down and >read it. Normally I would look at 6 Gig of data and say there is no way to >knock it down to 1/10th it’s size and still have it accessible as live data >but since I see that the other vendors have done this, then it must be >possible. I just don’t know what format they are putting the data and what >language/engine they are using to access it.
>
>Are they using Windows? Are they using every field from the text file in the application or just the data they need?
>
>On a windows platform, I don't think you can get any faster than FoxPro. I work with Postal Service data quite often. I don't have a size requirement like 1 CD but if I did I would question the wisdom of this requirement. We keep 20 Gig of information on a dataserver for several obvious reasons, the most important being integrity. For the purposes of speed and size I would create aggregate tables and lean and mean lookup tables for specific uses. I would only distribute such things on CD if it had a disclaimer that it is not valid data once it leaves the server environment on a disk.
>
>If someone has 6+ Gig compressed down to 650 meg and are using it in runtime as a data source on a Windows platform, it has to be excrutiatingly (sp) slow. I would be surprized to see acceptable performance even on a high powered HP-UX box since it is on a CD anyway.
>
>My $2

Eric, et.al.,

My potential competitors are using Windows and they are accessing this data on-the-fly in compressed format. The speed is very fast all things considered. If you are familiar with it, I'm refering to CASS software from such vendors as Mailers+, AccuMail, etc. The data on those CDs represents what would be 6 Gigs.

I just now discovered how they are doing it. The Post Office has developed their own API. They said that they don't advertise it because they are not in the software business. They compress the data and then the developer uses the Post Office's API to access the data. The bad part is that they license the data with the API at $5000/year and insist that all CDs be burned by their contractor at a cost of $6.50/disk. The data alone without their API would have only cost me $900/year with a royality of about $4.50/user that I distributed it to and I could have had my own CDs burned which usually only cost about $3.00/disk.

It looks like I'm going to have to go with the PO's API. I have a feeling that I could waste a hell of a lot of time trying to roll my own and may still fail in the end.

I want to thank everybody for their help and suggestions.

Ed