Plateforme Level Extreme
Abonnement
Profil corporatif
Produits & Services
Support
Légal
English
Distributing and using VERY large tables.
Message
 
À
05/08/2001 18:30:04
Gerry Schmitz
GHS Automation Inc.
Calgary, Alberta, Canada
Information générale
Forum:
Visual FoxPro
Catégorie:
Base de données, Tables, Vues, Index et syntaxe SQL
Divers
Thread ID:
00539842
Message ID:
00539976
Vues:
10
>>The data comes in as all text. Even “numeric” data is in text format. Getting rid of any padding doesn’t seem to help much because I had created an ASCII Delimited Text file from my 2 Gig dbf and the text file was almost 2 Gig itself.
>
>Perhaps I did not make myself clear.
>
>(Text) "Numerics" can be "packed". For example, if the Zip code is a separate Field, of say 9 or 10 digits, it can be converted to a 4 byte binary field; eg.
>
>X = BINTOC( VAL( "123456789" ) )
>
>You've just reduced the storage requirements for zip codes by over 50%; the same applies for other numerics.
>
>I'm not familiar with the contents of your text files, but if street names, building names, etc. are being duplicated, they should be spun off into a separate table and reduced to a simple binary key in the main table.
>
>You can't assume that other vendors have simply taken some "off the shelf" solution; you may have to develop a custom pack-and-search engine (as they might have), or license one.
>
>>Do you know of any other database who’s data is this compressed? Paradox (DB) or whatever?
>
>Paradox extensions are .DB, and their indexes are "huge", often bigger than the data.
>
>Paradox supports binary fields, as does VFP. You cannot simply work with only text if you hope to reduce storage requirements.

Gerry,

BINTOC() and CTOBIN() are worth looking into and normalizing the table as much as possible. The only thing I would have to do is be careful of the type of text "numbers" that I was converting because leading zeros are important in zipcodes. The original zipcode "00212" would come back as 212.00 but I could just add leading zeros in this case.

I'll have to play with this to see how much it will help. I appreciate you pointing me to this possibility.

Ed
Précédent
Répondre
Fil
Voir

Click here to load this message in the networking platform