Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
BIG!!!! Pictures in General fields and VERY slow
Message
From
14/12/2003 15:24:30
 
 
To
13/12/2003 20:12:23
Hilmar Zonneveld
Independent Consultant
Cochabamba, Bolivia
General information
Forum:
Visual FoxPro
Category:
Pictures and Image processing
Miscellaneous
Thread ID:
00858269
Message ID:
00858994
Views:
16
> With this partitioning scheme, how do you link the images to other data records -
> that is, do you use some key value to find them, comparable to the filename,
> if you store each image in a separate file?

Yes. Lookup is via the primary key of what the images represent (medical prescription forms), which is a composite key that consists of the billing month and a running number within the month. App-level code never plays with the image tables directly, instead it asks a global image manager object for the image for a certain key, and the image manager in turn relays the request to the appropriate candidates among whatever image sources are currently configured or mounted. The images are passed around as objects with an OS-level memory handle for the raw image data and a couple of other attributes in properties (time created and so on), to provide isolation from the actual physical storage.

When an image source is 'mounted' then it is represented by an object that provides two services to the manager: determine min/max for the keys in that image source, and return an image by key (if it exists). The manager keeps an array of these objects and it maintains an in-memory cursor with min/max info obtained from the objects in the array, which allows quick, indexed access to all image sources that could possibly contain a given key/image. E.g.:
define class CImageQuelle_MultiIMG as session

   dimension a_Quellen[1]

  * ...

   procedure ErzeugeLeerenQuellenCursor_
      * ...
      create cursor w_Quellen ( ;
         min_monat  i, ;
         max_monat  i, ;
         min_beleg  i, ;
         max_beleg  i, ;
         obj_index  i  )
      * ...
      index on bintoc(MAX_MONAT) + bintoc(MIN_MONAT) tag MAMO_MIMO

   * ...   

   function FindeImage (nMonat, nBeleg, roImageInfo)
      seek bintoc(m.nMonat)
      scan while between(m.nMonat, MIN_MONAT, MAX_MONAT)
         if between(m.nBeleg, MIN_BELEG, MAX_BELEG)
            if this.a_Quellen[OBJ_INDEX].FindeImage(m.nMonat, m.nBeleg, @m.roImageInfo)
               return .t.
            endif
         endif
      endscan
      return .f.

enddefine
The manager doesn't care whether the image sources represent .DBFs with image data in memo fields, or directory trees with files, or whatever. OK, I'm lying. There are actually two managers - one for the fast, .DBF-based sources and one for the slow, file-based ones, and both report to the real global image manager which tends to ask the fast source first. ;-)

In our case .DBF memo retrieval outperforms file retrieval by about one order of magnitude for completely random accesses over the whole database (which happens only in my regression test benchmark *g*) and by about two orders of magnitude if accesses are localized. The main reason is probably that each separate image file is a separate OS-level object, and opening/reading/closing a file over the network requires several roundtrips. The image tables are already open and the only network traffic is for transferring CDX/DBF/FPT data. For localized accesses Fox will already have cached most of the CDX and DBF data most of the time, so only the memo data needs to travel across the network. Similar considerations may apply to other high-latency media like CD-ROMs.

Of course, if the primary key does not lend itself to easy partitioning then things get more complicated. It may then be necessary to introduce a surrogate key and a table that maps the PK to the surrogate, similar to the PK-to-filename mapping required for file-based image storage.

> Interessant.

¡Gracias! *g*
Previous
Next
Reply
Map
View

Click here to load this message in the networking platform