Plateforme Level Extreme
Abonnement
Profil corporatif
Produits & Services
Support
Légal
English
Why Processing Performance is Not Increasing ?
Message
De
16/02/2014 14:59:52
 
 
À
16/02/2014 13:28:57
Information générale
Forum:
Visual FoxPro
Catégorie:
Codage, syntaxe et commandes
Versions des environnements
Visual FoxPro:
VFP 9 SP2
OS:
Windows 8
Network:
Windows XP
Database:
Visual FoxPro
Application:
Desktop
Divers
Thread ID:
01594373
Message ID:
01594462
Vues:
48
Yes, if you divide the records there's a way you don't have to change your program at all. You can use 5 separate folders on your hard disk, each with a copy of your program and one of the 5 divided tables. If your program expects the table to be named SomeTable, you could rename A1 to "SomeTable" in the first folder, A2 to "SomeTable" in the second etc.

Then, just start your program in each folder using Windows Explorer.

Windows will automatically "load balance" and figure out the best way to run the 5 copies of your program on the CPU cores. You don't have to specify which copy runs on which core, in all but the rarest of circumstances it's actually bad practice to do so.

As I mentioned in another post 5 copies may not give the best performance. You might want to test with 3, 4 or 6 as well. You should not try anything higher than 8, and I suspect 8 or 7 will be too many.

ALSO: a couple of housekeeping things you should check, regardless of what method you eventually use:

1. Check your power plan setting (Control Panel...Power Options in Win7). By default it may be "Balanced". In the "Show additional plans" you can show, then select the "High Performance" plan.

2. In Control Panel...System, go into Advanced system settings, Advanced tab, Performance section, click on the Settings... button. On its Advanced tab, make sure the Processor Scheduling section is set for "Programs" and not "Background services".

>In that case you mean to say That of I have 20000 Records in a Table say A. I shall divide it in A1,A2,A3,A4,A5 Each with 4000 Records first.
>And than I should process Each table in a separate CPU. Is it what is suggested.
>
>If yes, than kindly tell me how will I bind my A1 table and related records to CPU1 and A2 to CPU 2 ?
>
>Harsh
>
>
>>Harsh, make sure you read Al's post regarding not using Parallel Fox and instead modifying your application to accept a parameter to process a section or group of files. And then you load several copies of your main application, each processing a different group of files. The advantage of this are:
>>
>>1) Far simpler model - no need to mess around with Parallel Fox, threads, etc. Your app is a simple file conversion app (but needs to take a parameter to determine files to process, or it must ask the user for which files to process, or you can create a database of file names and lock the record you are processing and each copy of your app just looks for the next not-locked record. There are various options.
>>
>>2) By running multiple copies of your app it allows the processor to decide which CPU core or thread to run each copy on. More efficient.
>>
>>3) You can load as many copies of your app as you like - watching CPU and disk usage to judge performance - stopping copies and loading additional copies without code changes.
>>
>>
>>
>>>Thanks very much for the Guidance Sir.
>>>
>>>>>Hello Experts,
>>>>>
>>>>>I am converting file from Type A to Type B using a I7 Machine with 8 core CPU, 8 GB RAM, 2TB HDD.
>>>>>
>>>>>Using simple Program (i.e using single core) I am able to convert around 800 files in a Minute. my CPU Shows 17%-18% Usage. Memory Shows around 15%-16% and HDD shows around 10 % usage in Task Manager.
>>>>>
>>>>>When I incorporate Parallel Fox in my code and use all the CPU. Following was the result found.
>>>>>
>>>>>No. of files converted: around 1700 Per Minute
>>>>>CPU Usage:-75%-95%
>>>>>Memory Usage 30%-35%
>>>>>HDD Usage :- 20%-40%
>>>>>
>>>>>Thus , using a Single CPU I am getting around 800 files but when I use Parallel fox and all cpu utilized, though my CPU works to the fullest I just get just twice the ouput than before.
>>>>
>>>>It's worth pointing out a few things about your i7-4770K processor ( http://ark.intel.com/products/75123 ):
>>>>
>>>>1. It's not a true 8-core CPU, it's a 4-core CPU with HyperThreading (HT). HT is a clever trick Intel uses to get more performance out of a computing core. Under ideal conditions Intel claims this can improve performance of a core by about 30%. In general usage it will be somewhat less. From a performance point of view you can consider your 4770K to have ~5 "real" cores, although 8 "logical" cores appear in Windows Task Manager
>>>>
>>>>2. When running only 1 intensive task, the 4770K can speed up a single core from 3.5 to 3.9GHz (about 10% faster) ("Turbo Boost/Frequency"). It won't do this if all cores are busy. So your 1-core test was probably running at 3.9GHz, and the multi-core test at 3.5GHz
>>>>
>>>>3. Windows needs processing power to run other tasks while your jobs are running. During your single core test there are other cores free for Windows to do these other things so your task can run unimpeded on its single core. If you're trying to run your job in parallel using all cores then from time to time your job threads will be interrupted so Windows can service its own threads. Performance will be reduced because of these context switches
>>>>
>>>>Comparing apples to apples, 800 files/min @3.9GHz is about 720 files/min @3.5GHz.
>>>>
>>>>Your performance gain using "all cores" @ 3.5GHz is 1700/720, or about 2.36. The theoretical best performance improvement you could expect would be about 5.0 times better, with zero overhead for Windows and no losses from context switching. In practice, it will be less than that.
>>>>
>>>>If you're consistently getting 75 - 95% CPU utilization while running your jobs, that's actually pretty impressive, that's not easy to achieve with a multi-core processor. From the stats you give it looks like the CPU is your bottleneck, despite being one of the most powerful desktop CPUs currently available.
>>>>
>>>>I don't know how ParallelFox uses multiple logical cores/threads or if you can configure it to use only a certain number of cores/threads (instead of all available). If you can limit the number of threads being used I would experiment with 3, 4, 5, and 6 threads, rather than 8 (one each for the 8 "logical" cores that appear in Windows). You may find this reduces context switching and improves your job's overall throughput.
>>>>
>>>>If your job is CPU-bound, you have some options:
>>>>
>>>>1. Make your job processing more efficient
>>>>
>>>>2. Get more desktop computers and divide the files you're processing amongst them
>>>>
>>>>3. Move to a server platform with one or more Intel Xeon processors ( http://www.intel.com/content/www/us/en/processor-comparison/compare-intel-processors.html?select=server ). E5 Xeons are available with up to 12 physical cores, and some versions with fewer cores run at up to 3.5GHz. However, server processors, mainboards and RAM are much more expensive than desktop systems; it may be more cost-effective to use multiple desktop systems.
Regards. Al

"Violence is the last refuge of the incompetent." -- Isaac Asimov
"Never let your sense of morals prevent you from doing what is right." -- Isaac Asimov

Neither a despot, nor a doormat, be

Every app wants to be a database app when it grows up
Précédent
Suivant
Répondre
Fil
Voir

Click here to load this message in the networking platform