Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Articles
Search: 

The hidden face of SELECT SQL
Christian Desbourse, January 1, 2001
One of the most powerful command available with FoxPro’s language, SELECT-SQL allows you to extract from your databases the right data you need. For the beginner, FoxPro provides a built-in RBQE dialog which builds the SELECT-SQL statement and stores these in QPR files, which can be executed from w...
Summary
One of the most powerful command available with FoxPro’s language, SELECT-SQL allows you to extract from your databases the right data you need. For the beginner, FoxPro provides a built-in RBQE dialog which builds the SELECT-SQL statement and stores these in QPR files, which can be executed from within an application. The RBQE dialog doesn’t cover all the possibilities of the SELECT-SQL command and you will sooner or later be forced to write your own statements without the help of RBQE. The SELECT-SQL command has many clauses and can sometimes behave in the most unexpected ways and return totally incorrect results. The documentation available does not cover all of these behaviours, which can best be illustrated by examples. Although traditional xBase techniques will often let you do the same things as with SELECT-SQL, the shortest and most efficient way is in most cases the SELECT-SQL way. There are situations, though, where a combination of both will give you the best results. It is the purpose of this series of articles to provide you with solutions to a number of specific problems, originating from real life applications. Rather than a systematic exploration of the SELECT-SQL command and its clauses, these articles are a hodgepodge of questions and answers, recommendations, and solutions to business problems.
Description
One of the most powerful command available with FoxPro’s language, SELECT-SQL allows you to extract from your databases the right data you need. For the beginner, FoxPro provides a built-in RBQE dialog which builds the SELECT-SQL statement and stores these in QPR files, which can be executed from within an application. The RBQE dialog doesn’t cover all the possibilities of the SELECT-SQL command and you will sooner or later be forced to write your own statements without the help of RBQE. The SELECT-SQL command has many clauses and can sometimes behave in the most unexpected ways and return totally incorrect results. The documentation available does not cover all of these behaviours, which can best be illustrated by examples. Although traditional xBase techniques will often let you do the same things as with SELECT-SQL, the shortest and most efficient way is in most cases the SELECT-SQL way. There are situations, though, where a combination of both will give you the best results. It is the purpose of this series of articles to provide you with solutions to a number of specific problems, originating from real life applications. Rather than a systematic exploration of the SELECT-SQL command and its clauses, these articles are a hodgepodge of questions and answers, recommendations, and solutions to business problems.

Most of these examples are applied to Testdata.dbc from Visual FoxPro Samples directory.

Array or Cursor or Table output?

Output to array for maximum speed when you don’t need permanent data for further processing and you are sure the size of the array will not use up all available memory. Output to table or cursor is limited only by the space available on disk. A cursor is always read only. You will need a table to edit the data resulting from the SELECT SQL statement after quitting the application or the work session.

Output toPermanent outputEditableLimited by
ARRAYNOYESMEMORY
CURSORNONODISK SPACE
TABLEYESYESDISK SPACE

How to get the top ten records?

This is an example where the SELECT-SQL is best divided in two statements, with an intermediate table. Applying the WHERE clause to the first statement would give you the first ten records of the Customer file, regardless of the unit_price value.

SELECT prod_name,unit_price FROM products ORDER BY 2 DESCENDING INTO TABLE tmp
SELECT * FROM tmp WHERE RECNO()<10
Don’t forget to get rid of the tmp table. You have to close the table first, as SELECT SQL leaves tables open and you have to do the cleanup afterwards.
USE IN tmp
DELETE FILE tmp.dbf
In case your tmp file contains memo fields, don’t forget to delete also the corresponding .fpt file.

RECCOUNT() does not match number of records

Even if you require output to a cursor, FoxPro may elect to use the original data table in another work area, under a different alias, as you would obtain with a statement like :

USE MyFile AGAIN IN 0 ALIAS MyAlias
In that case, the RECCOUNT() function gives the number of records in the original file, to which FoxPro has applied a filter to select the records you want. The only way to have the correct number of records selected in all situations is using _tally.
SELECT * FROM products WHERE product_id=‘    14’ INTO CURSOR tmp
SELECT tmp
? _tally     &&  =1
? RECCOUNT() &&  =77
SELECT * FROM products WHERE prod_name = "Tofu" INTO CURSOR tmp2  
SELECT tmp2
? _tally     &&  =1
? RECCOUNT()  &&  =1
In the first example, FoxPro has used the original table, while in the second example, a temporary cursor has been created. If you want to be sure FoxPro creates a temporary cursor, add a dummy field to the fields list or add a dummy "TRUE" condition to your WHERE clause :
SELECT *,"" AS Dummy FROM products WHERE product_id='    14' INTO CURSOR tmp3
select tmp3
? _tally   && =1
? reccount() && =1
SELECT * FROM products WHERE product_id='    14' AND .T. INTO CURSOR tmp4
select tmp4
? _tally  && =1
? reccount()  && =1
Fields disappear from the query output

When you join several tables in a query, and use the * instead of listing fields individually, you might believe some of the fields have disappeared from the query output. The "Variable not found " error message will be the symptom in most cases. Your fields are there, but what happens is that FoxPro renames fields that are present in several tables :

SELECT * FROM Customer, Orders ;
                WHERE Customer.Cust_id = Orders.Cust_id
? Cust_id   &&  Variable not found
If you look at the resulting cursor, you will find out that Customer.Cust_id has been renamed as Cust_id_a and Orders.Cust_id as Cust_id_b. You might have anticipated this one , but maybe not the next one we find : PostalCode_a and PostalCode_b. If you want to be certain how fields are named in the output of your SELECT-SQL statement, you have to list those fields individually and use the AS clause. If your tables have many fields and you don’t want to list all fields, you can combine the * with individual listings as in this example :
SELECT *, Customer.Cust_Id AS CustId, Customer.PostalCode AS PostCode ;
                   FROM Customer, Orders ;
                   WHERE Customer.Cust_id = Orders.Cust_id
? CustId  &&  FRANS
? PostCode &&  10100
Note that I have chosen different names for these two fields. Choosing the same name will force FoxPro to rename your added fields in Cust_Id_c and PostalCode_c as in the following example :
SELECT *, Customer.Cust_Id AS Cust_Id, Customer.PostalCode AS PostalCode ;
                   FROM Customer, Orders ;
                   WHERE Customer.Cust_id = Orders.Cust_id
? CustId  &&  Variable not found
? PostCode &&  Variable not found
? Cust_Id_c  && FRANS
The same problem could happen after using one of the five field functions : COUNT(), MAX(), MIN(), AVG(), SUM(). Unless you have used the AS clause, FoxPro will rename those fields adding a prefix like Cnt_, Max_, Min_, Avg_, Sum_ before the field name.
SELECT SUM(order_amt) FROM Orders
? order_amt           && variable not found
? sum_order_amt   && 1641167.9900
Notice that the COUNT() function will give the same result regardless of the field you have chosen. You can also replace the field name by an asterisk. The output is then renamed as cnt. This allows you to count the number of records of your query output regardless of table or fields names :
SELECT COUNT(*) FROM Orders
? cnt   &&  1078
SELECT COUNT(*) FOM Customer
? cnt   &&  91
Mixed Ascending/Descending order

You can order output by any field of any of the tables included in the FROM clause of your SELECT-SQL statement., or by any of the columns of your query output. You can designate ordering fields by their name, or by their order in the fields list :

SELECT Cust_id, Order_amt FROM orders ORDER BY Cust_id
or
SELECT Cust_id, Order_amt FROM Orders ORDER BY 1
Ordering is Ascending by default but you can specify DESCENDING for each field individually :
SELECT Cust_Id, Order_amt FROM Orders ORDER BY 1, 2 DESCENDING
Ordering is by customer Id (ascending alphabetical order) and by decreasing order amount for each customer.

The missing Outer Join

When joining two or more tables, the query will include the records that appear in all tables. For instance, the customers that have orders in the Orders table will be included in the results of the following query :

SELECT Customer.Cust_id FROM customer, orders WHERE customer.cust_id = orders.cust_id
Those customers that don’t have any orders will be left out. Other development tools feature, as part of their SQL language, the Outer Join : in an outer join, there will be one record in the results for each record that appears in one or several of the tables. An outer join would thus include also those customers without orders. FoxPro has no Outer Join capability and you have to simulate it, combining two or more queries with a UNION clause.

This second query provides one record for each customer that has no orders in the orders file.

SELECT customer.cust_id FROM customer WHERE cust_id NOT IN ;
       (SELECT DISTINCT cust_id FROM orders)
In the following example, the first part of the query will provide all customers with orders, and the total amount of their orders. The second part of the query provides the customers without order. To make the two subqueries compatible, the second one must also contain a tot_amt field, which we have to build from a constant, corresponding to the orders table order_amt field length (8), converted to currency type using the NTOM() function.
SELECT customer.cust_id, SUM(order_amt) AS tot_amt ;
       FROM customer,orders ;
       WHERE customer.cust_id=orders.cust_id ;
       GROUP BY 1 ;
       UNION ;
       SELECT customer.cust_id, NTOM(00000000) AS tot_amt FROM customer ;
       WHERE cust_id NOT IN (SELECT DISTINCT cust_id FROM orders)
Out output now includes PARIS customer with a total amount of 0.0000.
Christian Desbourse, Ir C Desbourse
Christian Desbourse is an independant consultant providing services in database development and Visual Foxpro programming. Independent developer of business applications. Business experience in logistic and industrial project management. Microsoft MVP from 1996 to 2002. MCP Visual Foxpro Desktop and distributed applications
More articles from this author
Christian Desbourse, January 1, 2001
Our sample data consists of two tables, an employee table and a table defining the timetable we want to display. The timetable is defined with the starting time (date, hour and minutes) and the employee in charge (hempl_id) of a number of time slot allocations of 30 minutes each. The data usually co...
Christian Desbourse, January 1, 2001
If you include two tables in a query without specifying a join condition, every record in the first table will be joined with every record in the second table, with huge results as a consequence.
Christian Desbourse, January 1, 2001
Simulate SEEK, DO..WHILE and SCOPE Do you want to retrieve the five companies following ‘France restauration’ in Customers ? Oh I know, xBase junkies can do that with a SEEK, and a DO WHILE and ....but aren’t we talking SQL ? We will use a UDF to control the scope : FUNCTION incremnt PAR...
Christian Desbourse, January 1, 2001
You have set up an extensive security system to control access to your application through user log-in procedures, and password entry. But some people tend to be security reluctant, they write their password on the wall, or worse, everybody is using the same "Hello" password.