Dear Doug,

We thought it best to consult you first by mail but having not gotten a reply we are posting the problem here. After implementing SDT we are finding that data in tables becomes corrupted and we come to know after few months.

It may not be because of SDT but due to improper implementation.

As SDT was designed by you and you may have sample implementation code for the same.

1) Can you post a sample code for
checking the return value of the TABLEUPDATE() statements? If not, do so and use AERROR() if any of them return .F. AND Reindex and Repair may fail under some conditions. Check the return value of these methods; if they return .F., check the aErrorInfo array of DBCXMgr to see what the problem is.

2) You said " The record count in the header is incorrect, so new records aren’t added to the table properly. In VFP 9, you will get an error message, so it would be worthwhile to upgrade if possible."
We have upgraded to VFP 9. Should we use SET TABLEVALIDATE to 15 (1+2+4+8)?

My Last Email:

-----Original Message-----
From: aman [mailto:aman@jaguarc.com]
Sent: Tuesday, May 06, 2008 4:18 PM
To: 'Doug Hennig'
Subject: RE: SDT - Table stops updating without Error

Dear Doug,

I wanted to get help related to my problems in universalthread, but we thought it best to consult you first as SDT was designed by you and you may have sample implementation code for the same.

1) Can you send a sample code for
checking the return value of the TABLEUPDATE() statements? If not, do so and use AERROR() if any of them return .F.
AND
Reindex and Repair may fail under some conditions. Check the return value of these methods; if they return .F., check the aErrorInfo array of DBCXMgr to see what the problem is.

2) You said " The record count in the header is incorrect, so new records aren’t added to the table properly. In VFP 9, you will get an error message, so it would be worthwhile to upgrade if possible."
We have upgraded to VFP 9. Should we use SET TABLEVALIDATE to 15 (1+2+4+8)?

Regards,

Aman S Bains

Hi Aman.

One thing to be aware of is that SDT doesn’t prevent corruption from occurring; it only helps clean up afterward.

1. The problem with data loss is a header issue; the record count in the header is incorrect, so new records aren’t added to the table properly. Also, no error occurs when the table is opened, so there’s no way to detect the problem in VFP 6. In VFP 9, you will get an error message, so it would be worthwhile to upgrade if possible. This has nothing to do with SDT; something is causing the header to be damaged in this way, and using SDT does not make the matter worse or prevent any error messages. The SDT Repair method will fix this problem.

2. SDT only repairs the header for tables, not the data area, so it doesn’t fix problems when the fields contain corrupted data.

3. NeedReindex simply checks whether the indexes in the table match those in the meta data. It does not check for anything else.

4. Reindex and Repair may fail under some conditions. Check the return value of these methods; if they return .F., check the aErrorInfo array of DBCXMgr to see what the problem is.

You need to resolve the problem causing the corruption. Typically it is power failure, the user turning off the computer without exiting the application and shutting down properly, or viruses.

Doug

From: aman (By way of support@stonefieldquery.com) [mailto:aman@jaguarc.com]
Sent: Monday, November 26, 2007 7:52 AM
To: Doug Hennig
Subject: SDT - Some unhandled cases corrupting data

Dear Sir,

We are using SDT version 6.2b with VFP 6 SP3. Our data is getting corrupted despite implementing SDT. We may not have implemented SDT as envisaged by the Developers.

Some cases of data corruption are as follows:-

Users call us to report that reports are not showing data. Our checking reveals that data was actually not being saved in some tables, although the user doesn’t get any error messages and keeps punching in data even for as long as a month. Before implementing SDT our users used to get error messages and they came to know that something is wrong, but now huge data loss is being caused which is highly alarming. Prior to SDT we used to correct this kind of problem by modifying table structures and moving any one field ordinal position. After this the table used to start saving data, but now this can’t be done as SDT doesn’t allow it.
We sometimes find that some tables contain corrupted or junk data (data from one field is mixed up in two or more other fields as if the data has shuffled randomly, ascii char.s are also added sometimes). Prior to SDT an “Internal consistency error” was shown, but now reindex() or repair() lets the corrupted data be in tables, without showing any messages. User comes to know about the problem after a long time gap which is causing huge data loss.
Prior to SDT we used to get “cannot resolve back-link” error, SDT file check using needreindex() does not show any message in this case.
Sometimes repair() and reindex() do not correct an index problem. We have to Reindex using VFP.
Our implementation of SDT is as follows:-

SDT tables are stored in the same folder as the Data tables, and can never be out of sync.
On startup needreindex() runs on all tables (Tables are in DBC), if problem encountered reindex() is run silently.
On opening data entry forms needreindex() runs on involved tables, if problem encountered reindex() is run silently.
“Utlilities -> Repair” option has been provided in the software, which can be clicked by the user it runs repair() on all tables. (Tables are in DBC, also some free tables are present)
NeedUpdate() is run prior to “Utlilities -> Repair”. But we have our own method to update data structures, which is to provide a fresh copy of Database along with it’s SDT tables in a separate folder. A utility moves the data in all tables from the original folder to a new folder & into the new database, after which these new location files are copied to the original folder overwriting the old files.
Please treat it as most urgent and suggest how to handle the above mentioned cases.

Regards,

Aman S Bains

-----Original Message-----
From: Doug Hennig [mailto:dhennig@stonefield.com]
Sent: Thursday, July 05, 2007 9:34 PM
To: 'aman (By way of support@stonefieldquery.com)'
Subject: RE: SDT - Table stops updating without Error

Hi Aman.
Are you checking the return value of the TABLEUPDATE() statements? If not, do so and use AERROR() if any of them return .F.

If so, then it’s likely you’re running into table header corruption. Use the SDT Repair method to correct this problem. Unfortunately, there isn’t a way to detect that the problem exists if you’re running VFP 7 or earlier. VFP 8 added the SET TABLEVALIDATE command that checks for this problem and gives an error you can trap for when the table is opened.

Doug


From: aman (By way of support@stonefieldquery.com) [mailto:aman@jaguarc.com]
Sent: Thursday, July 05, 2007 9:24 AM
To: Doug Hennig
Subject: SDT - Table stops updating without Error
Hi !
I recently purchased SDT. I'm facing a problem that is not addressed in FAQ or the help.
My application updates 5 tables within a transaction clearly encapsulated within a Begin Trans. - End Trans.

When data entry is being done, at any time the records stop being saved to 1 or 2 tables out of 5. No error is raised and the user doesn't come to know that the data is only partially saved. The user only comes to know a day later when a report is generated.

The Data Entry is continous and so the base tables are not being opened and closed after each entry. Although in Form Init I do check all 5 tables with NeedReindex(), but that doesn't help in this scenario. Please suggest a solution with SDT.

Please treat it as most urgent.
Regards,
Aman S Bains