Instructions

<= previous

^ top

next =>

There are two steps to using ecpSE. The first step is to define what database, table, and fields data will be entered into. This is done using ecpSE.py. ecpSE.py is more or less a wizard type program that generates a program called ecpSEa.py and several support files.

For the second step ecpSEa.py is used to take data from a scanner and put it into the database fields already defined. While using ecpSEa.py one can control exactly how the data is dealt with, changed, and edited on its way to the database.

---

Generating an ecpSEa Program

This is the first step to using ecpSE.

• Start the program by double clicking on the ecpSE icon (or by starting a command prompt, going to the directory where ecpSE resides, and typing ecpse.py). (If the program does not start correctly Python and all of the components are probably not installed correctly.) If everything goes well you will see a window like Fig. 3.2. Notice that a command window opens as well (Fig 3.1). This window will contain information and instructions.

  
  Fig. 3.1

  If the program was started from the command prompt, and there is an error, the error will be printed to this window. If the program is started from its icon, the error information may be lost. For this reason, while this program is still being developed, it is preferable to start from the command prompt.

• First select a database to use by selecting File | Select DB.
• Next select the DB table to scan into by clicking on the table's name.

  
  Fig 3.2

• This will bring up a list of the table's fields. Include them by clicking on them once (or inlude them all with the Add All >> button).
• Select the NCS form definition (.sds file) to use (this only matters if you are using SFWLink) by selecting File | .SDS File or enter its 3 digit number into the Entry Box.
• Enter a name for the project into the Save Location Text Box. The Path (C:/ecpse/apps/) should exist, but the project name (project1) should be new. ecpSE will generate this directory.
• Select Make | Write Files. This writes the ecpSEa file and the supporting files.
• Exit the program. You should generate a new ecpSEa for each database/.sds file combination.

Running an ecpSEa Program

Once your ecpSEa file has been generated you can either scan or import a text file into a database. These are the general instructons. There is an example with more detailed instructions below.

• Go (cd) to the directory that was just created.
• Either click the ecpSEa icon, or type ecpsea.py from a command prompt.
• Select which error handling option you want to use [Disable, Enable (recommended), or Manual].
• Select whether the program will import from a .dat file, or run the scanner [You will need SFWLink DK and ecp_lnk.dll to do this.].
• Press Scan.

Flow Chart

#
#              -------------
#                  START       
#              -------------
#                    |
#                    |
#                    V
#              -------------
#  --------->    SCAN/OPEN   <---------------------------------------------
#  |           -------------                                               |
#  |                 |                                                     |
#  |                 |                                                     |
#  |                 V                                                     |
#  |           -------------              -------------                    |
#  |           IS THERE DATA? ---(no)-->  ENDSCAN/CLOSE                    |
#  |           -------------              -------------                    |
#  |                 |                                                     |
#  |               (yes)                                                   |
#  |                 V                                                     |
#  |           -------------                                               |
#  |               PARSE                                                   |
#  |           -------------                                              (no)
#  |                 |                                                     |
#  |                 |                                                     |
#  |                 V                                                     |
#  |           -------------              -------------             -------------
#  |           CHECK FOR ERR? ---(yes)-->  ERR EXISTS?  ---(yes)-->   FIX ERROR?
#  |           -------------              -------------             -------------
#  |                 |                          |                          |
#  |                (no)                       (no)                      (yes)
#  |                 V                          |                          |
#  |           -------------                    V                          V
#  ----------    ENTER DATA  <---------------------------------------------
#              -------------
#

Fig. 3.3

Editing Data Parsing Parameters

The Data Parsing Parameters can be viewed and edited in the Parsing Window. They are saved (and can also be viewed) in Paramlist.txt. Each line of text consists of 5 items separated by commas. For example:

Field_0, 40, 4, Len, == 4

All five items relate to one database field. Thus there is one line of parameters for each field in the database that will recieve data. If you want to make multiple checks on some data you will ned to preface the field name with a ., !, or ]. See below for details. These parameters determine how the data from the scanner will be parsed, and what sort of error checking will be done.
The first item is the name and index of the database field. This is generated by ecpSE and should not need editing (and should not be changed. If you need to add new fields, add them, end the program, and then restart it.). The field name is relatively unimportant, and is there for user benefit. If the index is changed (either the underscore or the index number), the program will not work.
The second item determines where the data string will be sliced. Note that because Python strings are based upon C arrays of characters, Python consideres the first character of a string to be in position 0. Thus to begin a parse at the 41st character one would put '40' as the second parameter.
The third item determines the length of the parse. In the example above the cut would be 4 characters long.
The last two parameters determine how the slice of data should be analysed. The 4th parameter determines the method of analysis while the 5th supplies arguments as to how the analysis is done. Each possibility for the fourth parameter (and the arguments it will support) is dealt with below. [These are not case sensitive, so conv = CONV = Conv]

• LEN

  Len is used to check the length of the data. Empty characters are stripped from the front and end of the parse. Len can be used to determine if fields (or parts of fields) were left blank.

  Len accepts two arguments. The first is an operator. The second is a number for comparison. Acceptable operators (and descriptions) are as follows:

  == (equal to)

  != (not equal to)

  <= (less than or equal to)

  >= (greater than or equal to)

  << (less than)

  >> (greater than)

  >< (between, but not equal to) - note that >< requires 2 numbers separated by an underscore like 4_5.

  -- (between or equal to) - note that -- requires 2 numbers separated by an underscore like 4_5.

• VAL

  Val is used to check the integer value of the data. Empty characters are stripped from the front and end of the parse. Val can be used to determine if fields are within a certain range of numbers. (*Note that decimal amounts will cause errors. Use Chr for decimals.)

  Val accepts two arguments. The first is an operator. The second is a number for comparison. Acceptable operators (and descriptions) are as follows:

  == (equal to)

  != (not equal to)

  <= (less than or equal to)

  >= (greater than or equal to)

  << (less than)

  >> (greater than)

  >< (between, but not equal to) - note that >< requires 2 numbers separated by an underscore like 4_5.

  -- (between or equal to) - note that -- requires 2 numbers separated by an underscore like 4_5.

• DATE

  Date is used to determine if data is an actual date. It accepts arguments in the form of a string of M's, D's and Y's. These should be positioned to coincide with data characters. For example, MMDDYYYY could be used to test if a string of 8 characters is a valid date. In this case 03311974 would be valid (and entered in the database as 3 / 31 / 1974) where as 31031974 would not be and would generate an error.

  The minimum requirement for the Date argument is 2 Ms and 2 Y's (either MMYY or YYMM). In the case of a 2 digit year, any value greater than 37 (38 - 99) will have a 19 appended to it. Values below 38 (00 - 37) will have a 20 appended.

  In case this is confusing, valid arguments for the Date parameter include, but are not limited to (MMDDYYYY, MMDDYY, MMYYYY, YYYYMMDD, DDYYYYMM, etc.).

  Invalid arguments include, but are not limited to (MY, DDYYYY, MMDYYYY, etc.)

• CHR

  Chr is used to determine if the characters in a parsed string are within a set list of acceptable characters. This is especially useful for double bubble type entries as ScanTools generates a * for double entries.

  Chr accepts 1 argument that can be as long as one wants. In it one should place all acceptable values for the parse. Any character not in the list of characters will generate an error.

  For example, to check that only numbers are used, the argument might be 1234567890.

  *Note that this is case sensative. Including a does not imply that A is included. There is no limit on the length of the list of characters, but it must be on one line in the editor. Also, if you want to accept blank spaces, embed one between 2 other characters i.e. ABCDEFG HIJ. Repeating characters in the list will not cause any problems. Commas are not allowed.

• CONV

  Conv is used to convert one character (or string) to another. The user defines what characters are converted to other characters. Any character for which there is no definition will generate an error.

  Conversions are defined like A->B (this would convert an A to a B) and multiple conversions are separated by an underscore. For example A->B_C->D would convert A to B and C to D.

  This is useful where there is a list of data for a user to select from. For example 0->Male_1->Female would convert 0s to Male and 1s to Female and generate an error for any other value (like blank spaces or double entries.)

  Conv can also be used to convert longer strings. For example XY->Male_XX->Female.

  Conv can also be used to convert blank entries into other values (Chr should be used to simply check for blanks). To convert an empty field use Blank->X where X is the new value.

• CONVC - [CONVert Characters]

  ConvC is a special case of Conv which converts parts of strings to other strings [Conv converts the entire string]. The user defines what characters are converted to other characters

  These are useful to change one string part to another. For example W->P would change COW to COP.

  ConvC can also be used to "brand" data be using the FRONT or BACK keywords. FRONT->FOO would append FOO to a string so BAR would become FOOBAR (BACK works the other way).

  Using FRONTSP or BACKSP will embed a space between the two strings being combined. FRONTSP->FOO would append FOO to a string so BAR would become FOO BAR.

  These conversions are done in order from left to right, so double conversions are possible. For example A->B_B->C would convert A to C whereas B->C_A->B would convert A to B.

  ConvC supports the keywords BLANK, EMPTY [Only as the second parameter.], and COMMA [because using a , instead of COMMA will cause an error as it delimits the parameters]. Either can be used in either the first or second position (converted to or from). BLANK will only detect blank space between two non-blank spaces, so BLANK->COMMABLANK would change SMITH JOE to SMITH, JOE.[These must be all CAPS!]

• TF

  TF is essentially a special case of Conv which converts values to True and False. The user defines what to convert to True and False with the arguments separated by an underscore. For example 1_0 would convert 1 to True and 0 to False.

• CONVDB - [CONVert according to a DataBase]

  This is the most complicated form of convert. The user sets up a database and converts according to matching pairs. This is, of course, possible via standard CONV, but ConvDB can be used for large or frequently changed lists. To use this one either sets up a text file with matching pairs, a pickled python dictionary, or an Access DB accesses via ADO (or anything else you might want to add).

  Here, the fifth parameter is information about the database and is analysed before scanning. There should be 2 arguments separated by an underscore. The first is either TXT, DICT, or DB and how to use each is dealt with in turn.

  • TXT
    • To use a text file, make a file of key value pairs, one on each line, separated by a colon.

      key1:value1
      key2:value2
      key3:value3
      key4:value4
      key5:value5

    • The 5th parameter should be TXT and the location of the text file separated by an underscore. TXT_C:/directory/text_file.txt
  • DICT
    • The First important thing about setting up the dictionary is that ecpSE considers everything to be a string. Thus a = {1:'a', 2:'b'} won't work. Instead use a = {'1':'a', '2':'b'}.
    • The 5th parameter should be DICT and the location of the dict separated by an underscore. DICT_C:/directory/pickle_file.txt
  • DB
    • To use a database, one must set up an MS Access database such that some table has 2 columns one of which contains keys while the other has values.
      
    • The 5th parameter should be DB, an underscoreand, then the location of the database, the table name, the key column, and the value column separated by *. DB_C:/directory/database.mdb*table_name*key_column*value_column

• CASE

  Case is used to adjust the case of the data. It will do nothing to numerical data. Use the keyword LOW to change all text to lower case. Use the keyword UP to change all text to upper case. Use the keyword CAP to capitalize the first letter of each word.

[The next two parsing types should be used mainly with secondary and error catching parameters (explained below)]

• REPLACE

  Replace is used to replace whatever data exists with its own parameter value. The only option is the keyword BLANK which will replace the data with an empty string. It should be used for error catching purposes (see below) or to always put specific data into a database field.

• COMB

  Comb is used to combine 2 fields. Using th keywords FRONT, FRONTSP, BACK, BACKSP [See ConvC for how these work]one can append the contents of another field to the present one.

  The syntax is Field_2, 40, 2, COMB, BACKSP->Field_1 which will place a space and the already edited contents of Field_1 after the parse from 41 to 43.

  This will mainly be used for error catching purposes (see below).

Secondary and Error Catching Parameters

Using these types of data handlers one can check fields multiple times and correct errors that occur. There ar three types of specifiers. These are ".", "!", and "]"

• . is used to visit a specified field a second time. It is run when there are no errors from previous parameters.
• ! is used to automatically handle previous errors. It is run only when there are errors from previous parameters. It will undo errors.
• ] is run regardless of errors. If there was an error previously it will still register as such [! can undo errors] but the second editor will run.

To use these one uses the same error handlers described above, but does not specify data parses. The data that is checked or edited is the already checked data. Thus parameter entried look like the following [With 3 parameters instead of 5]:

• .LastName_2, COMB, FRONTSP->FirstName_0
  This would append (the already edited) "FirstName" field to the front of the LastName field assuming there were no errors in the LastName field the first time through.
• !LastName_2, REPLACE, Data This would replace whatever data in the LastName field caused an error the first time through with the string "data".
• ]LastName_2, CASE, CAP This would capitalize the first letter of words in the LastName field regardless of errors that occured. If there was an error it will still pause.

Error Checking

• If some data is considered invalid by the defined parameters the scanning process will stop. The error will be printed to the command prompt window, and the field containing the error will be highlighted. The user may edit the data. The Accept Form button and the Reject Form button will be enabled. Pressing the Accept Form button will enter the edited data into the database and go to the next sheet. Pressing the Reject Form button will ignore the data and simply go to the next sheet.
• If the user selects Manual Error Checking, the program will still check for errors, but it will pause after every sheet.
• If the user selects Disable Error Checking only the first 3 parameters will be used, and no error checks or data conversions will be done. Note that even though the last 2 parameters will be ignored, they must be valid for the program to run. Also note that a data error will cause the program to pause. This is not an error caused by the parameters. It is the database refusing the data. The error returned by the database will print to the command prompt window, but there will be little indication in the editor of which field has the error. As with other data errors you can edit the error to accept the data, or skip the form. Pressing accept without editing the data will cause the same database error again.

Disabling Fields

There are times that you may want to disable a fields entry into a database. Either because you are parsing data from different parts of a string then combining them into one field [COMB], or an error has occured and you want to disable a field for a single form. To do the former, simply uncheck the box next to the field's data editor and leave the Reset Checks check unchecked.

If you need an extra field to edit data in you can add a dummy parameter. Make sure that the field name and number are both unique like Dummy Field_16, 40, 10, LEN, !=10. Save your changes, close ecpSEa and restart it. The new field will appear.

If you need to disable a field for one form because there is an error in the data, uncheck the box next to the field's data editor, and check the Reset Checks check.

Example

This example will go through the process of scanning two forms, one with errors, and one without.

First, notice the parsing parameters in Figure 3.4. For a line-by-line explanation see below.

Fig 3.4

There are examples of most types of parameters. The names of the database fields correspond to their data types.

When the scan button is pressed the first string of data is collected (in this case from a .dat file).

||||||||||||||||||||||||||||||||||||||||XXXXXXXXXXABCDE800751103311974112102

NCS reserves the first 40 characters (blue) for their use. The next ones (green) will be parsed according to the parameters above. After parsing the form looks like Figure 3.5 with no errors.

Fig 3.5

Pressing Accept Form will enter the data into the database and scan the next form. Normally with error free data you will not have to accept it. In this case the program is running in manual error checking mode.

The next string of data is ...

||||||||||||||||||||||||||||||||||||||||XXXXXXXXXXABCD*110750013311974043102

After parsing the program finds errors. It shows where these errors are like in the Figure 3.6.

Fig 3.6

The details of the errors are also displayed in the bottom of the window. The errors can be corrected and the data entered, or the Reject Form button can be pressed to discard the data. If bad data is not corrected and the database rejectes the INSERT, the user will be returned to the edit position. (Even in non-error check mode a database error will result in the chance to edit the data.)

Line-By-Line Explanation of Example

• Text Field a_1, 40, 10, Len, ==10

  This will cut characters 41-50 from the data. If there are 10 characters there will be no error. If there are more (not possible) or less (due to blanks on either end of the data) an error will be generated.

• !Text Field a_1, CONVC, BLANK->EMPTY

  This will run if there was an error in the first pass. It will convers blank spaces to empty strings and undo the error. [Note the problems that cound occur here. Blanks in the middle and on the ends will be changed to empty strings. Thus AB CD will become ABCD which may or may not be a problem.]

• ]Text Field a_1, CASE, CAP

  This will run regardless of any previous errors. It will convert the first letter of words to caps, and all other letters to lower case.

• Text Field b_2, 50, 5, CHR, ABCDE

  This will check that all characters in Text Field b are either A, B, C, D, or E. All others will generate errors (including a, b, c, d, and e).

• .Text Field b_2, CASE, CAP

  If there were no errors in Text Field b, this will convert the first letter of words to caps, and all other letters to lower case.

• .Text Field b_2, COMB, BACKSP->Text Field a_1

  This will place the text from Text Field a after Text Field b's text (in Text Field b) with a space separating them.

• ]Text Field b_2, CONVC, BLANK->COMMABLANK

  This will convert the space between Text Field a and Text Field b to a comma then a space. It will run regardless of errors. Imagine that Text Field a had a first name and Text Field b had a last name. They are now combined (with the first letter a capital) and separated by a comma. This might be a good place for Text Field a to be unchecked with Text Field b entered into a name field.

• Integer Fld_3, 55, 1, Val, >>2

  This will check that the character in position 56 of the data is a number greater than 2. A number less than 2 or any other character will generate an error.

• Lng Integer_4, 56, 4, Val, ><50_100

  This will check that the characters from positions 57-60 are a number between (and not including) 50 an 100.

• Float Fld a_5, 62, 2, Chr, 1234567890.

  This will check that the characters from positions 63-64 are within the set {1,2,3,4,5,6,7,8,9,0,.}.

• !Float Fld a_5, REPLACE, 0

  If one of the characters was not in the above set, the entire field will be replaced by 0.

• Float Fld b_6, 58, 2, Chr, 1234567890.

  This will check that the characters from positions 59-60 are within the set {1,2,3,4,5,6,7,8,9,0,.}.

• .Float Fld b_6, VAL, >>6

  If those characters were in the set, it will be checked to see if the number is greater than 6.

• Boolean tf_7, 60, 1, TF, 1_0

  This will convert 1's to True and 0's to False. All other characters will generate an error.

• Boolean yn_8, 40, 1, TF, X_Y

  This will convert X's to True and Y's to False. All other characters will generate an error.

• Date Field_9, 62, 8, Date, MMDDYYYY

  This will check that the characters from 63 to 70 are a valid date of the format MMDDYYYY. If not an error will be generated.

<= previous

^ top

next =>