Flat File Parser Grouping Example
Purpose
This example shows how to group data from multiple lines into a block which will be passed to a processing program.
Data Format
The data is read from a stream file on the IFS.
This is the data from the echo example.
00012010-12-201234567890123450 00012010-12-201234567890987900
The same data could come in a different format with the data spread over multiple lines with each line containing a line identifier and the data.
1000001 1102010-12-20 2101234567890123 220450 999 1000001 1102010-12-20 2101234567890987 220900 999
or
1000001 1102010-12-20 2101234567890123 220450 2101234567890987 220900 999
Each line type is identified with an identifier (columns 1 - 3). A block of data begins and ends with a "special" line. This line can be fix or contain variable data.
In this case a complete block of data starts with a line with an identifier "100" and ends with a line with an identifier "999". So the starting line format has an image "100####" and the ending line format "999".
Line Formats
Identifier | Data | |
---|---|---|
Store ID | 100 | 4 digits numeric |
Date | 110 | date ISO format yyyy-MM-dd |
Item | 210 | 13 alphanumeric (EAN) |
Quantity | 220 | 4 digits numeric |
End | 999 | no extra data |
Code
Create a new parser instance
parser = ffp_create();
Configure the line formats
lineFormatImage = '100####' + x'00'; ffp_addLineFormatFromImage(parser : 'store' : %addr(lineFormatImage));
lineFormatImage = '110####-##-##' + x'00'; ffp_addLineFormatFromImage(parser : 'date' : %addr(lineFormatImage));
lineFormatImage = '210@@@@@@@@@@@@@' + x'00'; ffp_addLineFormatFromImage(parser : 'item' : %addr(lineFormatImage));
lineFormatImage = '220####' + x'00'; ffp_addLineFormatFromImage(parser : 'quantity' : %addr(lineFormatImage));
lineFormatImage = '999' + x'00'; ffp_addConstantLineFormat(parser : 'end' : %addr(lineFormatImage));
Create line processor
lineProcessor = ffp_processor_grouping_create(parser);
Configure block start and end
// define where a block begins and ends ffp_processor_grouping_setStartLineFormat(lineProcessor : ffp_getLineFormat(parser : 'id')); ffp_processor_grouping_setEndLineFormat(lineProcessor : ffp_getLineFormat(parser : 'end'));
Set data processing program
// configure which program will process the grouped data ffp_processor_grouping_setExecutor(lineProcessor : 'FFPEX02EX');
Set input provider
inputProvider = rng_input_stmf_create(%addr(inputPath)); ffp_setInputProvider(parser : inputProvider);
We don't want the parser to skip any invalid lines because we may need them in the block processing and the line processor needs to know if invalid data has been passed at all. So we set a flag to tell the parser to also pass invalid lines to the line processor. The grouping line processor will skip a block if it contains an invalid line. This block will not be passed to the processor. An error message will be sent by the logger and a registered serializer will backup the invalid block of data.
ffp_setProcessInvalidLines(parser : *on);
Parsing
// do the actual parsing ffp_parse(parser);
Cleanup
// cleanup (all resources registered at the parser will also be cleaned up) ffp_finalize(parser);
Processing Program
The processing program gets the block of data from the line processor in a list data structure (though not data structure as in the RPG language, see Linked List). The program must implement the following prototype:
D program PR D logger * const D data * const
Each entry in the list is a data structure in the format of ffp_processor_grouping_line. The data structure contains the associated line format, the raw data and the line number.
D ffp_processor_grouping_line... D DS qualified D lineFormat * D data 65535A varying D lineNumber 10I 0
The following code iterates through the passed data:
D entry DS likeds(ffp_processor_grouping_line) D based(entryPtr) D lineFormat DS likeds(ffp_lineFormat) D based(entry.lineFormat) ... list_abortIteration(data); entryPtr = list_getNext(data); dow (entryPtr <> *null); // process data here entryPtr = list_getNext(data); enddo; ...
Note: If you have defined fields in your configuration, you can extract the data for each field with the procedure ffp_util_getLineFormatFieldData.