The ability to derive multiple tokens from the same overlapping strings.
Requirement:
From the example stream name “PIDAHou_Flow.PV”, we need to be able to derive the following asset ID and metadata information.
Asset ID: ”PIDAHou”
Asset property “Measurement”: “Flow.PV”
Value of Asset metadata “Location” : “Houston” (mapped val from Hou)
Value of Asset Metadata “Product” : “PI Data Archive”
Solution Option 1:
Support: Overlapping tokenization (Note that Token1, Token3 and Token4 has overlapping string parts of the stream name).
Token1: string before the first “_”, and use this token to express Asset ID : {Token1}
Token2: string after the first “_”, and use this token for Measurement property
Token3: Three letter string prior to the first “_” with the mappings {“HOU”:”HOUSTON”, “DAL”:”Dallas”}. use this token to assign to the metadata Location
Token4: Two letter string starting at 3rd position with the mappings {“DA” : ”Data Archive”, “AF” : “Asset Framework”}. Use this token to assign the metadata Product
Use the values of these tokens to extract required info as follows
Asset ID={Token1}
Measurement = {Token2}
Location={Token3}
Product={Token2}
Solution Option2:
Use both raw unmapped token as well as mapped value of a token
Token1: First Two letter as literals
Token2: Two letter string starting at 3rd position with the mappings {“DA” : ”Data Archive”, “AF” : “Asset Framework”}
Token3: Three letter string prior to the first “_” with the mappings {“HOU”:”HOUSTON”, “DAL”:”Dallas”}.
Token4: string after the first “_”, and use this token for Measurement property
Use the values of these tokens to extract required info as follows
Asset ID={Token1}{Raw unmapped value of Token2}{Raw unmapped value of Token3}
Measurement = {Token4}
Location={Token3}
Product={Token2}
