TOP

Converting to and from CSV Format

Applies to: ASN-1Step 10.5

ASN-1Step supports conversion of ASN.1 messages to and from CSV (Comma-Separated Values) format. To represent a structured ASN.1 message in CSV format, ASN-1Step flattens it into a fixed number of columns by recursively expanding complex SET, SEQUENCE, and CHOICE types up to the innermost fields with simple ASN.1 types. The resulting record includes a place for each simple type value, which can be empty for optional fields with absent values.

ASN-1Step supports two CSV formats, as follows:

  • A row format, where each row is a record containing values separated by a comma or some other separator.
  • A column format, where each value is placed on a separate line, possibly following an optional CSV header name and a column separator.

The CSV format can include an optional CSV header created by parsing the input PDU type definition. The header record consists of complete absolute references or single ASN.1 names that match the values in the rows or columns. The column values are generated by parsing encoded messages or value notation.

Example

ASN.1 type notation

PersonnelRecord ::= SEQUENCE {
              name     PrintableString,
              number   INTEGER OPTIONAL,
              children SEQUENCE OF ChildInformation
     }
     ChildInformation ::= SET {
              name    PrintableString,
              date    GeneralizedTime OPTIONAL
     }

ASN.1 value notation

value PersonnelRecord ::= {
             name "Nancy",
             number 20,
             children {
                     {name "James", date "20020312000000"},
                     {name "Martha", date "20100820000000"}
             }
     }

CSV row format

name,number,children_name,children_date
     "Nancy",20,"James",03/12/02 00:00:00
     "Nancy",20,"Martha",08/20/10 00:00:00

CSV Conversion Restrictions

There are some restrictions imposed on CSV conversion because

  • ASN.1 PDU types are usually structured and can be nested to an arbitrary depth.
  • Some ASN.1 types can be defined recursively.
  • Open types or types with contents constraints can include other type values, known or unknown.

These restrictions are as follows:

  • The values for nested recursions are supported only with the full CSV header format, which includes absolute references for all fields. Multiple CSV records of recursive types must have the same number of initialized nesting levels, so all records will have the same number of columns.
  • The values of SET OF and SEQUENCE OF types with multiple components are mapped into several CSV records where only one component differs and the rest are repeated. SET OF and SEQUENCE OF types can be nested inside these types. By default, to avoid significant performance degradation, ASN-1Step supports an additional CSV for up to 50 components inside each type. To set the maximum number of components, pass an extra number parameter to the -csv option.
  • Automatic decoding of open types is supported only with the full CSV header format, which includes nested PDU names or numbers inside parentheses. When open types are included, at any nesting level, within SET OF or SEQUENCE OF types, their containing PDUs must be the same for all components, so that the CSV structure does not change.

    Example
    ASN.1 value:
          ErrorEntry ::=  SEQUENCE {
                  errorCode ERROR-CLASS.&code ({ErrorSet}),
                  errorInfo ERROR-CLASS.&Type ({ErrorSet}{@.errorCode})
          }
          e ErrorEntry ::= { errorCode 10, errorInfo Int : 20}
    
    CSV:
          errorCode,errorInfo(Int)
          10,20
  • The values of types with contents constraints and known containing types are always automatically decoded. If the CSV record includes values of such types in the form of xmlbstring (0101...) or xmlhstring (AB09...), an error is issued.
  • By default, CSVs with string type values containing Unicode characters are written to the output file in binary format with UTF-8 character encoding. These CSVs are displayed using \x##, a special C-style character escape sequence. To save CSV in this format, use the text parameter of the -csv option.

    Example
    ASN.1 Value:
          myLabelUtf8 LabelUtf8 ::= "Umlaute: äÄöÖüÜ"
    
    CSV in text format:
           LabelUtf8
              "Umlaute: \xc3\xa4\xc3\x84\xc3\xb6\xc3\x96\xc3\xbc\xc3\x9c"
  • When the input ASN.1 syntax includes SET or CHOICE types with fields whose tags are not in ascending order as required for PER, encoding and decoding in CSV format should be consistent with any field sorting for PER. So, if a CSV is created while sorting fields by tag because -per is specified (implicitly or explicitly), then -per should also be specified when decoding that CSV. This ensures that the order of the field names in the CSV header is the same as in the related SET or CHOICE internal structure, regardless of whether sorting occurs.
  • Parameters in the -csv option affect both -encodeValue and -decodePDU options. When -decodePDU is specified with the input .csv file and -csv <flags> option, the flags are used to decode one or more CSVs from the input .csv file. If decoding succeeds, CSV re-encoding with the same flags is not performed. Conversely, when decoding is successful while using BER, PER, or other encoding rules, re-encoding is performed. If decoding fails, ASN-1Step attempts to determine the format of the input .csv file and decode it. If this decoding succeeds, then the output .csv file is re-encoded using the -csv option flags. If it fails, the error from the first decoding is re-issued.
  • The values of SET OF and SEQUENCE OF types with multiple components are mapped into several CSV records where only one component differs and the rest are repeated. SET OF and SEQUENCE OF types can be nested inside these types. By default, to avoid significant performance degradation, ASN-1Step supports an additional CSV for up to 50 components inside each type. To set your own maximum number of components, pass an extra number parameter to the -csv option, which no longer has the limit of 50 as of version 10.1.

Type Mapping and Data Conversion

CSVs are in human-readable format similar to ASN.1 value notation. The following table shows differences sorted by ASN.1 type. Additional representations are available for CSVs by passing special flags to the -csv option.

ASN.1 Type ASN.1 Value CSV Value
BIT STRING bstring
'0101011'B
xmlbstring
010101
BOOLEAN TRUE or FALSE XML boolean extended items
TRUE or FALSE
1 or 0
Restricted Character String types Quoted cstring or list notation Quoted strings
UTF-8 encoding is used for Unicode characters; a special C-style character escape sequence \x## is used in the CSV text format.
CHOICE The selected alternative name is included as "chosen : <value>" All alternatives are present, the ones that are not chosen have empty values with extra separators for all their nested components.
ENUMERATED identifier identifier
number
GeneralizedTime "20120808131347+0500" The same non-quoted value
One of these time stamp formats:
06/01/00 00:00:00
2007-11-07 11:36:49
NULL NULL NULL
OBJECT IDENTIFIER, RELATIVE-OID {arc1(num1) arc2 ... } or encoded format Dot notation: 1.3.6.1.4.1.1900
OCTET STRINGS hstring
'6B646F726C793031'H
xmlhstring
6B646F726C793031
BCD or TBCD
ASCII format: kdorly01
Open Type Either encoded or automatically decoded Automatically decoded only if the full CSV header is present.
XML format
Hexadecimal format
Types with Contents Constraint bstring
automatically decoded
Automatically decoded if the containing PDU is known.
REAL Different representations, including {mantissa, base, exponent} notation Decimal format
SET, SEQUENCE Values for fields with OPTIONAL/DEFAULT can be absent. Columns for all fields with all their nested components are present.
Missing values for optional fields are marked with extra separators.
SET OF, SEQUENCE OF Component values are included in
{val1, val2, ...}
One column is present for only one component. The row for the complete PDU is repeated for each extra component.
TIME, DATE and other time types "2010-04-15T17:18:18-07:00" The same quoted value
UTCTime "200808131347Z" The same non-quoted value
Circularly defined types Values for nested types are printed. CSVs for nested recursions are created only when the full CSV header is present.
Fields marked with the ASN1.Remove directive Ignored Ignored without extra column separators

Conversion of OCTET STRING type values to and from ASCII, BCD, or TBCD format is supported using the -csv option with the ascii, bcd, or tbcd parameter, or by using the OSS.PrintFunctionName directive with an OSS-specific function name for that conversion, for example "ossPrintOctetAsBCDString" for a BCD conversion. Applying the OSS.PrintFunctionName directive to OCTET STRING types converts their values to or from the specified format. For example, you can mark OCTET STRING types for conversion to TBCD, BCD, ASCII, IP address, or a special time stamp format using the following directives:

--<OSS.PrintFunctionName MOD.Seq.*.soctVarBcd  "ossPrintOctetAsBCDString">--
--<OSS.PrintFunctionName MOD.Seq.*.soctVarTbcd "ossPrintOctetAsTBCDString">--
--<OSS.PrintFunctionName MOD.Seq.*.soctVarIp   "ossPrintOctetAsIPAddress">--
--<OSS.PrintFunctionName MOD.Seq.*.soctVarAsc  "ossPrintOctetAsASCII">--
--<OSS.PrintFunctionName MOD.Seq.*.soctVarTime "ossPrintOctetAsTimeStamp">--

If the -csv option containing the ascii, bcd, or tbcd parameter is specified, the type values to which the directives are applied will not be affected. However, other OCTET STRING type values will be converted to the format specified in the -csv option.

When using ASCII to convert an OCTET STRING hstring value that starts with one or more space characters (20), the resulting CSV value is enclosed in quotation marks, which indicates that spaces are part of the value. When decoding, ASN-1Step ignores quotation marks that follow space characters.
Example

ASN.1:
      Oct ::= OCTET STRING
      oct Oct ::= '20206C65616469616720737061636573'H

CSV:
      Oct
      "  leading spaces"

This documentation applies to release 10.5 and later of ASN-1Step®.

Copyright © 2024 OSS Nokalva, Inc. All rights reserved.
No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means electronic, mechanical, photocopying, recording or otherwise, without the prior permission of OSS Nokalva, Inc.
Every distributed copy of ASN-1Step is associated with a specific license and related unique license number. That license determines, among other things, what functions of ASN-1Step are available to you.