How to parse a text file with hex codes?-CodePudding

I'm trying to write my first parsing script for this file;

;$FILEVERSION=1.3
;$STARTTIME=44512.7163933106
;
;-------------------------------------------------------------------------------
;   Bus  Name         Connection          Protocol  Bit rate
;   1    Connection1  Untitled6@_usb      CAN       250 kbit/s
;   2    Connection2  Untitled5@_usb      CAN       500 kbit/s
;   3    Connection3  Untitled1@_usb      J1939     500 kbit/s
;   4    Connection4  Untitled2@_usb      CAN       250 kbit/s
;   5    Connection5  Untitled3@_usb      CAN       250 kbit/s
;   6    Connection6  Untitled7@_usb      J1939     250 kbit/s
;   7    Connection7  Single@_usb         CAN       250 kbit/s
;-------------------------------------------------------------------------------
    ;   Message Number
;   |         Time Offset (ms)
;   |         |       Bus
;   |         |       |    Type
;   |         |       |    |       ID (hex)
;   |         |       |    |       |    Reserved
;   |         |       |    |       |    |   Data Length Code
;   |         |       |    |       |    |   |    Data Bytes (hex) ...
;   |         |       |    |       |    |   |    |
;   |         |       |    |       |    |   |    |
;--- -- ------ ------  - -- -- ---- ---  - - -- -  -- -- -- -- -- -- --
     1)         0.015 7  Rx        047A -  6    88 13 AD 92 11 00 
     2)         0.093 5  Rx    18FF6221 -  8    C0 F0 00 0C C0 FF 05 FF 
     3)         0.318 4  Rx        01FB -  8    02 30 00 C0 00 00 00 80 
     4)         0.484 7  Rx        02FA -  8    88 13 88 13 00 88 13 01

The file is logged data from a vehicle over CAN, I'm trying to read this using python to save me having to use another application that is slow and outdated.

How can I read this into python in a way that pandas can work with the data?

I'm new to python and newer to parsing files, any direction is really appreciatted.

CodePudding user response：

If your file has fixed column length, use read_fwf:

widths = [8, 14, 3, 6, 9, 3, 5, 9999]
names = ['Message Number', 'Time Offset (ms)', 'Bus', 'Type', 'ID (hex)',
         'Reserved', 'Data Length Code', 'Data Bytes (hex)']

df = pd.read_fwf('data.txt', index_col=None, comment=';',
                 header=None, widths=widths, names=names)

Output:

>>> df

  Message Number  Time Offset (ms)  Bus Type  ID (hex) Reserved  Data Length Code         Data Bytes (hex)
0             1)             0.015    7   Rx      047A        -                 6        88 13 AD 92 11 00
1             2)             0.093    5   Rx  18FF6221        -                 8  C0 F0 00 0C C0 FF 05 FF
2             3)             0.318    4   Rx      01FB        -                 8  02 30 00 C0 00 00 00 80
3             4)             0.484    7   Rx      02FA        -                 8  88 13 88 13 00 88 13 01