I am currently trying to use Pandas to sort through Can-Bus data however when I try to make a DataFrame with 2 similar text files, I get two completely different DataFrames.
When I make a data frame with the txt file "CANDUMP With Codes.txt" I get the following DataFrame:
0 ID B0 B1 B2 B3 B4 B5 B6 B7
0 1.656033e 09 0D0 4E FD 00 00 00 00 FF FF
1 1.656033e 09 0D1 00 00 00 00 NaN NaN NaN NaN
2 1.656033e 09 0D2 00 00 FF FF 00 00 00 00
3 1.656033e 09 0D3 00 06 C0 0F 48 00 72 NaN
4 1.656033e 09 0D4 00 00 00 00 00 00 00 00
5 1.656033e 09 018 02 91 70 00 00 00 00 23
6 1.656033e 09 140 00 0F 89 42 00 00 04 01
7 1.656033e 09 141 84 26 5C 27 87 82 A7 00
8 1.656033e 09 142 84 26 00 00 00 00 00 00
9 1.656033e 09 144 C0 00 51 0A 05 A4 A0 00
10 1.656033e 09 156 00 00 00 00 00 00 00 00
11 1.656033e 09 018 02 91 70 02 00 00 00 25
12 1.656033e 09 140 00 00 88 42 00 00 04 01
13 1.656033e 09 141 84 26 5C 27 87 82 A7 00
14 1.656033e 09 142 84 26 00 00 00 00 00 00
15 1.656033e 09 360 B9 00 7C 82 28 80 01 00
16 1.656033e 09 361 00 29 00 D9 04 70 74 75
17 1.656033e 09 152 F0 E4 00 00 00 02 08 00
18 1.656033e 09 0D0 4E FD 00 00 00 00 00 FF
19 1.656033e 09 0D1 00 00 00 00 NaN NaN NaN NaN
20 1.656033e 09 0D2 00 00 FF FF 00 00 00 00
21 1.656033e 09 0D3 00 06 C0 0F 48 00 73 NaN
22 1.656033e 09 0D4 00 00 00 00 00 00 00 00
23 1.656033e 09 018 02 91 70 04 00 00 00 27
24 1.656033e 09 140 00 01 88 42 00 00 04 01
25 1.656033e 09 141 84 26 5C 27 87 82 A7 00
26 1.656033e 09 142 84 26 00 00 00 00 00 00
27 1.656033e 09 144 C0 00 51 0A 05 A4 A0 00
28 1.656033e 09 156 00 00 00 00 00 00 00 00
29 1.656033e 09 6E1 A1 80 00 00 00 60 00 00
30 1.656033e 09 018 02 91 70 06 00 00 00 29
31 1.656033e 09 140 00 02 88 42 00 00 04 01
32 1.656033e 09 141 84 26 5C 27 87 82 A7 00
33 1.656033e 09 142 84 26 00 00 00 00 00 00
34 1.656033e 09 152 F0 F4 00 00 00 02 08 00
35 1.656033e 09 0D0 4E FD 00 00 00 00 FF FF
36 1.656033e 09 0D1 00 00 00 00 NaN NaN NaN NaN
37 1.656033e 09 0D2 00 00 FF FF 00 00 00 00
38 1.656033e 09 0D3 00 06 C0 0F 48 00 74 NaN
39 1.656033e 09 0D4 00 00 00 00 00 00 00 00
40 1.656033e 09 4C3 03 00 00 00 00 00 00 00
41 1.656033e 09 018 02 91 70 08 00 00 00 2B
42 1.656033e 09 140 00 03 88 42 00 00 04 01
43 1.656033e 09 141 84 26 5C 27 87 82 A7 00
44 1.656033e 09 142 84 26 00 00 00 00 00 00
45 1.656033e 09 144 C0 00 51 0A 05 A4 A0 00
46 1.656033e 09 156 00 00 00 00 00 00 00 00
47 1.656033e 09 370 00 00 00 00 00 05 00 00
48 1.656033e 09 018 02 91 70 0A 00 00 00 2D
49 1.656033e 09 140 00 04 8B 42 00 00 04 01
50 1.656033e 09 282 88 94 FA 60 61 03 30 00
51 1.656033e 09 152 F0 04 00 00 00 02 08 00
52 1.656033e 09 0D0 4E FD 00 00 00 00 FF FF
53 1.656033e 09 0D1 00 00 00 00 NaN NaN NaN NaN
54 1.656033e 09 0D2 00 00 FF FF 00 00 00 00
55 1.656033e 09 0D3 00 06 C0 0F 48 00 75 NaN
56 1.656033e 09 0D4 00 00 00 00 00 00 00 00
57 1.656033e 09 018 02 91 70 0C 00 00 00 2F
58 1.656033e 09 140 00 05 8B 42 00 00 04 01
59 1.656033e 09 141 84 26 5C 27 88 82 A7 00
60 1.656033e 09 142 84 26 00 00 00 00 00 00
61 1.656033e 09 144 C0 00 51 0A 05 A4 A0 00
62 1.656033e 09 360 BA 00 7C 82 28 80 01 00
63 1.656033e 09 361 00 29 00 D9 05 E0 7C 75
64 1.656033e 09 156 00 00 00 00 00 00 00 00
65 1.656033e 09 440 42 02 00 00 00 00 00 00
66 1.656033e 09 372 00 70 00 00 00 00 00 00
67 1.656033e 09 018 02 91 70 0E 00 00 00 31
68 1.656033e 09 140 00 06 8B 42 00 00 04 01
69 1.656033e 09 141 84 26 5C 27 88 82 A7 00
70 1.656033e 09 142 84 26 00 00 00 00 00 00
71 1.656033e 09 4C1 01 00 00 00 00 00 00 00
72 1.656033e 09 152 F0 14 00 00 00 02 08 00
73 1.656033e 09 0D0 4E FD 00 00 00 00 00 FF
74 1.656033e 09 0D1 00 00 00 00 NaN NaN NaN NaN
75 1.656033e 09 0D2 00 00 FF FF 00 00 00 00
76 1.656033e 09 0D3 00 06 C0 0F 48 00 76 NaN
77 1.656033e 09 0D4 00 00 00 00 00 00 00 00
78 1.656033e 09 018 02 91 70 00 00 00 00 23
79 1.656033e 09 140 00 07 8B 42 00 00 04 01
80 1.656033e 09 141 84 26 5C 27 88 82 A7 00
81 1.656033e 09 142 84 26 00 00 00 00 00 00
82 1.656033e 09 144 C0 00 51 0A 05 A4 A0 00
83 1.656033e 09 156 00 00 00 00 00 00 00 00
84 1.656033e 09 018 02 91 70 02 00 00 00 25
85 1.656033e 09 140 00 08 8B 42 00 00 04 01
86 1.656033e 09 141 84 26 5C 27 88 82 A7 00
87 1.656033e 09 142 84 26 00 00 00 00 00 00
88 1.656033e 09 370 00 00 00 00 00 06 00 00
89 1.656033e 09 152 F0 24 00 00 00 02 08 00
90 1.656033e 09 0D0 4E FD 00 00 00 00 FF FF
91 1.656033e 09 0D1 00 00 00 00 NaN NaN NaN NaN
92 1.656033e 09 0D2 00 00 FF FF 00 00 00 00
93 1.656033e 09 0D3 00 06 C0 0F 48 00 77 NaN
94 1.656033e 09 0D4 00 00 00 00 00 00 00 00
95 1.656033e 09 018 02 91 70 04 00 00 00 27
96 1.656033e 09 140 00 09 89 42 00 00 04 01
97 1.656033e 09 141 84 26 5C 27 88 82 A7 00
98 1.656033e 09 142 84 26 00 00 00 00 00 00
99 1.656033e 09 144 C0 00 51 0A 05 A4 A0 00
100 1.656033e 09 156 00 00 00 00 00 00 00 00
When I make a DataFrame with the txt file "ID 00d0.txt" I get the following DataFrame:
0 ID B0 B1 B2 B3 B4 B5 B6 B7
0 1.656033e 09 0d0 A0 0 0 0 0 0 ff 1
1 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
2 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
3 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
4 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
5 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
6 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
7 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
8 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
9 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
10 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
11 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
12 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
13 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
14 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
15 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
16 1.656033e 09 0d0 30 0 1 0 0 0 00 1
17 1.656033e 09 0d0 30 0 1 0 0 0 ff 1
18 1.656033e 09 0d0 30 0 1 0 0 0 ff 1
19 1.656033e 09 0d0 30 0 1 0 0 0 00 1
20 1.656033e 09 0d0 30 0 1 0 0 0 ff 1
21 1.656033e 09 0d0 30 0 1 0 0 0 ff 1
22 1.656033e 09 0d0 30 0 1 0 0 0 ff 1
23 1.656033e 09 0d0 30 0 1 0 0 0 ff 1
24 1.656033e 09 0d0 30 0 1 0 0 0 ff 1
25 1.656033e 09 0d0 30 0 1 0 0 0 ff 1
26 1.656033e 09 0d0 30 0 1 0 0 0 ff 1
27 1.656033e 09 0d0 30 0 1 0 0 0 ff 1
28 1.656033e 09 0d0 30 0 1 0 0 0 ff 2
29 1.656033e 09 0d0 30 0 1 0 0 0 ff 1
30 1.656033e 09 0d0 30 0 1 0 0 0 ff 1
31 1.656033e 09 0d0 30 0 1 0 0 0 ff 1
32 1.656033e 09 0d0 30 0 1 0 0 0 ff 1
33 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
34 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
35 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
36 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
37 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
38 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
39 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
40 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
41 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
42 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
43 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
44 1.656033e 09 0d0 30 0 1 0 0 0 ff 1
45 1.656033e 09 0d0 30 0 1 0 0 0 ff 1
46 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
47 1.656033e 09 0d0 30 0 0 0 0 0 00 2
48 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
49 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
50 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
51 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
52 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
53 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
54 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
55 1.656033e 09 0d0 30 0 1 0 0 0 ff 1
56 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
57 1.656033e 09 0d0 30 0 0 0 0 0 00 1
58 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
59 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
60 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
61 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
62 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
63 1.656033e 09 0d0 30 0 1 0 0 0 ff 1
64 1.656033e 09 0d0 30 0 1 0 0 0 ff 1
65 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
66 1.656033e 09 0d0 30 0 0 0 0 0 00 1
67 1.656033e 09 0d0 30 0 0 0 0 0 ff 1
68 1.656033e 09 0d0 2f 0 0 0 0 0 ff 1
69 1.656033e 09 0d0 2f 0 0 0 0 0 ff 1
70 1.656033e 09 0d0 2f 0 1 0 0 0 ff 1
71 1.656033e 09 0d0 2f 0 0 0 0 0 ff 1
72 1.656033e 09 0d0 2f 0 0 0 0 0 ff 2
73 1.656033e 09 0d0 2f 0 0 0 0 0 ff 1
74 1.656033e 09 0d0 2f 0 1 0 0 0 fe 1
75 1.656033e 09 0d0 2f 0 1 0 0 0 ff 1
76 1.656033e 09 0d0 2f 0 1 0 0 0 00 1
77 1.656033e 09 0d0 2f 0 1 0 0 0 ff 1
78 1.656033e 09 0d0 2f 0 1 0 0 0 ff 1
79 1.656033e 09 0d0 2f 0 0 0 0 0 ff 1
80 1.656033e 09 0d0 2f 0 0 0 0 0 ff 1
81 1.656033e 09 0d0 2f 0 1 0 0 0 ff 1
82 1.656033e 09 0d0 2f 0 1 0 0 0 ff 1
83 1.656033e 09 0d0 2f 0 1 0 0 0 ff 1
84 1.656033e 09 0d0 2f 0 1 0 0 0 ff 1
85 1.656033e 09 0d0 2f 0 1 0 0 0 00 2
86 1.656033e 09 0d0 2f 0 1 0 0 0 ff 1
87 1.656033e 09 0d0 2f 0 1 0 0 0 fe 1
88 1.656033e 09 0d0 2f 0 1 0 0 0 ff 1
89 1.656033e 09 0d0 2f 0 0 0 0 0 00 1
90 1.656033e 09 0d0 2f 0 0 0 0 0 ff 1
91 1.656033e 09 0d0 2f 0 0 0 0 0 ff 1
92 1.656033e 09 0d0 2f 0 1 0 0 0 ff 1
93 1.656033e 09 0d0 2f 0 1 0 0 0 ff 1
94 1.656033e 09 0d0 2f 0 0 0 0 0 ff 2
95 1.656033e 09 0d0 2f 0 0 0 0 0 ff 1
96 1.656033e 09 0d0 2f 0 0 0 0 0 ff 1
97 1.656033e 09 0d0 2f 0 0 0 0 0 fe 1
98 1.656033e 09 0d0 2f 0 0 0 0 0 ff 1
99 1.656033e 09 0d0 2f 0 1 0 0 0 ff 1
100 1.656033e 09 0d0 2f 0 1 0 0 0 ff 1
When I try to add some values together from ID 00d0 I get errors of mixing str with int values however with the "CANDUMP With Codes" file this does not happen. The problem is that I need ID 00d0 to be stored using hex (str) like the CANDUMP file.
Down below is the sample code I used to get the DataFrames as well as the content of the text files (I dont know if you can add txt files to a post so I put it in code below). If anyone has any idea why this is happening and has a potential solution, I am all for it.
Thank you all for your time.
Code I am using:
import pandas as pd
import time
startTime = time.time()
pd.set_option('display.max_rows', None)
df = pd.read_table("ID 00d0.txt", header=None, delim_whitespace=True)
df.columns = ['0','ID',"B0","B1","B2","B3","B4","B5","B6","B7"]
df2 = pd.read_table("CANDUMP With Codes.txt", header=None, delim_whitespace=True)
df2.columns = ['0','ID',"B0","B1","B2","B3","B4","B5","B6","B7"]
print(df2)
executionTime = round((time.time() - startTime),10)
print('Execution time in seconds: ' str(executionTime))
CANDUMP With Codes.txt:
1656033437.571007 0D0 4E FD 00 00 00 00 FF FF
1656033437.571007 0D1 00 00 00 00
1656033437.571007 0D2 00 00 FF FF 00 00 00 00
1656033437.571007 0D3 00 06 C0 0F 48 00 72
1656033437.571007 0D4 00 00 00 00 00 00 00 00
1656033437.571007 018 02 91 70 00 00 00 00 23
1656033437.571007 140 00 0F 89 42 00 00 04 01
1656033437.571007 141 84 26 5C 27 87 82 A7 00
1656033437.571007 142 84 26 00 00 00 00 00 00
1656033437.571007 144 C0 00 51 0A 05 A4 A0 00
1656033437.571007 156 00 00 00 00 00 00 00 00
1656033437.571007 018 02 91 70 02 00 00 00 25
1656033437.571007 140 00 00 88 42 00 00 04 01
1656033437.571007 141 84 26 5C 27 87 82 A7 00
1656033437.571007 142 84 26 00 00 00 00 00 00
1656033437.571007 360 B9 00 7C 82 28 80 01 00
1656033437.571007 361 00 29 00 D9 04 70 74 75
1656033437.571007 152 F0 E4 00 00 00 02 08 00
1656033437.571007 0D0 4E FD 00 00 00 00 00 FF
1656033437.571007 0D1 00 00 00 00
1656033437.571007 0D2 00 00 FF FF 00 00 00 00
1656033437.571007 0D3 00 06 C0 0F 48 00 73
1656033437.571007 0D4 00 00 00 00 00 00 00 00
1656033437.571007 018 02 91 70 04 00 00 00 27
1656033437.571007 140 00 01 88 42 00 00 04 01
1656033437.571007 141 84 26 5C 27 87 82 A7 00
1656033437.571007 142 84 26 00 00 00 00 00 00
1656033437.571007 144 C0 00 51 0A 05 A4 A0 00
1656033437.571007 156 00 00 00 00 00 00 00 00
1656033437.571007 6E1 A1 80 00 00 00 60 00 00
1656033437.571007 018 02 91 70 06 00 00 00 29
1656033437.571007 140 00 02 88 42 00 00 04 01
1656033437.571007 141 84 26 5C 27 87 82 A7 00
1656033437.571007 142 84 26 00 00 00 00 00 00
1656033437.571007 152 F0 F4 00 00 00 02 08 00
1656033437.571007 0D0 4E FD 00 00 00 00 FF FF
1656033437.571007 0D1 00 00 00 00
1656033437.571007 0D2 00 00 FF FF 00 00 00 00
1656033437.571007 0D3 00 06 C0 0F 48 00 74
1656033437.571007 0D4 00 00 00 00 00 00 00 00
1656033437.571007 4C3 03 00 00 00 00 00 00 00
1656033437.571007 018 02 91 70 08 00 00 00 2B
1656033437.571007 140 00 03 88 42 00 00 04 01
1656033437.571007 141 84 26 5C 27 87 82 A7 00
1656033437.571007 142 84 26 00 00 00 00 00 00
1656033437.571007 144 C0 00 51 0A 05 A4 A0 00
1656033437.571007 156 00 00 00 00 00 00 00 00
1656033437.571007 370 00 00 00 00 00 05 00 00
1656033437.571007 018 02 91 70 0A 00 00 00 2D
1656033437.571007 140 00 04 8B 42 00 00 04 01
1656033437.571007 282 88 94 FA 60 61 03 30 00
1656033437.571007 152 F0 04 00 00 00 02 08 00
1656033437.571007 0D0 4E FD 00 00 00 00 FF FF
1656033437.571007 0D1 00 00 00 00
1656033437.571007 0D2 00 00 FF FF 00 00 00 00
1656033437.571007 0D3 00 06 C0 0F 48 00 75
1656033437.571007 0D4 00 00 00 00 00 00 00 00
1656033437.571007 018 02 91 70 0C 00 00 00 2F
1656033437.571007 140 00 05 8B 42 00 00 04 01
1656033437.571007 141 84 26 5C 27 88 82 A7 00
1656033437.571007 142 84 26 00 00 00 00 00 00
1656033437.571007 144 C0 00 51 0A 05 A4 A0 00
1656033437.571007 360 BA 00 7C 82 28 80 01 00
1656033437.571007 361 00 29 00 D9 05 E0 7C 75
1656033437.571007 156 00 00 00 00 00 00 00 00
1656033437.571007 440 42 02 00 00 00 00 00 00
1656033437.571007 372 00 70 00 00 00 00 00 00
1656033437.571007 018 02 91 70 0E 00 00 00 31
1656033437.571007 140 00 06 8B 42 00 00 04 01
1656033437.571007 141 84 26 5C 27 88 82 A7 00
1656033437.571007 142 84 26 00 00 00 00 00 00
1656033437.571007 4C1 01 00 00 00 00 00 00 00
1656033437.571007 152 F0 14 00 00 00 02 08 00
1656033437.571007 0D0 4E FD 00 00 00 00 00 FF
1656033437.571007 0D1 00 00 00 00
1656033437.571007 0D2 00 00 FF FF 00 00 00 00
1656033437.571007 0D3 00 06 C0 0F 48 00 76
1656033437.571007 0D4 00 00 00 00 00 00 00 00
1656033437.571007 018 02 91 70 00 00 00 00 23
1656033437.571007 140 00 07 8B 42 00 00 04 01
1656033437.571007 141 84 26 5C 27 88 82 A7 00
1656033437.571007 142 84 26 00 00 00 00 00 00
1656033437.571007 144 C0 00 51 0A 05 A4 A0 00
1656033437.571007 156 00 00 00 00 00 00 00 00
1656033437.571007 018 02 91 70 02 00 00 00 25
1656033437.571007 140 00 08 8B 42 00 00 04 01
1656033437.571007 141 84 26 5C 27 88 82 A7 00
1656033437.571007 142 84 26 00 00 00 00 00 00
1656033437.571007 370 00 00 00 00 00 06 00 00
1656033437.571007 152 F0 24 00 00 00 02 08 00
1656033437.571007 0D0 4E FD 00 00 00 00 FF FF
1656033437.571007 0D1 00 00 00 00
1656033437.571007 0D2 00 00 FF FF 00 00 00 00
1656033437.571007 0D3 00 06 C0 0F 48 00 77
1656033437.571007 0D4 00 00 00 00 00 00 00 00
1656033437.571007 018 02 91 70 04 00 00 00 27
1656033437.571007 140 00 09 89 42 00 00 04 01
1656033437.571007 141 84 26 5C 27 88 82 A7 00
1656033437.571007 142 84 26 00 00 00 00 00 00
1656033437.571007 144 C0 00 51 0A 05 A4 A0 00
1656033437.571007 156 00 00 00 00 00 00 00 00
ID 00d0.txt:
1656033437.571007 00d0 A0 00 00 00 00 00 ff 01
1656033437.590747 00d0 30 00 00 00 00 00 ff 01
1656033437.610978 00d0 30 00 00 00 00 00 ff 01
1656033437.630766 00d0 30 00 00 00 00 00 ff 01
1656033437.650933 00d0 30 00 00 00 00 00 ff 01
1656033437.670918 00d0 30 00 00 00 00 00 ff 01
1656033437.690742 00d0 30 00 00 00 00 00 ff 01
1656033437.710797 00d0 30 00 00 00 00 00 ff 01
1656033437.730938 00d0 30 00 00 00 00 00 ff 01
1656033437.750732 00d0 30 00 00 00 00 00 ff 01
1656033437.770774 00d0 30 00 00 00 00 00 ff 01
1656033437.790819 00d0 30 00 00 00 00 00 ff 01
1656033437.810714 00d0 30 00 00 00 00 00 ff 01
1656033437.830694 00d0 30 00 00 00 00 00 ff 01
1656033437.850853 00d0 30 00 00 00 00 00 ff 01
1656033437.870689 00d0 30 00 00 00 00 00 ff 01
1656033437.890834 00d0 30 00 01 00 00 00 00 01
1656033437.910732 00d0 30 00 01 00 00 00 ff 01
1656033437.930933 00d0 30 00 01 00 00 00 ff 01
1656033437.950721 00d0 30 00 01 00 00 00 00 01
1656033437.970946 00d0 30 00 01 00 00 00 ff 01
1656033437.990844 00d0 30 00 01 00 00 00 ff 01
1656033438.010802 00d0 30 00 01 00 00 00 ff 01
1656033438.030679 00d0 30 00 01 00 00 00 ff 01
1656033438.050847 00d0 30 00 01 00 00 00 ff 01
1656033438.070932 00d0 30 00 01 00 00 00 ff 01
1656033438.090849 00d0 30 00 01 00 00 00 ff 01
1656033438.110787 00d0 30 00 01 00 00 00 ff 01
1656033438.130893 00d0 30 00 01 00 00 00 ff 02
1656033438.151006 00d0 30 00 01 00 00 00 ff 01
1656033438.170744 00d0 30 00 01 00 00 00 ff 01
1656033438.190754 00d0 30 00 01 00 00 00 ff 01
1656033438.210979 00d0 30 00 01 00 00 00 ff 01
1656033438.230791 00d0 30 00 00 00 00 00 ff 01
1656033438.250794 00d0 30 00 00 00 00 00 ff 01
1656033438.270932 00d0 30 00 00 00 00 00 ff 01
1656033438.290712 00d0 30 00 00 00 00 00 ff 01
1656033438.310696 00d0 30 00 00 00 00 00 ff 01
1656033438.330892 00d0 30 00 00 00 00 00 ff 01
1656033438.35081 00d0 30 00 00 00 00 00 ff 01
1656033438.370926 00d0 30 00 00 00 00 00 ff 01
1656033438.390713 00d0 30 00 00 00 00 00 ff 01
1656033438.410782 00d0 30 00 00 00 00 00 ff 01
1656033438.430785 00d0 30 00 00 00 00 00 ff 01
1656033438.450719 00d0 30 00 01 00 00 00 ff 01
1656033438.470731 00d0 30 00 01 00 00 00 ff 01
1656033438.490977 00d0 30 00 00 00 00 00 ff 01
1656033438.510828 00d0 30 00 00 00 00 00 00 02
1656033438.530749 00d0 30 00 00 00 00 00 ff 01
1656033438.55073 00d0 30 00 00 00 00 00 ff 01
1656033438.570944 00d0 30 00 00 00 00 00 ff 01
1656033438.590847 00d0 30 00 00 00 00 00 ff 01
1656033438.610798 00d0 30 00 00 00 00 00 ff 01
1656033438.630972 00d0 30 00 00 00 00 00 ff 01
1656033438.650738 00d0 30 00 00 00 00 00 ff 01
1656033438.670883 00d0 30 00 01 00 00 00 ff 01
1656033438.690741 00d0 30 00 00 00 00 00 ff 01
1656033438.710854 00d0 30 00 00 00 00 00 00 01
1656033438.730805 00d0 30 00 00 00 00 00 ff 01
1656033438.750711 00d0 30 00 00 00 00 00 ff 01
1656033438.770848 00d0 30 00 00 00 00 00 ff 01
1656033438.790796 00d0 30 00 00 00 00 00 ff 01
1656033438.81073 00d0 30 00 00 00 00 00 ff 01
1656033438.830729 00d0 30 00 01 00 00 00 ff 01
1656033438.850798 00d0 30 00 01 00 00 00 ff 01
1656033438.870746 00d0 30 00 00 00 00 00 ff 01
1656033438.890738 00d0 30 00 00 00 00 00 00 01
1656033438.910826 00d0 30 00 00 00 00 00 ff 01
1656033438.930735 00d0 2f 00 00 00 00 00 ff 01
1656033438.950811 00d0 2f 00 00 00 00 00 ff 01
1656033438.970722 00d0 2f 00 01 00 00 00 ff 01
1656033438.990992 00d0 2f 00 00 00 00 00 ff 01
1656033439.010762 00d0 2f 00 00 00 00 00 ff 02
1656033439.03074 00d0 2f 00 00 00 00 00 ff 01
1656033439.050974 00d0 2f 00 01 00 00 00 fe 01
1656033439.070785 00d0 2f 00 01 00 00 00 ff 01
1656033439.090808 00d0 2f 00 01 00 00 00 00 01
1656033439.110784 00d0 2f 00 01 00 00 00 ff 01
1656033439.130923 00d0 2f 00 01 00 00 00 ff 01
1656033439.150792 00d0 2f 00 00 00 00 00 ff 01
1656033439.170964 00d0 2f 00 00 00 00 00 ff 01
1656033439.190788 00d0 2f 00 01 00 00 00 ff 01
1656033439.210718 00d0 2f 00 01 00 00 00 ff 01
1656033439.230836 00d0 2f 00 01 00 00 00 ff 01
1656033439.250775 00d0 2f 00 01 00 00 00 ff 01
1656033439.270921 00d0 2f 00 01 00 00 00 00 02
1656033439.290832 00d0 2f 00 01 00 00 00 ff 01
1656033439.310932 00d0 2f 00 01 00 00 00 fe 01
1656033439.330775 00d0 2f 00 01 00 00 00 ff 01
1656033439.350807 00d0 2f 00 00 00 00 00 00 01
1656033439.37097 00d0 2f 00 00 00 00 00 ff 01
1656033439.390729 00d0 2f 00 00 00 00 00 ff 01
1656033439.411024 00d0 2f 00 01 00 00 00 ff 01
1656033439.430715 00d0 2f 00 01 00 00 00 ff 01
1656033439.450694 00d0 2f 00 00 00 00 00 ff 02
1656033439.47096 00d0 2f 00 00 00 00 00 ff 01
1656033439.490835 00d0 2f 00 00 00 00 00 ff 01
1656033439.51084 00d0 2f 00 00 00 00 00 fe 01
1656033439.530714 00d0 2f 00 00 00 00 00 ff 01
1656033439.550823 00d0 2f 00 01 00 00 00 ff 01
1656033439.570697 00d0 2f 00 01 00 00 00 ff 01
CodePudding user response：
IIUC, use dtype=str as parameter of read_table to prevent Pandas infer the datatype:
df = pd.read_table('ID 00d0.txt', header=None, delim_whitespace=True, dtype=str,
names=['0','ID',"B0","B1","B2","B3","B4","B5","B6","B7"])
Output:
>>> df
0 ID B0 B1 B2 B3 B4 B5 B6 B7
0 1656033437.571007 00d0 A0 00 00 00 00 00 ff 01
1 1656033437.590747 00d0 30 00 00 00 00 00 ff 01
2 1656033437.610978 00d0 30 00 00 00 00 00 ff 01
3 1656033437.630766 00d0 30 00 00 00 00 00 ff 01
4 1656033437.650933 00d0 30 00 00 00 00 00 ff 01
.. ... ... .. .. .. .. .. .. .. ..
96 1656033439.490835 00d0 2f 00 00 00 00 00 ff 01
97 1656033439.51084 00d0 2f 00 00 00 00 00 fe 01
98 1656033439.530714 00d0 2f 00 00 00 00 00 ff 01
99 1656033439.550823 00d0 2f 00 01 00 00 00 ff 01
100 1656033439.570697 00d0 2f 00 01 00 00 00 ff 01
[101 rows x 10 columns]
You can convert column '0' to float by chaining .astype({'0': float}).
CodePudding user response：
Why does this happen?
If you don't tell pandas in which format the data is, it will guess this for you. Since ID 00d0.txt contains only 00 in some columns, the type is ambigous and pandas chose to parse it as an int. You can see all datatypes for each dataframe by calling
df.info()
and looking at the Dtypes column.
What is the fix?
An easy fix would be to read in everything as a string. This is done by adding the keyword argument dtype=str to your read_table call like so:
df = pd.read_table("file.txt", header=None, delim_whitespace=True, dtype=str)
This way you can merge and combine dataframes as you please. Later, however, you may want to do some calculations on the columns, you could then convert them using an arbitrary function. For example to covert the hex strings into integers:
df["B0"] = df["B0"].apply(lambda x: int(x, 16))
Hope that helps :)