Don't be afraid of the file is big, 20 m small files,
For what do you think of the file, do first filing
Consider you deal with the frequency of this file, if is a one-time processing, so using readily available tools for formatting work to go, if it is to consider the need to handle such frequent files use script processing,
No matter use a program or a tool, the first step, first make clear the rules of the file, such as data lines of the separator is what, field use delimiters or treated with fixed length, there are always rules, in accordance with the rules, for some uncertain long useless data such as Spaces, if you can differentiate into a field that is assigned to the fields to trim,
If really don't know the rules, but just want to meet the continuous space when the separator, so I like to use UE text editing tools, such as search two Spaces, and replaced with a space, repeated several times, when he said can't find a continuous two Spaces, and all documents of continuous Spaces have been replaced with a space,

So processed again data rows and fields such as separation of simple, : this guy, thank you for your answer, 20 m, text files, more than one hundred ten thousand records, or trouble to handle some, just open the file, the computer card for a few seconds, probably thinking know, won't always deal with, but to meet two or three times a year, if use tools, what tools can format, specific code is what? Application zha a format, how to determine the data line separators, feel the text file is to use the TAB key to separate, but I saw some white space is short, some long Spaces, with input# 1 reads newspaper beyond the end-of-file error, still hope good friend guidance again, thank you!

CodePudding user response:

reference 3/f a toast to invite the bright moon response:

line read, each row "pretreatment" (replace operation), and then to "field separation,"
The separation results into Excel each line...

So easy........

Thanked the man answer, read one line, each line of "pretreatment" (replace operation), and then to "field separation,"
The separation results into Excel each line... , the key is how the processing of a line, can have a detailed code?

CodePudding user response:

You put "text editor" (especially the "wen to death" system "notepad")
To open the larger files, it cause caton, common,
Although 20 MB file, also is not too big,

Since a year to meet two, three times, then directly use Excel to open,
Take advantage of its "respectively" function to deal with, not line?

CodePudding user response:

This code is VBS script, note: no treatment contains a comma in the text:

 
The Set of fso=createobject (" scripting. Filesystemobject ") 
Curdir=fso. Getparentfoldername (wscript. Scriptfullname) 
Inputpath=fso. Buildpath (curdir, "data. TXT") 
Set the stream=fso. Opentextfile (inputpath, 1, False) 
The content=stream. Readall () 
Stream. Close 
The Set reg=createobject (" vbscript. Regexp ") 
Reg. Global=True 
Reg. The ignorecase=True 
Reg. Multiline=True 
Reg. The pattern="[\ t] +" 
The content=reg. Replace (content, ", ") 
Outputpath=fso. Buildpath (curdir, "data. CSV") 
Set the stream=fso. Opentextfile (outputpath, 2, True) 
Stream. Write the content 
Stream. Close 
Msgbox "done!" 

Run the example (marked two columns need to adjust the cell format) :

Download address:
Link: https://pan.baidu.com/s/1NSeqvxgfSt1z4WB6-Wz5ZQ
The extracted code: lk9y

CodePudding user response:

To open large files with tools like UE, let alone 20 m, 2 g are easy to drive