I have a text file like this:

Viruses/GCF_000820355.1_ViralMultiSegProj14361_genomic.fna.gz
Sclerophthora macrospora virus A
Viruses/GCF_000820495.2_ViralMultiSegProj14656_genomic.fna.gz
Influenza B virus RNA
Viruses/GCF_000837105.1_ViralMultiSegProj14079_genomic.fna.gz
Tomato mottle virus

And I need to get a csv file like this:

Viruses/GCF_000820355.1_ViralMultiSegProj14361_genomic.fna.gz,Sclerophthora macrospora virus A
Viruses/GCF_000820495.2_ViralMultiSegProj14656_genomic.fna.gz,Influenza B virus RNA
Viruses/GCF_000837105.1_ViralMultiSegProj14079_genomic.fna.gz,Tomato mottle virus

Because later I want to use this like a tuple to find the compressed file, read it and get a final file with names like:

Viruses/GCF_000837105.1/Tomato mottle virus.fna

I just need to learn how to do the first part of the problem. It could by with:

sed
awk
R
Python

Any help would be very appreciated. This is hard for me to accomplish because the original filenames are very messed up.

Thank you all for your time.

Paulo

PS- I have tried this:

sed -z 's/\n/,/g;s/,$/\n/' multi_headers

However it put comma in all \n.

CodePudding user response：

Using any awk in any shell on every Unix box and only storing 1 line at a time in memory so it'll work no matter how large your input file is:

$ awk '{ORS=(NR%2 ? "," : RS)} 1' file
Viruses/GCF_000820355.1_ViralMultiSegProj14361_genomic.fna.gz,Sclerophthora macrospora virus A
Viruses/GCF_000820495.2_ViralMultiSegProj14656_genomic.fna.gz,Influenza B virus RNA
Viruses/GCF_000837105.1_ViralMultiSegProj14079_genomic.fna.gz,Tomato mottle virus

There's a lot happening in a small amount of code above so here's an explanation:

ORS is the builtin variable containing the string to be printed at the end of each output record (record = line in this case), a newline by default.
RS is the builtin variable containing the string (or regexp) that separates each input record, a newline by default.
NR is the builtin variable containing the current record/line number so NR%2 is 1 for odd numbered records and 0 for even numbered.
NR%2 ? "," : RS is a ternary expression resulting in , for odd numbered lines, \n (or whatever else you have set RS to, e.g. \r\n) for even numbered.
1 is a true condition which causes the default action of printing the current record to be executed.

So the above script says "if the current line number is odd print it with a , at the end, otherwise print it with a newline at the end", hence it's joining every pair of lines with a , between.

CodePudding user response：

Bash

You can do a paste (thanks @glenn jackman for pointing out my previous useless use of cat).

# or cat mytext.txt | paste -d "," - -
paste -d "," - - < mytext.txt 

Viruses/GCF_000820355.1_ViralMultiSegProj14361_genomic.fna.gz,Sclerophthora macrospora virus A
Viruses/GCF_000820495.2_ViralMultiSegProj14656_genomic.fna.gz,Influenza B virus RNA
Viruses/GCF_000837105.1_ViralMultiSegProj14079_genomic.fna.gz,Tomato mottle virus

R

The R function is also paste, together with sapply:

mytext <- scan("mytext.txt", character(), sep = "\n")

sapply(seq(1, length(mytext), 2), function(x) paste(mytext[x], mytext[x   1], sep = ","))
[1] "Viruses/GCF_000820355.1_ViralMultiSegProj14361_genomic.fna.gz,Sclerophthora macrospora virus A"
[2] "Viruses/GCF_000820495.2_ViralMultiSegProj14656_genomic.fna.gz,Influenza B virus RNA"           
[3] "Viruses/GCF_000837105.1_ViralMultiSegProj14079_genomic.fna.gz,Tomato mottle virus"

CodePudding user response：

Using sed

$ sed '/^Viruses/{N;s/\n\(.*\)/,\1/}' multi_headers
Viruses/GCF_000820355.1_ViralMultiSegProj14361_genomic.fna.gz,Sclerophthora macrospora virus A
Viruses/GCF_000820495.2_ViralMultiSegProj14656_genomic.fna.gz,Influenza B virus RNA
Viruses/GCF_000837105.1_ViralMultiSegProj14079_genomic.fna.gz,Tomato mottle virus

/^Viruses/ - Match lines starting with the string Viruses
{N; - Read/append the next line of input into the pattern space.
s/\n\(.*\)/,\1/ - Remove the \n from the pattern space and replace it with a comma ,

CodePudding user response：

This might work for you (GNU sed and paste):

sed 'N;s/\n/,/' file

Append the next line to the current line and replace the newline between then with a comma.

or:

paste -sd',\n' file

Paste the file as one long string, replacing every other newline with a comma.

CodePudding user response：

What about this.

with open('test.txt') as f:
    data = f.read().split('\n')
new_data = []

for a in range(0,len(data),2):
    new_data.append(data[a] ',' data[a 1] '\n')
    
with open('result.txt','w') as f:
    f.writelines(new_data)

or

with open('test.txt') as f_read, open('result.txt','w') as f_write:
    data = f_read.read().split('\n')
    new_data = []

    for a in range(0,len(data),2):
        new_data.append(data[a] ',' data[a 1] '\n')

    f_write.writelines(new_data)

CodePudding user response：

A simple writerows():

import csv

with open("text.txt", "r") as f:
    with open("data.csv", "w", newline="") as w:
        writer = csv.writer(w)
        # May want to add headers
        writer.writerow(["Heading1", "Heading2"])
        x = f.read().split("\n")
        writer.writerows([x[i:i 2] for i in range(0, len(x), 2)])

Which yields:

Heading1,Heading2
Viruses/GCF_000820355.1_ViralMultiSegProj14361_genomic.fna.gz,Sclerophthora macrospora virus A
Viruses/GCF_000820495.2_ViralMultiSegProj14656_genomic.fna.gz,Influenza B virus RNA
Viruses/GCF_000837105.1_ViralMultiSegProj14079_genomic.fna.gz,Tomato mottle virus

CodePudding user response：

Another R approach, relying on vector recycling.

t = readLines("txt.txt")
paste0(t[c(T, F)], ",", t[c(F, T)]) |> writeLines("txt.csv")

or for final file names

t = readLines("R/txt.txt")
sub("(?<=\\.\\d).*", "", t, perl = T) |>
  (\(.) paste0(.[c(T, F)], "/", .[c(F, T)], ".fna"))()

#> [1] "Viruses/GCF_000820355.1/Sclerophthora macrospora virus A.fna"
#> [2] "Viruses/GCF_000820495.2/Influenza B virus RNA.fna"           
#> [3] "Viruses/GCF_000837105.1/Tomato mottle virus.fna"

CodePudding user response：

Simple python3 solution, let file.txt content be

Viruses/GCF_000820355.1_ViralMultiSegProj14361_genomic.fna.gz
Sclerophthora macrospora virus A
Viruses/GCF_000820495.2_ViralMultiSegProj14656_genomic.fna.gz
Influenza B virus RNA
Viruses/GCF_000837105.1_ViralMultiSegProj14079_genomic.fna.gz
Tomato mottle virus

and script.py

with open("file.txt","r") as f:
    for inx, line in enumerate(f):
        print(line.rstrip(), end='\n' if inx%2 else ',')

then

python script.py

output

Viruses/GCF_000820355.1_ViralMultiSegProj14361_genomic.fna.gz,Sclerophthora macrospora virus A
Viruses/GCF_000820495.2_ViralMultiSegProj14656_genomic.fna.gz,Influenza B virus RNA
Viruses/GCF_000837105.1_ViralMultiSegProj14079_genomic.fna.gz,Tomato mottle virus

Explanation: I use .rstrip to jettison trailing newline, then depending on whatever line is odd or even I apply \n or , respectively as line end. Note that enumerate default is starting at 0 as opposed to GNU AWK starting at 1. Note that using for ... in filehandle does prevent loading whole file as once, so this solution could be used also for files bigger than available RAM space.