Remove a single line from a very large file using sed

quinn

I am trying to remove a single line (the first match) towards the beginning of a 250GB file. The command i have is this:

sed -i '0,/matchstring/{/matchstring/d;}' file

This works great on a smaller file, but on the big file, it never exits. I understand there is q to tell sed to exit early, but I can't figure out how to add that into what I have here.

pynexj

Since OP mentioned in the comment that

... something that replaces it with blanks works great, the line doesn't need to be removed.

And the following solution assumes the line is near to the begining of the file (otherwise we need a bit more tricks to get the exact offset of the line and the intermediate "header" file can be avoided).

[STEP 101] $ cat file
hello 1
hello 2
hello 3
foo matchstring bar
hello 4
hello 5
hello 6
[STEP 102] $ sed '/matchstring/{s/./ /g;q;}' file > header
[STEP 103] $ cat header
hello 1
hello 2
hello 3

[STEP 104] $ dd conv=notrunc if=header of=file
0+1 records in
0+1 records out
44 bytes copied, 0.000892102 s, 49.3 kB/s
[STEP 105] $ cat file
hello 1
hello 2
hello 3

hello 4
hello 5
hello 6
[STEP 106] $

And if it works by converting the matched line into a comment line, just change s/./ /g to, for example, s/./#/ in the sed command.

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at
0

Comments

0 comments
Login to comment

Related

Remove line from php file using sed

Using integers from a large single line text file for testbench

Remove Empty Second Line From Very Large CSV File in Windows 10

Remove line from file in bash script using sed command

Removing a large multi-line string from a very large file

Using sed to retrieve and modify a single line in the file

Remove Duplicates On a Single Line From a Text File

How to speed-up sed that uses Regex on very large single cell BAM file

using sed to remove equals sign from a file

Remove parameters from configure file using sed

using sed in a for in loop to remove words from file

Using sed with regex to remove text from a file

Substring from very large text file using Powershell

How to remove similar elements from beginnign of each line in a text file using sed or similar?

Remove prefix of each line in a file and output to another file using sed

Search and write line of a very large file in bash

How to remove only the first occurrence of a line in a file using sed

how to remove both first and last line of csv file using sed

Remove single slash but not double slash from string using sed

remove line comment from file by using regex

how to remove last comma from line in bash using "sed or awk"

Is there a way to remove a string from a line of a csv using sed in bash scripts?

Delete a line using SED from a file and redirect the output to same file

git remove single line from file in git history

How To Read Line Numbers x Through (x+y) From A Very Large File

Most efficient way to parse every fourth line from a very large file

Efficient way of Reading a very large text file from a specific line number to the end in C#

Using Chardet to find encoding of very large file

sed: Removing \r\r before \n in a very large file