How often do you come across a situation where you need to parse a file, change some value and write the content back to the file? This mostly occurs when you are dealing with configuration files, which requires some script automated editing. More often than not you will involve a temperory file to do the intermediate changes and then later overwrite the original file. The reason you use temporary file is because if you use pipes and redirections then the output content stream starts flowing before the input stream gets completed. To understand lets look at this example
Here is the content of the file pattern.txt
I test this code with code
Now lets try to replace the string “test” with “tested” and remove the last two lines.
$sed "s/test/tested/" pattern.txt|head -n -2 I tested this code
Now lets write the resulting output back to the same file i.e pattern.txt and see what
$sed "s/test/tested/" pattern.txt|head -n -2 >pattern.txt $cat pattern.txt
You see the file is empty thats why a temperory file is used.
$sed "s/test/tested/" pattern.txt|head -n -2>tmp.txt $cp tmp.txt pattern.txt
But the above solution is a two step process and it a work around as compared to a one step process. Now lets try to solve this problem using the command sponge. sponge reads standard input and writes it out to the specified file.
Unlike a shell redirect,sponge soaks up all its input before opening the output file. This allows constricting pipelines that read from and write to the same file.
sudo apt-get install moreutils
$sed "s/test/tested/" pattern.txt|head -n -2|sponge pattern.txt $cat pattern.txt I tested this code
Now the stream is piggybacked by sponge smoothly such that we neither need a temporary file
nor need two steps of execution.
While I was testing the examples for the blog I overwrote my pattern file many times.
But I had a backup of that file, So my piece of advice is always to have a backup of
what you are working whether you are working with pattern.txt or Pixar.
Playing with your original file is like “Cutting a branch on which you are sitting”