Sponge – A Linux tool which makes you feel you missed it for years

How often do you come across a situation where you need to parse a file, change some value and write the content back to the file? This mostly occurs when you are dealing with configuration files, which requires some script automated editing. More often than not you will involve a temperory file to do the intermediate changes and then later overwrite the original file. The reason you use temporary  file is because if you use  pipes and redirections then the output content stream starts flowing  before the input stream gets completed. To understand lets look at this example

Here is the content of the file pattern.txt

I
test
this
code
with
code

Now lets try to replace the string “test” with “tested” and remove the last two lines.

$sed "s/test/tested/" pattern.txt|head -n -2
I
tested
this
code

Now lets write the resulting output back to the same file i.e pattern.txt and see what
happens.

 $sed "s/test/tested/" pattern.txt|head -n -2 >pattern.txt 
$cat pattern.txt 

You see the file is empty thats why a temperory file is used.

$sed "s/test/tested/" pattern.txt|head -n -2>tmp.txt
$cp tmp.txt pattern.txt

But the above solution is a two step process and it a work around as compared to a one step process. Now lets try to solve this problem using the command sponge. sponge reads standard input and writes it out to the specified file.
Unlike a shell redirect,sponge soaks up all its input before opening the output file. This allows constricting pipelines that read from and write to the same file.

sudo apt-get install moreutils

$sed "s/test/tested/" pattern.txt|head -n -2|sponge pattern.txt
$cat pattern.txt
I
tested
this
code

Now the stream is piggybacked by sponge smoothly such that we neither need a temporary file
nor need two steps of execution.

While I was testing the examples for the blog I overwrote my pattern file many times.
But I had a backup of that file, So my piece of advice is always to have a backup of
what you are working whether you are working with pattern.txt or Pixar.

Playing with your original file is like “Cutting a branch on which you are sitting”