3

I have a directory where there are multiple folders, each folder contains multiple .gz files with the same zipped file name "spark.log". How can I unzip all of them at once and rename them like the gz file?

My data looks like this

List of folders

A B C D

In every of them there are files as

A spark.log.gz spark.log.1.gz spark.log.2.gz spark.log.3.gz B spark.log.gz spark.log.1.gz spark.log.2.gz spark.log.3.gz C spark.log.gz spark.log.1.gz spark.log.2.gz spark.log.3.gz D spark.log.gz spark.log.1.gz spark.log.2.gz spark.log.3.gz

in each of the gz file contains spark.log, I'd like to be able to unzip and rename them according to their gz name. For example: spark.log.1.gz -> spark.log.1.log

Improve this question
2
  • 2
    If you are uncompressing the logs so that you can search them, keep in mind that there are tools like zcat and zgrep that can read gzipped files without having to first uncompress them. Commented Oct 7, 2022 at 2:25
  • 2
    Using gunzip on spark.log.2.gz would give you spark.log.2, which is the name of the file that was compressed. I don't quite see why you want to add an extra .log to the end of that name. Commented Oct 7, 2022 at 9:03

2 Answers 2

Reset to default
2

While gzip does or can store the original name, which you can reveal by running gzip -Nl file.gz:

$ gzip spark.log $ mv spark.log.gz spark.log.1.gz $ gzip -l spark.log.1.gz compressed uncompressed ratio uncompressed_name 170 292 51.4% spark.log.1 $ gzip -lN spark.log.1.gz compressed uncompressed ratio uncompressed_name 170 292 51.4% spark.log

gunzip will not use that for the name of the uncompressed file unless you pass the -N option and will just use the name of the gzipped file with the .gz suffix removed.

You may be confusing it with Info-ZIP's zip command and its related zip format which is a compressed archive format while gzip is just a compressor like compress, bzip2, xz...

So you just need to call gunzip without -N on those files:

gunzip -- */spark.log*.gz

And you'll get spark.log, spark.log.1, spark.log.2... (not spark.log.1.log which wouldn't make sense, nor spark.1.log, which could be interpreted as a log file for a spark.1 service as opposed to the most recent rotation of spark.log).

Having said that, there's hardly ever any reason to want to uncompress log files. Accessing the contents is generally quicker when they are compressed. Modifying the contents is potentially more expensive, but you generally don't modify log files after they've been archived / rotated. You can use zgrep, vim, zless (even less if configured to do so) to inspect their contents. zcat -f ./*.log*(nOn) | grep... if using zsh to send all the logs from older to newer to grep, etc.

Improve this answer
1
  • great explanation, though the reason why i'd like to have .log extension is because I'm using mac os and I'd like to be able to click on the .log files to view them, rather than typing it out in terminal. So I suppose my question should be in mac stackexchange to see if there's an app that can view gzipped log files. Commented Oct 10, 2022 at 14:41
1

This bash script should do what you want:

#! /usr/bin/env bash for file in ./*/*.gz; do newfile="${file/%.gz/.log}" printf 'Extracting %s to %s\n' "$file" "$newfile" gzip -dc "$file" > "$newfile" # Use this instead if the .gz file should be deleted # gzip -dc "$file" > "$newfile" && rm "$file" done

You have to be in the path where you have your folders A,B,C,D, etc...

The line: newfile="${file/%.gz/.log}" assigns the new name of the file that will be renamed (e.g. spark.log.1.log should be the new filename if the gz file is spark.log.1.gz)
With ${file/%.gz/.log} I'm replacing the .gz at the end of the current file.

The printf statement is not necessary but could be useful if you want see a detailed output about what's happening in each iteration.

Using gzip -dc with a redirection means this will work without having to worry about the name of the extracted file.

Improve this answer
0

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.