Questions tagged [sparse-files]
69 questions
38
votes
9 answers
Clone whole partition or hard drive to a sparse file
I like to clone a whole partition or a whole hard drive onto a larger external disk but like to create a sparse file. I often use dd for cloning, but it doesn't support sparse files. As a workaround I used something like:
cp --sparse=always <(dd…
Martin Scharrer
- 900
- 1
- 12
- 16
33
votes
4 answers
Can a file that was originally sparse and then expanded be made sparse again?
I know that copying or transferring what was originally a sparse file without using a utility that understands sparse files will cause the 'holes' to be filled out. Is there a method or utility to turn what was once a sparse file back to sparse?
For…
user25849
- 491
- 5
- 4
20
votes
4 answers
Is there a reason why /var/log/lastlog is a huge sparse file (1.1TB)?
I have read some question, that ask advice how to rsync sparse files efficiently mentioning the files /var/log/lastlog and /var/log/faillog. Indeed I myself have stumpled over those files being an "issue" as their being backup via rsync turns them…
humanityANDpeace
- 13,722
- 13
- 61
- 107
18
votes
2 answers
How do I uncompress a file with lots of zeroes as a sparse file?
I have a compressed raw image of a very large hard drive created using cat /dev/sdx | xz > image.xz. However, the free space in the drive was zeroed before this operation, and the image consists mostly of zero bytes. What's the easiest way to…
jaymmer - Reinstate Monica
- 775
- 1
- 5
- 13
18
votes
3 answers
How can you trim files using the command line?
What is the console equivalent of the following Python code:
target = file("disk", "w") # create a file
target.seek(2*1024*1024*1024) # skip to 2 GB
target.write("\0")
target.close()
Maybe some dd incantation? The idea is making a file with the…
badp
- 2,977
- 2
- 24
- 32
17
votes
5 answers
Creating an arbitrarily large "fake" file
I would like to create a special file similar to /dev/null or /dev/random, where the file doesn't actually exist but you can read from it all the same, except that I could actually set a cap on the apparent size of the file.
To put it another way,…
Mediocre Gopher
- 272
- 1
- 2
- 6
15
votes
1 answer
What could explain this strange sparse file handling of/in tmpfs?
On my ext4 filesystem partition I can run the following code:
fs="/mnt/ext4"
#create sparse 100M file on ${fs}
dd if=/dev/zero \
of=${fs}/sparse100M conv=sparse seek=$((100*2*1024-1)) count=1 2> /dev/null
#show its actual used size before
echo…
humanityANDpeace
- 13,722
- 13
- 61
- 107
15
votes
3 answers
How do I output "sparseness" of a file?
How do I output how much of file nominal size is actually filled with data? Like vmtouch shows how much of file is currently in memory...
I expect the workflow to be like this:
$ fallocate -l 1000000 data
$ measure_sparseness data
100%
$ fallocate…
Vi.
- 5,528
- 7
- 34
- 68
13
votes
1 answer
Are there any disadvantages of `cp --sparse=always`?
Is there any reason not to use use --sparse=always with every invocation to cp?
info cp says:
‘--sparse=WHEN’
A “sparse file” contains “holes”—a sequence of zero bytes that does
not occupy any physical disk blocks; the ‘read’ system call…
Tom Hale
- 28,728
- 32
- 139
- 229
13
votes
1 answer
Converting sparse file to non-sparse in place
On Linux, given a sparse file, how to make it non-sparse, in place?
It could be copied with cp --sparse=never ..., but if the file is say 10G and the hole is 2G
(that is the allocated space is 8G), how to make the filesystem allocate the remaining…
Ivan
- 233
- 2
- 5
11
votes
3 answers
Detailed sparse file information on Linux
I have a sparse file, in which only some blocks are allocated:
~% du -h --apparent-size example
100K example
~% du -h example
52K example
I would like to know which blocks of the file are actually allocated. Is there a system call or kernel…
Juliano
- 2,468
- 18
- 20
11
votes
2 answers
Make grown extracted tar file small again
I packed and compressed a folder to a .tar.gz archive.
After unpacking it was nearly twice as big.
du -sh /path/to/old/folder = 263M
du -sh /path/to/extracted/folder = 420M
I searched a lot and found out that tar is actually causing this…
Permittivity
- 113
- 5
11
votes
1 answer
zerofree verbose returns what?
zerofree -v /dev/sda1 returned
123642/1860888/3327744.
The man page does not explain what those numbers are:
http://manpages.ubuntu.com/manpages/natty/man8/zerofree.8.html
I found some code on…
plamtrue
- 123
- 2
- 8
10
votes
2 answers
What is the difference between `fallocate --dig-holes` and `fallocate --punch-hole` in Linux?
I totally understand that --dig-holes creates a sparse file in-place. That is, if the file has holes --dig-holes options removes those holes:
Let's take it in a very simplified way, let's say we have a huge file named non-sparse:…
direprobs
- 944
- 14
- 29
10
votes
2 answers
How to make ext4 filesystem sparse?
I am using Linux as guest OS in VirtualBox. I deleted huge number of files from its filesystem. Now i want to shrink the filesystem image file (vdi). The shrinking works by compressing filesystem image wherever it has "null" value in disk.
It seems…
rajaganesh87
- 1,037
- 2
- 13
- 27