- > The obvious way to do it is:by emptiestplace - 11 hours ago
> $ gzip -c access.log > access.log.gz
Is it?
- There's also `progress` which works for tools mainly operating on a single file, but unlike `pv`, you don't have to start the tool differently. It'd e.g. work nicely for the `gzip` example. Just call `progress` on a different terminal while `gzip` is running.by heinrich5991 - 10 hours ago
- Yes! My `,pv` is approximately: (probably a better way to make the k, but I stop once something is adequate; maybe I just need to make a `,,kof`)by thanatos519 - 10 hours ago
Which gives me progress bars for big copies like:tar cpS "$@" --sort=name | pv -bratpes $(du -cks "$@"|sed -n '/.total$/ s/.total$//p')k,pv files/ | tar xp -C /destination ,pv files/ | ssh overthere tar xp -C /destination - Pipe viewer is excellent. I use it all the time.by mort96 - 9 hours ago
As of version 1.8.10[1], which includes my merge request[2] to add an '--output' option, it has even completely replaced my use of 'dd' for writing disk images: 'sudo pv -Yo /dev/mmcblk0 whatever.img' is nicer, has much better progress indication, automatically selects a more sensible buffer size, and begets fewer groans from UNIX neckbeards, than the old 'sudo dd of=/dev/mmcblk0 if=whatever.img'. (The '-Y' causes pv to sync after each write, which greatly improves progress indication in Linux.)
Though it's useful for much more of course. I use it for progress when compressing files ('pv blah | gzip ...'), when uploading files to the web ('pv blah | curl --upload-file - ...' — curl doesn't show progress when uploading for whatever reason), or just when I wanna see that something is happening with an operation which would otherwise take a while (even things like a slow 'du -h /some/path | sort -h' benefits from a 'pv' squeezed in the middle just to indicate that something is happening).
- A little more typing, but I find dd present on most systems already, so I tend to do this:by 6c696e7578 - 9 hours ago
tar ... | dd status=progress | ... - Pipe viewer? What's that? Let me check the post...oh, it's good old pv! Never noticed it had a full name, damn Unix utilities with their short names!by darkwater - 8 hours ago
- See also vipe(1) from the wonderful moreutils: https://manpages.debian.org/stretch/moreutils/vipe.1.en.htmlby ElevenLathe - 8 hours ago
- We have that in powershell also show-progressby sirjaz - 8 hours ago
- by mac3n - 7 hours ago
- pv is great.by jcul - 6 hours ago
It has a limit parameter so you can limit the speed. Great if you don't want to saturate some link or have additional costs for uploading above a certain rate per hour/day.
Also useful for testing behaviour on slow filesystem / connections.
It can take a pid argument too, -d IIRC, which will get it to display progress info for all the open file descriptors of a running process.
Really useful as a quick way to check what a IO process is doing if appears to be stuck.
- I love pv but how much does adding the pipe affect overhead? I feel like most of my big jobs I want to measure are on things where you want the program to have direct access to the underlying file or storage. `pv somefile | dd` is going to be slower than `dd somefile`. At least I think so? I have no idea what modern Linux I/O can optimize.by NelsonMinar - 6 hours ago
Also does pv necessitate doing single threaded I/O?
- Got me wondering, how does it works?by c0deR3D - 6 hours ago
- I like to use pv as a quick and dirty operations per second counter. Sometimes I will write a program or script that does a bunch of things in parallel (e.g. RPCs to a service I'm working on), and prints one line of output for every operation completed. Then I pipe that output to pv using the --lines option to count only lines. It shows how many lines are being printed per second, which roughly counts operations per second. (IIRC, also need to pipe to /dev/null to prevent pv's fancy output from clobbering the tool's output).by throwaway127482 - 5 hours ago
Fun stuff! Especially when combined with GNU parallel, in cases where the thing I'm measuring isn't already parallelized, and I want to be lazy.