Questions tagged [efficiency]
15 questions
45
votes
4 answers
Reverse grepping
Let's say, I have a really big text file (about 10.000.000 lines). I need to grep it from the end and save result to a file. What's the most efficient way to accomplish task?
user78433
33
votes
5 answers
How do I minimize disk space usage
One of my machines is the 2GB EeePC Surf, a neat netbook with very limited resources. So limited that right now, I have 22MB free space left.
On it, I'm running Arch Linux with the Openbox DE and a host of needed applications for it to function as a…
Stefan
- 24,830
- 40
- 98
- 126
28
votes
5 answers
Can I access Nth line number of standard output?
Assuming a script that outputs a list of files:
$> bash someScript.sh
path/to/some/file
path/to/the/file/i/want/to/work/with
path/to/yet/another/file
Now I want to have the second file-path as parameter for another command, e.g. vim. Is there a way…
k0pernikus
- 14,853
- 21
- 58
- 79
23
votes
4 answers
Are Linux utilities smart when running piped commands?
I was just running a few commands in a terminal and I started wondering, does Unix/Linux take shortcuts when running piped commands?
For example, let's say I have a file with one million lines, the first 10 of which contain hello world. If you run…
DisgruntledGoat
- 782
- 1
- 7
- 15
9
votes
3 answers
Why is `tac file | grep foo' (piping) faster than `grep foo < <(tac file)' (process substitution)?
This question was motivated by "Reverse grepping", about grepping a huge file from bottom up.
@chaos said:
tac file | grep whatever
Or a bit more effective:
grep whatever < <(tac file)
@vinc17 said:
The < <(tac filename) should be as fast as a…
c0rp
- 275
- 3
- 11
5
votes
1 answer
Process priority and cpu speed
In some devices the cpu speed is dynamic, being faster when there is more load.
I was wondering if it is possible to set nice level or priority of a process so that it does not influence an increase in cpu speed when it is running flat…
ctrl-alt-delor
- 27,473
- 9
- 58
- 102
5
votes
2 answers
How to install light-weight vim and to be able to efficiently load files into one instance vim?
On my laptop with Ubuntu I'm able to have just one instance of vim, launched with
vim --servername VIM
After this I'm able to open files from console with
vim --remote-silent filename
I use short aliases for both of them.
Also I do some…
fiktor
- 173
- 1
- 5
4
votes
2 answers
check patterns that don't exist in sqlite
I explained a similar situation with plain text files on Grep huge number of patterns from huge file. Many people there said I should, so now I'm migrating my data to a sqlite database:
I have a file from which I extract about 10,000 patterns. Then…
admirabilis
- 4,642
- 9
- 41
- 57
4
votes
1 answer
Efficiency of lots of inotify watches or stat calls
I am developing software that will utilize inotify to track changes on a large amount of files (tens to hundreds of thousands of files). I have come up with these ideas:
one watch per file
one watch per parent directory
avoid inotify and…
beatgammit
- 7,453
- 10
- 31
- 32
2
votes
3 answers
Compacting `find` name patterns
I am using
find . -name '*.[cCHh][cC]' -exec grep -nHr "$1" {} ';'
find . -name '*.[cCHh]' -exec grep -nHr "$1" {} ';'
to search for a string in all files ending with .c, .C, .h, .H, .cc and .CC listed in all subdirectories. But since this…
Arpith
- 1,091
- 3
- 14
- 23
2
votes
2 answers
How to compare usage of memory by Window Managers?
I'd like to test a few window managers to see which of them takes less memory and CPU in certain time period (so is better choice for older computers). I'd like to make the tests myself (not to read a review).
Which application for testing would you…
xralf
- 16,149
- 29
- 101
- 149
1
vote
1 answer
Do programs access tools like grep, sed, tr, or sort due to their efficiency?
Or it's meaningful only when you are in the command line and want something simple done right away.
Are these tools more efficient than a program library?
Quora Feans
- 3,806
- 7
- 29
- 46
0
votes
0 answers
centos cluster set core/each_program
I'm using unix server (OS: centos5) and I'm using several clusters.
Some clusters have 8 cores, other are 24 cores and etc.
I execute jobs using this command.
qsub -q cluster_name -d $PWD Program_name
To efficiently use clusters, I'm using OPENMP…
user2268721
- 21
- 2
0
votes
1 answer
Why doesn't rsync take advantage of the mv command when --remove-source-files is specified?
Assume I want to move a tree in the filesystem, e.g.
/path/to/a
a.txt
b.txt
to another location with rsync (3.0.9 protocol version 30), e.g. with rsync -a --remove-source-files /path/to/a/ /path/to/b/, why does rsync not find out that…
Kalle Richter
- 2,100
- 4
- 20
- 37
0
votes
1 answer
Check multiple directory properties
I'd like to make this test as fast as possible: (Bash)
if [[ -d $directory_path && -r $directory_path && -x $directory_path ]]
Maroloccio
- 145
- 6