Table of Contents
Task
Given a bunch of files in a directory, count the number of times a word occurs in each file. For example, given
% tail -n +1 * ==> junk1.txt <== foo bar foo bar foo bar foo bar ==> junk2.txt <== foo bar foo bar foo bar foo bar foo foo foo
count the number of occurrences of 'foo' in each file. The expected answer is
junk1.txt:4 junk2.txt:7
tags | Number of matches per file
sample code demoes | cat with filename
Solution using git grep and awk
If it is a git repository
git grep -o foo | awk -F':' '{freq[$1]++} END{for (file in freq) print file ":" freq[file]}'
If it is not a git repository
git grep --no-index -o foo | awk -F':' '{freq[$1]++} END{for (file in freq) print file ":" freq[file]}'
For example
% git grep --no-index -o foo | awk -F':' '{freq[$1]++} END{for (file in freq) print file ":" freq[file]}'
junk2.txt:7
junk1.txt:4
How it works
The git grep command gives
% git grep --no-index -o foo junk1.txt:foo junk1.txt:foo junk1.txt:foo junk1.txt:foo junk2.txt:foo junk2.txt:foo junk2.txt:foo junk2.txt:foo junk2.txt:foo junk2.txt:foo junk2.txt:foo
The awk command counts the number of hits per file.
References
- https://stackoverflow.com/questions/39945363/frequency-count-for-file-column-in-bash - count frequencies using awk
tags
awk frequency count, awk count breakdown, uniq reverse output, “git grep” count matches, count “grep -o”, “grep -o” counts, “grep -o” summarize
Solution using grep and awk
grep -ro foo * | awk -F':' '{freq[$1]++} END{for (file in freq) print file ":" freq[file]}'
Useful if git is not available.
Solution using find, grep and wc
find * -printf 'echo "%p:$(grep -o "foo" %p | wc -l)";' | sh
For example
% find * -printf 'echo "%p:$(grep -o "foo" %p | wc -l)";' | sh junk1.txt:4 junk2.txt:7
How it works
To see how it works, run the command without piping the output to sh
% find * -printf 'echo "%p:$(grep -o "foo" %p | wc -l)";' echo "junk1.txt:$(grep -o "foo" junk1.txt | wc -l)";echo "junk2.txt:$(grep -o "foo" junk2.txt | wc -l)";
So we are just building up a big command that would run “grep -o” on each file and then format the output.
find * - find the files -printf '' - format and print everything between the single-quotes. %p in -printf - will be replaced by the filename in find's output grep -o - print only the matched (non-empty) parts of a matching line, with each such part on a separate output line. | sh - execute the command
Note: You have to use “grep -o” and not “grep -c”. If a string occurs multiple times in a line, “grep -o” matches each of them separately. But “grep -c” counts them together. For example
% cat junk2.txt foo bar foo bar foo bar foo bar foo foo foo % grep foo junk2.txt foo foo bar foo bar foo bar foo foo foo % grep -o foo junk2.txt foo foo foo foo foo foo foo % grep -c foo junk2.txt 4 % grep -o foo junk2.txt| wc -l 7
References
- https://newbedev.com/counting-all-occurrences-of-a-string-within-all-files-in-a-folder - where I came across this solution.