Apache pipeline ignores log entries that do not contain "method url protocol" fields

[cvubrugier]

Hi,

Thank you for developing and maintaining Angle-grinder.

• Angle-grinder version: v0.19.2
• Angle-grinder binary: agrind-x86_64-unknown-linux-gnu.tar.gz
• OS: Debian testing (currently "trixie")
• Apache version: 2.4.57

I used Angle-grinder to count the number of unique IPs that connect to my Apache Web server and observed that the result differ between Angle-grinder and Awk:

$ awk '{print $1}' apache.log | sort -u | wc -l
5307

$ agrind  -f apache.log "* | apache | count_distinct(ip)"
_countDistinct
----------------------
5296

Angle-grinder ignored 11 log entries. I looked at the log entries skipped by Angle-grinder and observed that the "method url protocol" fields that follow the timestamp do not contain 3 values.

134.122.106.248 - - [29/Sep/2023:20:01:55 +0200] "" 400 5253 "-" "-"
79.40.161.137 - - [25/Sep/2023:19:23:59 +0200] "-" 408 4461 "-" "-"
118.189.186.52 - - [18/Sep/2023:10:36:45 +0200] "-" 408 4461 "-" "-"
146.148.12.228 - - [18/Sep/2023:20:21:35 +0200] "-" 408 4680 "-" "-"
136.226.101.20 - - [14/Sep/2023:22:32:37 +0200] "-" 408 4739 "-" "-"
69.242.2.103 - - [08/Sep/2023:06:34:39 +0200] "User User" 400 5455 "-" "-"
34.76.242.182 - - [07/Sep/2023:22:36:00 +0200] "-" 408 4678 "-" "-"
113.120.246.135 - - [06/Sep/2023:23:06:07 +0200] "-" 408 4467 "-" "-"
116.205.147.236 - - [05/Sep/2023:08:16:12 +0200] "-" 408 4463 "-" "-"
116.205.147.236 - - [05/Sep/2023:08:17:26 +0200] "-" 408 4463 "-" "-"
218.90.96.98 - - [04/Sep/2023:06:03:43 +0200] "-" 408 4463 "-" "-"
106.75.129.136 - - [04/Sep/2023:12:51:23 +0200] "{\"method\":\"login\",\"params\":{\"login\":\"45JymPWP1DeQxxMZNJv9w2bTQ2WJDAmw18wUSryDQa3RPrympJPoUSVcFEDv3bhiMJGWaCD4a3KrFCorJHCMqXJUKApSKDV\",\"pass\":\"xxoo\",\"agent\":\"xmr-stak-cpu/1.3.0-1.5.0\"},\"id\":1}\n" 400 5455 "-" "-"

Is this expected?

[rcoh]

yeah seems like some modifications to the apache built-in are required