#20 c-icap empty responses with 25 bytes sent

[jammy]

I am troubleshooting an issue with my c-icap setup where i see a number of responses with just 25bytes sent and the client code receives an empty response. My setup is as follows:

custom java icap client (not developed by me) -> OCI LBaaS -> two instances with c-iap + clamd

All failing request have one thing in common, the bytes sent (according to logs) are 25bytes with return code 200 .

c-icap-access.log:

21/May/2025:13:44:47 -0700, 10.40.51.156 10.40.44.168 RESPMOD 200 17320 25 http://10.40.44.168/20250521134447/tmp_2.js -

21/May/2025:13:45:09 -0700, 10.40.51.156 10.40.44.168 RESPMOD 200 17321 25 http://10.40.44.168/20250521134509/checkout_2.css -

21/May/2025:13:45:12 -0700, 10.40.51.156 10.40.44.168 RESPMOD 200 17332 25 http://10.40.44.168/20250521134512/templates_2.js -

I increased the verbosity of logs and was able to observe the following error when these 25_bytes responses occur

21/May/2025:04:05:35 -0700, 10.34.18.14 10.34.13.244 RESPMOD 200 522 25 http://10.34.13.244/20250521040535/Balance30_Project8+IE-09MAY-test+dev+oos+john%C3%BRwez+-+BUILDS+NA+CLOUD-78123 - 0 0 HTTP/1.1 200 OK[Content-Length] -[Encapsulated] - 1747825535 RESPMOD icap://clamav.example.internal:1344/avscan?profile=default ICAP/1.0 RESPMOD icap://clamav.example.internal:1344/avscan?profile=default ICAP/1.0[User-Agent] HTTP/1.1 200 OK - 

The error from server.log:

Wed May 21 04:05:35 2025, 3995280/3671443200, service arguments:allow204=on&sizelimit=off&mode=simple
Wed May 21 04:05:35 2025, 3995280/3671443200, Request type: 4. Preview size:512
Wed May 21 04:05:35 2025, 3995280/3671443200, service arguments:allow204=on&sizelimit=off&mode=simple
Wed May 21 04:05:35 2025, 3995280/3671443200, OK; the preview data size is 0
Wed May 21 04:05:35 2025, 3995280/3671443200, Preview handler continue reading more body data
Wed May 21 04:05:35 2025, 3995280/3671443200, Parse error:count=0,start=;
Wed May 21 04:05:35 2025, 3995280/3671443200, Error parsing chunks!
Wed May 21 04:05:35 2025, 3995280/3671443200, An error occured. Parse error or the client closed the connection (res:-1, preview status:1)
Wed May 21 04:05:35 2025, 3995280/3671443200, Releasing virus_scan data.....
Wed May 21 04:05:35 2025, 3995280/3671443200, There are unparsed data od size 10: "; ieof^M
^M
"
. Move to connection buffer

The only thing i changed recently was to enable TLS port using the following syntax

TlsPort 0.0.0.0:13440 cert=/etc/pki/clamav/chain.crt  key=/etc/pki/clamav/private.key tls-options=SSL_OP_NO_TLSv1|SSL_OP_NO_TLSv1_1 ciphers=ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-RSA-AES128-GCM-SHA256

Some observations and facts

• I can see those failing requests regardless if it is TLS connection or plain text
• when the client was set to use TLS connections, I noticed these responses increased a lot. Yet i can see such responses with non TLS connections
• i also noticed my server log contains a bunch of

Wed May 21 01:59:32 2025, 3995276/3730192128, Width: 0, Parameter:
Wed May 21 01:59:32 2025, 3995276/3730192128, Width: 0, Parameter:virus_scan:virus
Wed May 21 01:59:32 2025, 3995276/3730192128, Width: 0, Parameter:
Wed May 21 01:59:32 2025, 3995276/3730192128, Width: 0, Parameter:
Wed May 21 01:59:32 2025, 3995276/3730192128, Width: 0, Parameter:
Wed May 21 01:59:32 2025, 3995276/3730192128, Width: 0, Parameter:
Wed May 21 01:59:32 2025, 3995276/3730192128, Width: 0, Parameter:
Wed May 21 01:59:32 2025, 3995276/3730192128, Width: 0, Parameter:
Wed May 21 01:59:32 2025, 3995276/3730192128, Width: 0, Parameter:

Any idea how to approach this and figure out where is the actual problem ? Is it on the client code? is there an issue with the Loadbalacner?