Menu ▾ ▴

Post requests rom squid fails

Open Discussion
Fatih Akin
2010-10-03
2013-04-26

  • BrimarX

    BrimarX - 2010-10-08

    If you have a problem please post:
    1. A problem description
    2. AdaptIt and SQUID version
    3. AdaptIt and SQUID configuration
    4. AdaptIt and SQUID logs in TRACE mode for AdaptIt
    5. A snippet of the IcapService you use

    By the way, have you tested with the EchoService?

    FYI AdaptIt works fine with SQUID with all modes but PREVIEW (could work though, but not tested).

     

  • Fatih Akin

    Fatih Akin - 2010-10-09

    I use adapit 0.2.31. jar and squid version is 3.1.8 and  i use it in preview mode with preview size 0
    no problem with repsmod mode , but theresi problem wiht some of the post request to nimbuz and 0.facebook site
    even if i reply with 204 node modification.

    I also teste your echoservice,(again in preview mode with 0 preview size).
    There is no failure in icap trace logs. It is huge since there is too much traffic.  Its hard for us to take traces of the failed request on both squid and on adaptit side.

    Is there any improevemnt on adaptit-0.3 jar for this case .
    Thanks for your help. We need urgent help. The necessary configuration and the icap service i use is decribed below.

    squid conf is

    icap_service service_1 reqmod_precache 0 icap://localhost:1344/ttrm
    adaptation_service_set service_set_1 service_1
    adaptation_access service_set_1 allow all

    icap_service service_2 respmod_precache 0 icap://localhost:1344/ttrps
    adaptation_service_set service_set_2 service_2
    adaptation_access service_set_2 allow all

    #acl ttech1 external ttech
    acl manager proto cache_object
    acl localhost src 127.0.0.1/32
    acl to_localhost dst 127.0.0.0/8
    acl csd src 86.108.154.32/27
    acl localnet src 10.0.0.0/8     # RFC1918 possible internal network
    acl localnet src 172.16.0.0/12  # RFC1918 possible internal network
    acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
    acl SSL_ports port 443
    acl Safe_ports port 80          # http
    acl Safe_ports port 21          # ftp
    acl Safe_ports port 443         # https
    acl Safe_ports port 70          # gopher
    acl Safe_ports port 210         # wais
    acl Safe_ports port 1025-65535  # unregistered ports
    acl Safe_ports port 280         # http-mgmt
    acl Safe_ports port 488         # gss-http
    acl Safe_ports port 591         # filemaker
    acl Safe_ports port 777         # multiling http
    acl CONNECT method CONNECT

    http_access allow manager localhost
    http_access deny manager
    http_access deny !Safe_ports
    http_access deny CONNECT !SSL_ports
    http_access allow localnet
    http_access allow csd
    #http_access allow ttech1
    http_access deny all
    visible_hostname bran
    http_port 8080
    hierarchy_stoplist cgi-bin ?
    refresh_pattern ^ftp:           1440    20%     10080
    refresh_pattern ^gopher:        1440    0%      1440
    refresh_pattern -i (/cgi-bin/|\?) 0     0%      0
    refresh_pattern .               0       20%     4320
    coredump_dir /usr/local/squid/var/cache
    cache_mem 0 MB

    adaptit.properties

    adaptit.shutdown-listener.bind.host=localhost
    adaptit.shutdown-listener.bind.port=1344

    # Set to true to reassemble TCP/IP fragmented buffer; defaults to false.
    adaptit.icap.buffer.reassembly = true

    adaptit.icap.cnx.timeout.inactive = 10000
    adaptit.icap.cnx.timeout.keep-alive = 30000
    adaptit.icap.cnx.reuse = true
    adaptit.icap.cnx.dispatchers = 40

    adaptit.icap.bind.host=127.0.0.1
    adaptit.icap.bind.port=1344

    adaptit.icap.headers.all.Options-TTL=36000
    adaptit.icap.headers.all.Service-ID=toto
    adaptit.icap.headers.all.Mock-Header=mock-value

    adaptit.icap.headers.options.OptMock1=v1
    adaptit.icap.headers.options.OptMock2=v2

    adaptit.icap.service.ttrm=com.ttech.proxy.icap.IcapTTReqMod204Service
    adaptit.icap.service.ttrm.path=/ttrm

    #adaptit.icap.service.ttrm=com.ttech.proxy.icap.IcapTTReqModService
    #adaptit.icap.service.ttrm.path=/ttrm

    adaptit.icap.service.ttrps=com.ttech.proxy.icap.IcapTTRespPreviewService
    adaptit.icap.service.ttrps.path=/ttrps

    #
    # icap.req.method    ICAP request method string.
    # icap.req.url       ICAP request URL
    # icap.res.status    ICAP response status code.
    # icap.res.reason    ICAP response reason.
    adaptit.icap.logs.format=$(icap.req.method)\t$(icap.req.uri)\t$(icap.res.status)

    # Enable lazy parsing for NetCache compatibility.
    # (NetCache 6.0.1 and previous do not comply with the ICAP 1.0 RFC)
    adaptit.icap.parser.mode=lazy

    Icap service:

    public void doReqmod(RequestEvent event, IcapReqmodRequest request, IcapReqmodResponse response, IcapTransactionHandler handler)
    throws IcapServiceException {

    try {
    switch (event) {

    case REQUEST_COMPLETE:
    icapTotalCounter.increment();

    boolean isRejected = TTechProxyEnvironment.getResponseComposer().checkAccess(request, response, handler);

    if (isRejected) {

    response.setStatus(200);
    response.setReason("Rejected");
    /* if rejected  request header buffer is set to null , and response header buffer and response body buffer has been updated
    * so just update body buffer to null.*/
    response.setEncapsulatedRequestBodyBuffer(null);
    CommonEnvironment.getMainLogger().debug("Rejected.200");
    icapRejectCounter.increment();

    } else {
    /* not rejected, check if info forward enabled. if not just send 204 */
    if (!TTechProxyEnvironment.getRequestModifier().isInfoForward()){
    /* for 204 just set response header buffer to null. Do not care for other buffers*/
    response.setStatus(204);
    response.setReason("Info Fwd Disabled,Req NOT Modified,Sent 204");
    response.setEncapsulatedResponseHeaderBuffer(null);
    CommonEnvironment.getMainLogger().debug("Info Fwd Disabled,Req NOT Modified,Sent 204");
    icapInfoDisabledCounter.increment();
    } else {
    boolean isModified =  TTechProxyEnvironment.getRequestModifier().modifyRequestForInfoFwd(response, request);
    if (isModified){
    /* request header buffer updated with cookie ?*/
    response.setStatus(200);
    response.setReason("Info fwd.done.Req Modified");
    CommonEnvironment.getMainLogger().debug("Info fwd.done.Req Modified");
    response.setEncapsulatedRequestBodyBuffer(null);
    icapInfoFwdCounter.increment();

    } else {
    response.setStatus(204);
    response.setReason("No info fwd for url, Req NOT Modified, Sent 204");
    CommonEnvironment.getMainLogger().debug("No info fwd for url, Req NOT Modified, Sent 204");
    response.setEncapsulatedResponseHeaderBuffer(null);
    icapNoInfoFwdForUrlCounter.increment();
    }
    }

    }

    handler.getResponseHandler().handle(IcapResponseHandler.ResponseEvent.SEND_BODY_COMPLETE, response, null);
    break;

    default:

    break;
    }
    } catch (Exception e) {
    SquidHttpRequest httpRequest = ICAPUtils.decodeIcapRequest(request.getEncapsulatedRequestHeaderBuffer());
    TTechIcapServer.getIcapLogger().debug("Error Handled for request "+ httpRequest +" Error: " + e.getMessage());
    throw new IcapServiceException(e);
    }
    }

     

  • Fatih Akin

    Fatih Akin - 2010-10-10

    I retested my applicatron in normal mode, without preview mode with icapechoservice.
    It again gets the same error for some post requests. I decided not to use reqmod, and now i use just respmod and
    i have no problem for now. But i need to fix this request mode also. Because i need to change http request headers.

    Another very importan problem is.
    Again wiht the same squid and icap configuration. The system only supports 50 requests per second from squid.
    If the traffic increases , i get too much icap service failure errors. And when i grep with netstat the open connections between squid and my icap server, i see to much open connections lik 2000, 2500 and … it increases if i dont stop the traffic.

    I need to handle at least 500 requests per second from squid .

    I need urgent support.

    It would be great if you can help me solve this problem.

    Best Regards

     

Log in to post a comment.