Forum Discussion

Daniel_S_135288's avatar
Daniel_S_135288
Icon for Nimbostratus rankNimbostratus
Jun 25, 2015

catch bots

hey guys

can some one explain me why my irule does not work

when LB_SELECTED {
set uagent [string tolower [HTTP::header User-Agent]]  
if { [matchclass $uagent contains datagroup_crawler] } {

        pool pool_webserver_ch_de_bot

        log local0. "using pool_webserver_ch_de_bot for UA $uagent"
        }
   }

DG:

modify internal datagroup_crawler {
records replace-all-with {
    AppleWebKit { }
    SeznamBot { }
    YandexBot/3.0 { }
    ahrefsbot { }
    applebot { }
    applewebkit { }
    bingbot { }
    bitlybot { }
    cliqzbot { }
    dotbot { }
    exabot { }
    googlebot { }
    konqueror { }
    linkdexbot { }
    linkedInbot/1.0 { }
    livelapbot { }
    mail.ru_bot { }
    mj12bot { }
    mozilla { }
    msnbot { }
    msnbot-media { }
    openhosebot { }
    orangebot { }
    paperlibot { }
    rogerbot { }
    searchmetricsbot { }
    seznambot { }
    showyoubot { }
    slurp { }
    smtbot { }
    spiderbot { }
    ssearch_bot { }
    trovitbot { }
    tweetmemebot { }
    twitterbot { }
    twitterbot/1.0 { }
    umbot-ln { }
    uptimerobot { }
    woobot { }
    yandexbot { }
    yandexbot/3.0 { }
}
}

i have just some matches for googlebot and bingbot...

thanks

application delivery
devops
iRules

3 Replies

Sort By
  • Daniel_S_135288's avatar
    Daniel_S_135288
    Icon for Nimbostratus rankNimbostratus
    Jun 26, 2015

    hey Nathan

     

    Thanks for your answer. I tryd it first with HTTP_REQUEST, but i had the same results like LB_SELECTED..

     

    our LoadBalancers running on Version 11.6.0, and i use several other iRules with matchclass.

     

    what would you use instead of matchclass?

     

    thanks daniel

     

  • nathe's avatar
    nathe
    Icon for Cirrocumulus rankCirrocumulus
    Jun 26, 2015

    It may be the matchclass thing that prevented it working with HTTP_REQUEST. The CLASS command replaces matchclass. See: matchclass and class

    This should do it:

    when HTTP_REQUEST

   set user_agent [string tolower [HTTP::header User-Agent]]

   if { [class match $user_agent contains datagroup_crawler] } {

      pool pool_webserver_ch_de_bot

      log local0. "using pool_webserver_ch_de_bot for UA $uagent"

        }
   }