Eireann78_19953
Aug 28, 2008Nimbostratus
Prevent Data Mining ASM and LTM
Hey,
We will be soon releasing some new apps on our website pulling data from a content management system. A worry we have is that the content could be trawled and copied by some automated bot.
Can the ASM determine that something illegitimate (which would generally be anything not googlebot, yahoo, msn etc) is crawling our site based on rate, maybe using url +1 type logic that could stop this activity occurring (even if a botnet is used or spoofed ip / proxy etc) without harming legitimate traffic.
Secondly as we want to use the web accelerator product so may have to go without the ASM for a while until version 10 is released is there functionality in the LTM that can prevent this some type of iRule or class profile.
Sorry maybe this info is out there already I had a quick search on the forum and a read through the ASM config guide but didn't come across anything.
Thanks