After reading this article and this article which give frustratingly over-simplifications on user-agent spoofing to get past cloaked websites, I figured I should write something on how to REALLY behave like Google. Cloaking often goes well beyond this, using IP delivery, User Agent cloaking, javascript and cookie detection, and referer detection - all of which can be used to determine that you are you and not a bot. So, how do you beat all 5 major types of cloaking?
1. Beat IP Delivery: Use Google Translate as a Proxy, translating from spanish->english even though the site is already in English. 2. Beat User-Agent Cloaking: Use the FirefoxUser-Agent Switcher to spoof as GoogleBot 3. Beat Javascript Detection: Use the Firefox Web Developer Toolbar to turn off javascript. 4. Beat Cookie Detection: Use the Firefox Web Developer Toolbar to turn off cookies. 5. Beat Referer Detection: Use the Firefox RefControl Extension to prevent referer from being sent.
Using these in conjunction can be extremely effective, even at pay-for-information sites. Doing this may be against the terms of service of the site you are visiting. There are plenty of popular sites out their that cloak content which is normally only available to paying members. While these techniques work on those sites too, be careful.
Good browsing!
