Skip to content
Home » Blog » First Steps: Users and Search Engine Crawlers

First Steps: Users and Search Engine Crawlers

I made a rough step by step guide for SEO last month and posted Winning SEO Strategy.

I think those strategies/methods could be applied in a general way. It doesn’t matter what kind of project you run, it will work for almost all. Of course, higher competition, higher volume might be needed.

Google HCU 2023

With the recent Google update (September HCU) more and more people start questioning the way they are doing sites, after they got wiped out with a blink of an eye. Why should you put in any work and energy into your website and content, when it’s just for selling and you will get wiped out at some point anyway?

I think, many start to understand, that they didn’t build a legacy with their sites, that there will be never anything passive with their income, and that they put in too much work into something nobody really ever wanted to see.


I had similar experiences in the mid to end 2000s, which led me more and more to this technical approach of SEO.


Websites for Users / Websites for Search Engines

You have two variables in the SEO equation. One is your website, which should be optimized for users so it converts well, the other is traffic.

Those two often don’t go well together, because what a user likes to see often doesn’t match with what a search engines wants to see, and vice versa, which leads to cringe experiences.

The more you are able to mentally separate those two, the more free you are in your SEO related actions.

Cloaking

I think you might have already guessed it, we are talking about cloaking. The battlefield where you have absolute freedom over your websites and don’t have to care about the user experience because you redirect them to your properly done landing page anyway.

You don’t have to redirect users and make cloaking obvious, you could also cloak parts of the site, like a CTA overlay.

You are directly ‘talking’ to search engine bots with this approach. There’s no marketing crossing your way, preventing you from doing certain things. It’s just you and the search engine crawlers – as technical as it could be.


There are different ways to cloak. The easiest method is by user-agent, which could be even done on a client level through Javascript. ‘Professionals’ cloak by IP.

Use cases for Cloaking

There are plenty of uses for cloaking and it doesn’t have to be Blackhat SEO. You could show some links on your sites only to Google or you cloak Ahrefs and other crawlers to hide your links from them. Show certain things based on geo location or browser. Facebook Ads clocking it’s the same. You show one offer to users and another (legit) to the Facebook team. The circle closes and we are Blackhat again 🤡

In our western world Google is the dominant search engine, but in other areas of the world there could be other search engines more used or maybe even several. It might make sense to ‘optimize’ your website for each search engine separately. Not every search engines works like Google.

We’ll get into more detail on how to implement this properly in the near future.

For now you are good to go with cloaking by user-agent.



Take action / Your Homework

If you want a little bit of homework: Write a simple cloaker in your preferred language (PHP should work fine). Do a simple page for testing, output one thing for Google and another for the user. Index that page. Use different titles for example, so you can easily see that what got indexed and what is shown in your browser.

Good luck.

More in the next days.

Leave a Reply

Your email address will not be published. Required fields are marked *