Forum Moderators: goodroi

Message Too Old, No Replies

Mistake in robots.txt during a site merge, caused a big drop

         

Christian Evensen

7:30 pm on Sep 24, 2019 (gmt 0)

5+ Year Member



Hi guys

We recently merged an ecommerce site into another, and all the configs and redirect instructions on the webserver level were set up correctly. 302 first to test, and then 301.

Since we used a tool to trigger a message to the users redirected and landing on the new page, we had to include UTM-parameters. This also to analyze the redirected traffic coming in.

Example
https://www.oldpage.com --> (301) https://www.newpage.com/?utm_source=old-page&utm_medium=referral


Now what we forgot, is that our robots.txt on both sites had the following instruction
Disallow: /*?*


And suddenly we were blocked in Search Console. We now have added Allow-instructions, so everything seems to be OK.

Million-dollar question is how much do you think we will suffer from this mistake? We merged September 2nd and discovered the error Sept 23rd. So that's 3 weeks of "off the grid" for a great e-commerce domain ranking high on a huge amount of keywords in our marked.

Thanks a lot in advance.

Best,
Christian

not2easy

8:18 pm on Sep 24, 2019 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



Hi Christian and Welcome to WebmasterWorld [webmasterworld.com]

Things like this happen more often than you might think. Details. I would expect full recovery shortly and suggest it may help if you use the URL Inspection tool in your GSC account to let Google see that the problem was transient and has been repaired. You don't need to "inspect" every URL to get it moving.

Christian Evensen

8:50 pm on Sep 24, 2019 (gmt 0)

5+ Year Member



Thanks for a quick reply, not2easy:)

tangor

11:10 pm on Sep 24, 2019 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



@Christian Evensen ... Welcome to Webmasterworld!

With robots.txt as a voluntary thing it is unlikely an error there will have long term effects. Wait three weeks and see what happens.