Welcome to WebmasterWorld Guest from

Forum Moderators: goodroi

Message Too Old, No Replies

Google SEO robots.txt Install Issue

9:55 am on Feb 21, 2011 (gmt 0)

I am new with 10 blogs that I am tring to optimize with Google in particular being the problem and wordpress.com being more manageable as far as search engine inclusion. I read a thread concerning finding the root directory to add a script so this might be similar except I need to change my google robots.txt file. I have the setting set for "yes" to allow searches but The robot.txt is still blocking searches: Disallow:/ searches, so I made a new: Allow: to give access to all search engnes but I can not get to the root directory to either change the old which seems to be"write protected" or replace it with the new "allow" that I made. I installed Filezilla Client to do the file transfer but can not make it work. I keep getting an error message and it closes prematurely.Is there an easy way for a novice webmaster to change the robots.txt file?
Thank you
3:04 pm on Feb 21, 2011 (gmt 0)

WebmasterWorld Administrator goodroi is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

Are your 10 blogs being hosted on wordpress.com or do you have your own hosting?

Robots.txt is a simple text file. It is also a very important text file. One misplaced character can have a big impact of your site. I do not normally suggest that novice webmasters make many changes to robots.txt.
8:32 pm on Feb 21, 2011 (gmt 0)

Thank you for responding,goodroi. 6 are wordpress.com which I am doing quite well with. in fact Saturday a post I just wrote was selected as the featured post which I discovered when I did a web search. It is the four or actually three out of the four blogger blogs that are being excluded from searches by robots.txt that basicly state :user agent* disallow:/searches. My wordpress show up well in searches as far as I can tell. There might be one or two that are not indexed well but that is one of the reasons that I opened so many blogs/ to get my message "Out". I am disabled and living on $600./month so everything I DO, IS "ON A Budget".

My issue has been this: everything that I have read says to upload to the "root folder or domain root folder" but have not been explained in a way That I can understand, how to upload it! I tried Filezilla Client but was unable to get a stable connection on my dial-up ISP, or so it seems.It stated that there was either a router or firewall problem but the firewall is enabled and I even turned it off to be sure. I used Google to produce multiple robots.txt files: user agent* Allow:/ which it said would allow all spiders or bots to access my blogs but again, how exactly do I upload them to the root directory. I found the robots.text file online but it is apparently write protected which kept me from altering it so I need to replace or over write them. Sorry for the long explanation but I wanted to cover all of the bases.
9:35 pm on Feb 21, 2011 (gmt 0)

WebmasterWorld Administrator goodroi is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

This sounds like you are unable to upload your new robots.txt file because your ftp program can not access your site. If that is the case you might want to contact your ISP help desk or try a different ftp program or try to reinstall filezilla.
10:50 pm on Feb 21, 2011 (gmt 0)

Thank you, I have uninstalled filezilla twice already. I understand that my web address changes so would that be a static type connection. Even so I will try to contact them. I have had difficulties with their customer service in the past and they always wanted to blame everything on me or my system. I will give it a try any way. Do you know of a different ftp client that you would recommend?
10:36 am on Feb 22, 2011 (gmt 0)

WebmasterWorld Administrator goodroi is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

I am surprised you are having trouble with filezilla. If you search on google for ftp program you will quickly find many options.
5:32 pm on Feb 22, 2011 (gmt 0)

Tank you for all of your assistance. I provided a log file to Filezilla because the wiki did not cover my issue. I also contacted my ISP and did what they recommended but it made no difference. I am probably missing something simple but the log file should reveal what the problem is. I am surprised also! I asked if you might have a particular preference apart from Filezilla knowing that there would be a number of programs to choose from. I will just try my luck with a different ftp. Thank you again
3:34 pm on Feb 23, 2011 (gmt 0)

A quick update: A tech from Germany, The Filezilla-Project, examined the log file that he asked for and did some research on Blogger-Blogspot and said that it appeared that their server does not, or for some reason is not allowing ftp or sftp file uploads. It may not be my mistakes after all. He asked me to do further research to see if Google is actually blocking these uoloads.
1:21 am on Feb 24, 2011 (gmt 0)

This should be the answer to this very frustrating issue, at least for me! I found that the problem is with Google after all. They no longer allow ftp folder access or the ability to change the robots.txt file so the FileZilla.org tech in Germany was correct in his estimatation of this situation. Just thought you might want to know how this turned out and again I thank you for your assistance.

Featured Threads

Hot Threads This Week

Hot Threads This Month