There are several blogs and web sites discussing issues with WordPress sites not getting indexed by Google. I was hoping that the reason my site was not getting indexed would be revealed in this thread: Major Problem solved with WordPress & Google. Unfortunately, it wasn't.
WordPress Privacy Settings
I searched a lot more and finally realized that there was a WordPress setting that handled this. I had set my site to be publicly available in an older version, but somehow, somewhere along the way, it got changed.
My Settings>Privacy settings were set to "I would like to block search engines, but allow normal visitors". Once I changed the Privacy settings to "I would like my blog to be visible to everyone, including search engines (like Google, Sphere, Technochrati) and archivers", and clicked the "Save Changes" button, my site tools indicated that Google had pretty much instantly started indexing the site. Then a simple Google search of my site revealed several posts and content. I wish I could identify when this setting was changed, because I clearly would have remembered changing it, which I did not.
Mission accomplished, at least! I hope this helps someone else out...
-Dan
Loading ...
Luke said:
OMG… I lost all traffic for months… Was searching and searching, playing with codes etc. I never looked at that configuration section!!! Thank you so much!
Dan said:
@ Luke:
You’re welcome. Glad to be of help. I just wish I knew when this setting got changed…
Luke said:
Hey Dan,
I changed to a wordpress template, so I guess it was pre-configured to that. So far (1 day after), my site is still not indexed. I guess Google take some time to do that. You know how long this might take?
Dan said:
What is your web site address? It could take days, weeks or months for Google to index. Have you validated your site to make sure there are no coding errors? Try using the W3C Markup Validation Service. It’s awesome!
There are a ton of good articles out there on search engine optimization. I would Google that phrase… Repeating keywords and using terms that don’t appear in the heading tags of your blog can actually hurt you. It also helps to have links from other sites that point to yours.
If you use WordPress (I assume so), try out the Google XML Sitemaps plugin. When you change your blog, it will resubmit the sitemap to Google for indexing. You can also submit manually with the plugin options.
Note Google only allows one submission per day.
-Dan
Luke said:
Hi Dan,
My website is http://www.panamafutbol.com
Let me try your suggestions and get back to you.
Many thanks for your help!
Luke
Luke said:
Google XML Sitemaps is installed. Hope it wont take too long to get re-indexed
Really too bad, this site was for years number 1 in Google… Still glad to have this figured out.
Thanks again!
Dan said:
Glad to be of help. I would strongly recommend fixing the XHTML Transitional validation issues (41 errors, 18 warnings) using the W3C website I mentioned above…
And just out of curiosity, what search term did you expect to be listed #1 in Google for, Panama Futbol? Your site currently has duplicate keywords and uses the same phrases in different word order, which could be flagged by Google as “keyword padding”. This may hurt your rankings for sure.
-Dan
Luke said:
Well anything related with futbol (soccer) in Panama. Panama futbol, futbol en panama, futbol panameño etc.
How serious are those issues by the way? I have an other site, which shows even more errors, but it has pretty good results in google.
Dan said:
There are varying theories about page validity; I think the bottom line really is web browser compatibility, accessibility, etc..
If a search engine is spidering your site and it encounters errors, it may abort any further spidering, depending on the severity of the issue. They don’t try to figure out what you might have meant to type…
…which is unlike Internet Explorer which I have seen make automatic code corrections during page rendering such as converting forward-slashes to back-slashes… LOL
Luke said:
Hehe, well… I guess I will try to fix some of those. I think most are minor mistakes (spaces etc).
Thanks again Dan. I will report back once I am indexed again in Google
Luke said:
My page is back in Google! Number 1 again after one week.
Thanks again Dan, couldn’t have done this without you.
Cheers,
Luke
Dan said:
Excellent! You’re welcome and I’m glad you made out well!
Ryan Yokome said:
thanks a TON!!
I’ve been trying to figure this out for 2 weeks. No indexing on google.
Thanks again so much appreciated
Ashley said:
Sure it is. I was stuck up with this once ago. I was consulting many experts but couldnt find a solution, Thank God, one blogger helped me to solve this and Hurrah now I am indexed!
Dan said:
@Ryan Yokome and Ashley:
You’re welcome. Thanks for providing feedback. It’s nice to know when the information you post proves to be useful to others!
pc gamers said:
This same exact thing happened to my blog. I run 6 wordpress blogs and 3 vbulletin forums on a dedicated server. After setting up my most recent blog, google would not index it. Even after submitting a sitemap, and waiting 2 weeks, still no indexing on the articles.
2 days ago I changed that setting, and google is still not indexing the site. I had put my own .htaaccess file and my own robots.txt into the html root. I wonder if wordpress tried to change something on those files, but couldn’t.
I’am going to give it 24 – 48 more hours. If the search engines do not start indexing, I’am going to delete the site and start over.
Dan said:
@ pc gamers:
Is there any chance that the previous domain owner abused it? I did find mention of the sale of your domain name on AdminAddict.net…
pc gamers said:
@ Dan
I bought the domain name a couple of weeks ago. The name was transferred to my registor with no problems. The DNS servers were changed to point towards the new IP Address and server. I then setup a wordpress blog, verified the site through my google webmaster tools, installed the google analytics code, submitted a sitemap, the sitemap is being downloaded by google with no errors. Google Webmaster Tools shows that the sitemap contains 40 URLs, but only the homepage url is being indexed.
A link to the site was added to my signature on a gaming forum – this should give about 300 – 500 backlinks, submitted to digg, stumbleupon, posted links in various webmaster forums asking for reviews and this morning, the site and its feed was added to my Yahoo Blog Log.
All of the old URLs are from the previous owner had a forum on the site.
I have never had this many problems getting a site indexed.
Dan said:
There are a lot of validation errors on your site. While this isn’t necessarily a show-stopper, there is one that I found particularly interesting: an RSS feed from elgms.com. In your page source code, it is coded as
*.elgms.com/forum/external.php?&type=js
which seems interesting to me within a script tag and without a language definition. Just an observation. I’m wondering if Google may somehow see such things as misleading…
Masdin said:
Hello Dan,
Thank you very much for writing this. I was frustrated due to a not indexed wordpress blog. Now it is indexed. Many thanks
Dan said:
@Masdin:
You are welcome. It’s always nice to know when others have found my posts useful.
-Dan
thecitywithin said:
Hi,
I am having similar issues with google not indexing my site, http://www.thecitywithin.com
However, I believe that my issue is more complicated than just updating a setting in WP, which I double checked and confirmed it’s set to publicly available.
The full description of the problem is here:
http://www.webmaster-talk.com/the-google-forum/196564-google-does-not-index-my-page-2.html#post984334
I appreciate any thoughts, Thanks
General WordPress Tips and Info | Cartier Consulting, LLC said:
[...] WordPress Blog Not Indexed by Google [...]
Dan said:
I briefly viewed the discussion you referenced at webmaster-talk.com . Are you still having trouble, or did modifying your robots.txt file or disabling a plugin help? At first glance, your site looks OK to me, low keyword count and Valid HTML, however the DOCTYPE syntax is off: http://validator.w3.org/check?uri=http://www.thecitywithin.com/ .
One good plugin that I always try to set up is Google XML Sitemaps. This will submit your site, once a day max. to Google when content is changed. I see you are using Michael Torbert’s All in One SEO Pack, also an excellent choice.
I’m not sure how much traffic your site gets, but the Google XML Sitemaps plugin, along with setting up this domain in Google Webmaster Tools, can help immensely. Also, the more sites that link back to yours, the better your ranking can be.
-Dan
thecitywithin said:
so just a quick update on my site status; I can finally see google indexing my site, not sure how (maybe it just took some time that it needs) but here’s what I did after my initial posting here (whether it helped or not you be the judge):
- I continue to create postings on my WP blog (I think google may have initially thought that I am abusing the system as I have a number of links to other sites but they are necessary as I am providing references and information)
- I created another webmaster tool for the site using my other gmail account
- on Yahoo Answers, I started answering people’s questions related to Prague, which is what my blog is about. I included the link in a few of them
- I once again patiently wait for updates…
so now I can see the site is slowly but maturely being indexed in google search. Hope this helps to people who will have similar problems in the future,
Phil said:
Thanks. I have been trying all kinds of things but I had this privacy setting on too. It must be a default because I have never changed it.
Dan said:
Thanks Phil. Your feedback is appreciated.
-D
nsikun said:
@ashely you said one blogger helped you do you mind sharing the news?
nsikun said:
hi dan, I have similar issues (site not indexed in Google) – http://www.zedcloud.com; however it’s been picked up on bing. I have followed a few tips you’ve generously given here, however in the meantime I will appreciate any further tips.
So far I have done the following:
1. reduced my errors from 20 to 5 using | http://validator.w3.org
2. installed the XML-sitemap (both zipped and unzipped).
3. i’ve deliberately not set up the robots.txt to avoid blocking the google-bot.
4. Installed WP plugin ‘Follow URL’ to get rid of ‘rel=no follow’ on my links.
5. registered with Google web tools (blog still not appearing).
6. Yet, another new domain I set up with no traffic is already being indexed in Google (confused).
7. Thanks!
Dan said:
Hi Nsikun, How long have you had this domain name? Is it possible that someone before you abused it? I did check out the site and it looks good. The site errors look like the & character just needs to be replaced with
&in a few places, however I doubt that’s why you’re not getting indexed.I would suggest putting a robots.txt file in the root of the site and set the contents to:
User-agent: *Allow: /
Alternately, you could install a plugin like PC robots.txt that will create a customizable robots.txt file for you using convenient plugin settings. Note that you don’t want a real robots.txt file in your site if you use an option like this.
Other than that, I don’t have much else right now.
nsikub said:
thanks dan for the site comment!(not your fault misspelt my own name – nsikub).
Actually I suspect the domain name might have been abused, but is there anything one could do if it was abused. Do things work out in a such a way that over time perhaps the site could just get indexed by itself?
Will replace the & and use the suggested Plugin.
Thanks for your time.
nsikub said:
Hi dan actually found this link http://www.dcloud.nl saying zedcloud (£99) will you please advice?
nsikub said:
sorry dan sent wrong link (actually if you just type ‘zedcloud.com £99′ in Google you’ll see what I mean. Thanks.
Dan said:
@ Nsikub:
I had some difficulty finding the link you are mentioning. Seeing your domain for sale on web hosting, page parking, domain auction, or domain regristrar sites is not uncommon. I have seen this with my other site, CartierConsulting.com many times.
I would try to enter the purchase process for your domain name for whatever site is offering it to see if it really exists. You may find (hopefully) that it is indeed not for sale, and instead be offererd alternates or taken to a WhoIs page to search.
What I have done in my own case is contact the top 5 or so search results offering my domain for sale, inform them that it is not for sale, and ask them to remove my domain from their listings. I think the longest took 3 months to drop off. Regardless, it should drop off eventually.
How long have you had this domain name registered to you?
Google Webmaster Tools is probably your best reference right now, especially if you are indexed in other search engines. You can re-submit your site map through them, and even see what is reported for the contents of your robots.txt, either a real one or a virtual one.
-D
Dan said:
OK, I did see in your DNS registration record that the domain was registered on 19Dec2009.
Still kind of new, really, but I would also try going to the Add your URL to Google web site to submit it manually for indexing. You might get lucky and receive a reply from them with a reason for not indexing. I would be sure to tell them in the comments field that you haven’t had it for very long and suspect abuse by the previous owner (spam, malware hosting, etc.).
Hope this helps.
-D
nsikub said:
@dan thanks.
I tried the purchase process of my domain name and I terminated when I was prompted to go ahead and buy it. So I have followed your advice and notified the sellers about the situation thanks!
I also confirmed with my domain name provider that the domain actually has a ‘whois’ previous history since 2008-03-22. I tried to search for any potential abuse or backlinks from http://www.archive.org/web/web.php but nothing shows up.
Regarding the DNS registration you’re right I know my domain is a fairly new and I am trying my best using Google Webmaster tools to improve things. I have re-submitted the URL to Google and put a comment as suggested. Also re-submitted site maps etc.
Right now, I have just got rid of the WP plugin PC Robots.txt and manually put the robots.txt in my root folder allowing no other bots other than googlebot in the hope that crawling might start.
Again, thanks for the tips couldn’t have reached this far without you.
Dan said:
@ nsikub:
Sounds like you’re on the right track as much as you can be. Hopefully your site gets indexed soon in Google.
I trust that you do know that disallowing other bots may hurt your listings with them. I would just allow all for now.
Where is Zambia located?
-D
nsikub said:
thanks @dan.
Zambia is a beautiful country in Southern Africa.
Manou said:
I am using Kubrick theme. I can’t install any plugins. And my site doesn’t get indexed in Google search. Any solutions people?
Dan said:
@Manou:
Sounds like the permissions on your web host’s server may be set incorrectly. You might try setting them to 755 temporarily (chmod 755) to see if this is the case.
An assumption is being made here that you are not running on WordPress.com, where functionality is quite limited.
What is the URL of the blog in question?…
Patrick said:
Hi Dan,
I have recently started a WordPress blog at the site/directory specified. Initially google indexed two of my posts and then they dropped off again.. any idea what it could be?
At the moment none of my posts are indexed.
A few possible clues:
- I use the robots meta plugin to reduce duplicate content
- I have pretty similar text in my intro to the 4 articles posted… could this be it?
- I have stated my preference for pgcoaching.nl rather than http://www.pgcoaching.nl and use a 301 redirect plugin
Thanks in advance for any light you can shed…
Regards, Patrick
Dan said:
@ Patrick:
I may be misinterpreting your comments, but it looks like your blog is actually in a sub-directory of the URL you provided, namely at
http://pgcoaching.nl/tac/
You might try changing the pointer to your blog in your web host’s admin panel so inquiries point to this URL instead of the parent one.
If your main site is indexed fairly well already, it may just take some time for the blog portion to get indexed.
Hope this helps,
Dan
Patrick said:
Thanks Dan,
Yes – the blog is in a sub-directory of the main url.
Can you tell me how I could change the pointer from the main URL to send inquiries to my blog instead?
Regards, Patrick
Dan said:
@ Patrick:
It all depends on if you want your blog to appear as a separate web site or not. For any given domain, the hosting provider maintains the DNS records that send inquiries to your site. You can usually specify the destination of a domain (folder in your hosting space) where domain requests are sent.
If your blog is to remain in a sub-folder under your domain, you may have to wait.
If you can make the blog live at a domain name of its own, that may be better.
Either way, you should submit the blog URL to search engines, keep it active by posting, and try to post links to the actual blog URL in your web travels…
Hope this helps,
Dan