How to see index meta tags if a site's code is obfuscated?

Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

How to see index meta tags if a site's code is obfuscated?

ichthyous

4:13 pm on Jan 23, 2022 (gmt 0)

I paid for an "partnership" promo article on a high DA site and it has not been indexed for the last ten days. The other articles published in the same feed both before and after are all indexed. If I search with the article title or text nothing appears in Google's index. The site code is obfuscated somehow...maybe served via JS, I'm not sure. There is no header or meta information when you use view source. Are there any tools you can use to see the meta tags and confirm whether it's set to noindex, nofollow if it is obfuscated in this way?

NickMNS

5:13 pm on Jan 23, 2022 (gmt 0)

Open DevTools in the browser. Go to the "Elements" tab scroll to the top and expand the <head> tag. You should see all the <meta> tags as children to the head tag. If what you are looking for is not there, then it never was.

ichthyous

6:50 pm on Jan 23, 2022 (gmt 0)

Thanks @Nick, I had already done that and found no meta referring to the indexing of the site. There are other meta tags like "title" and "Canonical". The other articles that are indexed have the same meta data it looks like. So why would Google skip one article and index the ones published before and after? There is no trace of it, even if you search with blocks of unique text...so to me that means somewhere along the line they have intentionally blocked it

not2easy

7:47 pm on Jan 23, 2022 (gmt 0)

I have found that Google no longer searches for "unique text strings" for a least a month now. I have had this problem in the past and then it will be OK again, but it hasn't worked for me for anything. I wouldn't use that as a test for whether it is indexed or not.

ichthyous

12:40 am on Jan 24, 2022 (gmt 0)

@not2easy If you search for text for articles published before and after they all come up in Google.

NickMNS

12:52 am on Jan 24, 2022 (gmt 0)

I assume you suspect that the site you paid, put a no-index directive on the page, but you can't find where it is and that is why you are asking the question.

The other way to no-index a page is by using the "X-Robots-Tag" header.

The way to see the headers is to go back to devTools, but select the "Network" tab, then refresh the page. Then select the item at the top of the page, click it and all the headers should be listed on the right. Look a for "Response Headers" and look for:

X-Robots-Tag: noindex

ichthyous

5:54 pm on Jan 24, 2022 (gmt 0)

Thanks @NickMNS, I checked this as well...no sign of any X-Robots tag in the response header for any of the articles. Yet ALL of them are indexed except for the one I paid for. The way the page loads is very odd. It's very slow, and they have paid ads at top, then a video popup which slows down the load time. When you refresh and watch the network loading it's very slow compared to other sites. I see a lot of "Blob" entries, not sure what that is.

jediviper

3:30 pm on Feb 4, 2022 (gmt 0)

In the future, if you want to check the source of any page (even if it's served totally in JS) just use the "view rendered source" Chrome extension.

NickMNS

5:21 pm on Feb 4, 2022 (gmt 0)

just use the "view rendered source" Chrome extension.

Yes, that works.
Or you can right click and select "inspect element", and all the rendered code is displayed (an updated as you use the page), no browser extension needed. Also works, on Edge, Opera, Brave, Firefox, Safari, and IE.

There are also bonus features:
Console tab: shows any messages logged to the console, including warnings and errors.
Source tab: Let's you see the source code of the Javascript programs running on the page.
Network tab: See every network request made by the page, including tracking request, websockets, and all the request and response headers.
and more tabs that I wont get into.

It's all right there in your plain vanilla browser. It is very powerful stuff, not just for "spying" on other websites but also for making sure your website does what you expect it to.

Sorry for the rant!

Sgt_Kickaxe

5:49 am on Feb 5, 2022 (gmt 0)

I agree with NickMNS, "Inspect Element" is your best bet..

Additional, block all 3rd party scripts and frames before checking. Your link shouldn't be dependent on any added code.

... now stop buying links :)