Disallow: /var/resizes/

Astor

Joined: 2005-12-15
Posts: 154
Posted: Fri, 2014-01-24 09:31

i have this robots.txt :

User-agent: *
Disallow: /application/
Disallow: /installer/
Disallow: /lib/
Disallow: /modules/
Disallow: /system/
Disallow: /themes/
Disallow: /var/logs/
Disallow: /var/modules/
Disallow: /var/resizes/
Disallow: /var/thumbs/
Disallow: /var/tmp/
Disallow: /var/uploads/
Disallow: /digibug/print_photo/

but all images in /var/resizes/ are indexed in search engine and directory visible and navigable

can I prevent this ?
(excuse poor english)
 
MarkRH

Joined: 2007-05-25
Posts: 241
Posted: Fri, 2014-01-24 11:04

Those images were probably indexed before you changed the robots.txt file. Basically just have to wait for the image search engine to check that image again for it to know it should not index it. You might check out Google's Webmaster Tools: https://www.google.com/webmasters/tools/ and look at the Crawl -> Blocked Urls section to make sure it shows the most recent robots.txt from your site. It also has a Remove URLs section within the Google Index menu.

Of course to do all this with Google you have to have an account with them if you don't and may have to go through some steps to let it know you have control over the site by adding some Google verification code/file where the gallery is located.

I haven't messed with Bing or any of the others to know what you can do with those.

- Mark H.

Using Gallery 3.0.9 - gallery.markheadrick.com
 
Astor

Joined: 2005-12-15
Posts: 154
Posted: Fri, 2014-01-24 14:04

tnx MarkRH
i have checked Crawl -> Blocked in Webmaster Tools and the robots.txt of first post is not recognized.

i have one robots.txt in the main directory of my site and a second robots.txt (first post) in the directory of my gallery (in my case /photo/ ).

I have to move the controls in the main robots?

 
MarkRH

Joined: 2007-05-25
Posts: 241
Posted: Sat, 2014-01-25 01:13

One thing that might help is a link to your gallery so I have a better idea of how the directory structure is. In any case, search engines will only look at (whatever dot com)/robots.txt and not any located in sub-directories.

If your gallery is being viewed as (whatever dot com)/photo/ then the robots.txt file in your website's root directory needs to be:

User-agent: *
(whatever you have now)
Disallow: /photo/application/
Disallow: /photo/installer/
Disallow: /photo/lib/
Disallow: /photo/modules/
Disallow: /photo/system/
Disallow: /photo/themes/
Disallow: /photo/var/logs/
Disallow: /photo/var/modules/
Disallow: /photo/var/resizes/
Disallow: /photo/var/thumbs/
Disallow: /photo/var/tmp/
Disallow: /photo/var/uploads/
Disallow: /photo/digibug/print_photo/

Hope that helps,

- Mark H.

Using Gallery 3.0.9 - gallery.markheadrick.com