Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Googlebot is indexing all my bills as XLS #244

Open
martinszy opened this issue Jun 18, 2014 · 2 comments
Open

Googlebot is indexing all my bills as XLS #244

martinszy opened this issue Jun 18, 2014 · 2 comments

Comments

@martinszy
Copy link
Collaborator

I don't know if this is good or bad, but they are consuming the service:

Started GET "/proyectos/6103-D-2012.xls" for 66.249.64.173 at 2014-06-18 21:37:22 +0000
Processing by BillsController#show as XLS
Parameters: {"locale"=>"es-AR", "id"=>"6103-D-2012"}
Rendered bills/show.xls.eku (101.0ms)
Completed 200 OK in 3665ms (Views: 227.1ms | ActiveRecord: 0.0ms)
Started GET "/proyectos/2309-D-2012.xls" for 66.249.64.163 at 2014-06-18 21:39:34 +0000
Processing by BillsController#show as XLS
Parameters: {"locale"=>"es-AR", "id"=>"2309-D-2012"}
Rendered bills/show.xls.eku (31.1ms)
Completed 200 OK in 902ms (Views: 33.3ms | ActiveRecord: 0.0ms)
Started GET "/proyectos/2446-D-2012.xls" for 66.249.64.173 at 2014-06-18 21:43:41 +0000
Processing by BillsController#show as XLS
Parameters: {"locale"=>"es-AR", "id"=>"2446-D-2012"}
Rendered bills/show.xls.eku (30.1ms)
Completed 200 OK in 2336ms (Views: 32.5ms | ActiveRecord: 0.0ms)
Started GET "/proyectos/7476-D-2006.xls" for 66.249.64.173 at 2014-06-18 21:45:20 +0000
Processing by BillsController#show as XLS
Parameters: {"locale"=>"es-AR", "id"=>"7476-D-2006"}
Rendered bills/show.xls.eku (64.4ms)
Completed 200 OK in 1089ms (Views: 65.9ms | ActiveRecord: 0.0ms)

Should we write something in the robots.txt file?

@jdgarrido
Copy link

maybe you can use:
User-agent: Googlebot
Disallow: /*.xls$

@rezzo rezzo added this to the garibaldi con chochitos milestone Aug 26, 2014
@rezzo rezzo closed this as completed in 18b3f35 Aug 26, 2014
@martinszy
Copy link
Collaborator Author

La verdad es que el echo de que tenga un $ al final hace que sean indexables todas las urls con la forma;
proyectos/busquedas.xls?authors=Lino+Walter+Aguilar&creation_date_max=2014-09-17&creation_date_min=1990-03-01&page=10910

Al sacarle el $ las bloquea.
Quieren actualizar también su versión?
A mi me trae problemas de performance.

@martinszy martinszy reopened this Nov 4, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants