Q: Getting 403 forbidden when requesting images

I'm developing a web app using NodeJS.

Requesting an image using my browser, I get the appropriate response. Requesting the same image in my code (back-end server) I am getting this following response:

STATUS: 403 forbidden

Body:

<h1>Wordpress RSS Reader, Anonymous Bot or Scraper Blocked</h1>

<p>

Sorry we do not allow WordPress plugins to scrape our site. They tend to be used maliciously to steal our content. We do not allow scraping of any kind.

You can load our RSS feeds using any other reader but you may not download our content.

<a href='/feeds'>Click here more information on our feeds</a>

</p>

<p>

Or you're running a bot that does not provide a unique user agent.

Please provide a UNIQUE user agent that describes you. Do not use a default user agent like "PHP", "Java", "Ruby", "wget", "curl" etc.

You MUST provide a UNIQUE user agent. ...and for God's sake don't impersonate another bot like Google Bot that will for sure

get you permanently banned.

</p>

<p>

Or.... maybe you're running an LG Podcast player written by a 10 year old. Either way, Please stop doing that.

</p>

What am I doing wrong? As I understand it is not a problem with my API key, so what is it?

Thanks.

It says exactly what you need to do.

You need to also provide a unique User-Agent HTTP header (e.g. "My Cool App/1.0")

I've had the same problem. I have been developing/debugging this app off and on for about a year. This afternoon, I get the forbidden error through the App, but not when pasting into a browser. I am using a unique UA, but it is just a cartoon character's name.

The obvious solution might be to spoof the browser UA, but I don't want to be in a cat and mouse game, I just want to use the API correctly. If I'm doing something that violates the terms, I'd like to understand it, because everything I'm doing is core to the app I'm working on; and if these core functions violate the terms, I'd like to stop using the GB API altogether and focus on another DB. I'd prefer to keep the GB API, though, because of the distinction between releases.

Any help I can get is appreciated. Any further info I can give, let me know.

Thanks.

Maybe i've the same problem as you, does it say something when you browse at http://www.giantbomb.com/api right after getting the error ?

Like others have said you need to provide a unique User-Agent in the HTTP header.

Since you're using NodeJS, I'm assuming you're using a library like require.

Fixing this is as easy as changing code from this:

require(url, callback);

To this:

var options = { url: url, headers: { 'User-Agent': 'Unique user agent goes here.' } };
require(options, callback);

This is a pretty easy fix. Just look up how to change the HTTP Header in whatever library or node-module you're using.

I'm using .Net C# WPF and yes it seems it was a User-Agent problem. I don't understand why it worked some days ago and not now.

Here is a way to fix it (with the httpClient class) :

HttpRequestHeaders headers = client.DefaultRequestHeaders;

headers.UserAgent.ParseAdd("your_user_agent");

Using ref you need :

using System.Net.Http;

using System.Net.Http.Headers;

@ishiru: When I browse after getting the error, it works fine.

@ishiru: Still doesn't work.

Could be a user-agent issue--but below are the headers, which couldn't be less generic. And it was working fine until it wasn't. Doesn't look as if banned--I can paste the url the app is generating into any browser with my API key and it returns no trouble.

webclient.Headers["ACCEPT"] = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";

webclient.Headers["ACCEPT_ENCODING"] = "gzip, deflate, br";

webclient.Headers["ACCEPT_LANGUAGE"] = "en-US,en;q=0.5";

webclient.Headers["User-Agent"] = APPVERSION + "(" + Environment.OSVersion.ToString() + ")";

I'm pretty it' s a user-agent problem. I had the same kind of problem, changed it and now it works.

@ishiru: I appreciate you help. It may not be clear from my posts, but I had a unique user agent when it was working, and I did not change the user agent when it stopped working. I have changed it dozens of times since then, though, and that doesn't fix the problem.

I fixed it. I'm not certain how, but it is not outside the realm of possibility that I was using a different webclient than the one where I was setting the header.