C# web page authentication

I came across need the other day where I had to scrape a page. The issue was that if you are not logged in then you get different content to if you were logged in. It was a relatively easy solution but I came across some code that really stood out at stackoverflow.

The bulk of it a nice simple class to encapsulate the cookies and authentication credentials in a class as follows:

public class CookieAwareWebClient : WebClient
public CookieAwareWebClient()
CookieContainer = new CookieContainer();
public CookieContainer CookieContainer { get; private set; }

protected override WebRequest GetWebRequest(Uri address)
var request = (HttpWebRequest)base.GetWebRequest(address);
request.CookieContainer = CookieContainer;
return request;

Thus once you provide login details you can then easily call on the WebClient method as illustrated in sample code (also from the stackoverflow site):

using (var client = new CookieAwareWebClient())
var values = new NameValueCollection
{ "username", "john" },
{ "password", "secret" },
client.UploadValues("http://domain.loc/logon.aspx", values);

// If the previous call succeeded we now have a valid authentication cookie
// so we could download the protected page
string result = client.DownloadString(“http://domain.loc/testpage.aspx”);

Again this is not my code – I just liked it’s elegance in solving a simple problem.

Print Friendly, PDF & Email