Saturday, July 12, 2008

Cookie issue with 'HttpWebRequest' object

Recently i have a requirement to programatically get the HTML mark up of a given url with C# coding.After a bit R&D work i found 'HttpWebRequest' object in 'System.Net' namespace will server my purpose.Following lines of code can get the HTML mark up of a given URL.


HttpWebRequest req = (HttpWebRequest)WebRequest.Create("url");
HttpWebResponse res = (HttpWebResponse)req.GetResponse();
Stream str = res.GetResponseStream();
StreamReader streader = new StreamReader(str);
string htmlContent = streader.ReadToEnd();

in the above code 'htmlContent' contains the HTML mark up of the 'url' .Ideally this code should work for the all the urls.But there may be cases where the requested source is moved to some other web server & the request is redirected to new location by attaching some cookie to incoming request , in such a scenarios the above code wont work, it'l give excetion the reason is by default cookies are disable for 'HttpWebRequest'

The solution for this problem is to enable cookies for 'HttpWebRequest' object.This can be done with following piece of code.

req.CookieContainer = new CookieContainer();

Note: setting of 'CookieContainer' property of 'HttpWebRequest' should be done before calling 'GetResponse()' method.

With this code cookies are enable for 'HttpWebRequest'.

No comments: