Windows .NET in C# Major – Web Interaction With Self Signed SSL Certs. Part 2

Last week, we started making the web interaction class and did the self-signed compatibility functions as well as some setup ones. You can see all that here. Today we get into the meat and potatoes and get started on the functions that will actually send and receive the data we care about. So here we go!

First, we need a few helper functions to setup headers and change our NameValueCollection of post or get items into a string of &item=value stuff.

private static void LoadHeaders()
{
    wcClient.Headers.Clear();
    wcClient.Headers.Add("User-Agent", CLIENTAPP);
    wcClient.Headers.Add("Cookie", SerializeCookies());
    wcClient.Headers.Add("Accept", "text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5");
    wcClient.Headers.Add("Accept-Language", "en-us,en;q=0.5");
    wcClient.Headers.Add("Accept-Encoding", "gzip,deflate");
    wcClient.Headers.Add("Accept-Charset", "ISO-8859-1,utf-8;q=0.7,*;q=0.7");
    wcClient.Headers.Add("Keep-Alive", "300");
    wcClient.Headers.Add("Referer", sReferer);
}
private static string FixNameValues(NameValueCollection nvc)
{
    string buffer = "";
    string[] keys = nvc.AllKeys;
    for (int i = 0; i < keys.Length; i++)
    {
        buffer += "&" + keys[i] + "=" + nvc[i];
    }
    return buffer.Substring(1);
}

You can put whatever you want in the LoadHeaders for any other headers you may want. The other function just puts the NameValueCollection into the HTTP format of &item=value for posting or getting. We also will use some functions to handle the cookies, here they are:

private static void ParseCookies(string cookies)
{
    string[] parts = cookies.Split(';');
    string buffer;
    string[] aBuffer;
    for (int i = 0; i < parts.Length; i++)
    {
        buffer = parts[i].Trim();
        aBuffer = buffer.Split(new char[] { '=' }, 2);
        if (aBuffer.Length > 1 && aBuffer[0] != "path")
        {
            if (sCookie.ContainsKey(aBuffer[0])) { sCookie[aBuffer[0]] = aBuffer[1]; }
            else { sCookie.Add(aBuffer[0], aBuffer[1]); }
        }
    }
}
private static string SerializeCookies()
{
    string buffer = "";
    IDictionaryEnumerator e = sCookie.GetEnumerator();
    while (e.MoveNext())
    {
        buffer += (string)e.Key + "=" + (string)e.Value + ";";
    }
    if (buffer.Length > 2) { buffer = buffer.Substring(0, buffer.Length - 2); } 
    return buffer;
}

Since we are keeping our cookie stored in a hashtable, these two functions will serve to parse into the hashtable and stringify the hashtable for sending back to the server. Our Get functions are overloaded, so we can choose if we want to send a query string or not.

public static string GetURLText(string url, NameValueCollection queryData)
{
    if (url.Contains("?")) url += "&" + FixNameValues(queryData);
    else url += "?" + FixNameValues(queryData);
    return GetURLText(url);
}
public static string GetURLText(string url)
{
    LoadHeaders();
    sReferer = url;
    try
    {
        byte[] byteReply = wcClient.DownloadData(url);
        if (wcClient.ResponseHeaders["Set-Cookie"] != null)
        {
            ParseCookies(wcClient.ResponseHeaders["Set-Cookie"]);
        }
        return Encoding.ASCII.GetString(byteReply);
    }
    catch (WebException ex)
    {
        return ex.Message;
    }
}

Now, you can do whatever you want with the caught WebException, it usually means the host timed out. These are pretty easy to follow, One just fixes the query string for the URL, the other takes the URL, tries to download the data from the host into a byte array, sets any cookies, and returns a string from the byte array. If it fails, then it passes back the exception message. Again, you should probably throw the exception and catch it in the next layer and do something real with it.

public static string PostURLText(string url, NameValueCollection postData)
{
   try
   {
        LoadHeaders();
        sReferer = url;
        wcClient.Headers.Add("Content-Type", "application/x-www-form-urlencoded");
        byte[] data = Encoding.ASCII.GetBytes(FixNameValues(postData));
        byte[] reply = wcClient.UploadData(url, "POST", data);
        return Encoding.ASCII.GetString(reply);
    }
    catch (WebException ex)
    {
        return ex.Message;
    }
}

Post is very similar to get except instead of concatenating the request data into the url, we send it to the server as POST data. We also add content-type to the headers since this is a urlencoded form.

So there you have it! And it's simple and easy to use:

string url = "http://www.myspace.com/";
string page = WebInteraction.GetURLText(url);

Hope this helps you guys out as much as it has me!