The WebResponse class is similar to the WebRequest class. This class is used to receive data from the Internet resource.
These 2 classes are located into the System.Net namespace. Using these 2 classes, we have everything we need to download a complete Web page into a stream. But what I really want to show you is how to retrieve data (stock quotes and currency exchange rate) from the Yahoo web sites.
Note: For those of you who read the February 2004 edition of the UTMag, Tom Bellmer wrote an article that uses XMLHTTP with Visual FoxPro. The approach is different but the results are much the same.
The WebRequest class
Figure 1: The demo application in action
This class is used to establish a communication with a URI. It initiates a communication. We use the Create method that receives a URL as parameter to initialize a WebRequest object. This new object can then be used with the GetResponse method to synchronously download the content and store it into a WebResponse object instance. Asynchronous download is also possible if you use BeginGetResponse and EndGetResponse methods but I won’t use them here.
The WebResponse class
Once that the WebResponse instance has been filled using the GetResponse method (see previous paragraph), you can get its content and fill a stream with it. The easiest way is to fill a StreamReader object using the GetResponseStream method.
Our first method
This first method will be used by most examples of this article to read character contents.
Private Function RequestWebData(ByVal pstrURL As String) As String Dim objWReq As WebRequest Dim objWResp As WebResponse Dim strBuffer As String 'Contact the website objWReq = HttpWebRequest.Create(pstrURL) objWResp = objWReq.GetResponse() 'Read the answer from the Web site and store it into a stream Dim objSR As StreamReader objSR = New StreamReader(objWResp.GetResponseStream) strBuffer = objSR.ReadToEnd objSR.Close() objWResp.Close() Return strBuffer End Function
As you can see, this method uses the WebRequest and the WebResponse as described above to read the response of a URL passed as argument and returns the response into a string (named strBuffer).
Reading a complete Web Page
Now, using our RequestWebData method, we can read a whole page from a URL and display it within a message box simply by doing:
MessageBox.Show(RequestWebData("http://www.utmag.com"))
MessageBox.Show(RequestWebData("http://www.YourServer.com/YourFile.txt"))
Retrieving data from Yahoo finance web site
Some web sites are built to also return precise data (using a somewhat different URL) instead of a long page that you would need to parse. Don’t be fooled, we are not talking of WebServices here.
For example, open Internet Explorer and type http://quote.yahoo.com/q?s=msft to see the latest Microsoft quote. Notice the last trade price. Now type http://quote.yahoo.com/d/quotes.csv?s=msft&d=t&f=sl1d1t1c1ohgvj1pp2wern, and you will find much the same information but in a coma-separated string. This comes handy when you want to retrieve data.
As you know, the page layout of the first URL you typed may change from time to time which means that if you built something to parse the page source to find the data in it, your application will probably fail. Since the second URL returns only the data, it is not related to the page layout and your application will continue to run without any problem.
We can use our RequestWebData method to retrieve this data as well as the page source or the text file of the preceding example. It would then be easy to find the second element and return it as the price of the stock we are searching. But what if we passed 2 or more symbols in the URL? I built a simple wrapper around this to transform the comma-separated value string received into a XML string that is much easier to work it. Here is this method:
Public Function GetQuoteLatest(ByVal pstrSymbol As String) As String Dim strURL As String Dim strBuffer As String 'Creates the request URL for Yahoo strURL = "http://quote.yahoo.com/d/quotes.csv?" & _ "s=" & pstrSymbol & _ "&d=t" & _ "&f=sl1d1t1c1ohgvj1pp2wern" strBuffer = RequestWebData(strURL) 'Loop through the lines returned and transform it to a XML string Dim strReturn As New System.Text.StringBuilder strReturn.Append("" & Environment.NewLine) For Each strLine As String In strBuffer.Split(ControlChars.Lf) If strLine.Length > 0 Then strReturn.Append(TransformLatestLine(strLine) & Environment.NewLine) End If Next
strReturn.Append("" & Environment.NewLine) Return strReturn.ToString End Function
Private Function TransformLatestLine(ByVal pstrLine As String) As String Dim arrLine() As String Dim strXML As New System.Text.StringBuilder arrLine = pstrLine.Split(","c) strXML.Append("") strXML.Append("" & arrLine(0).Replace(Chr(34), "") & "") strXML.Append("" & arrLine(1) & "") strXML.Append("" & arrLine(2).Replace(Chr(34), "") & "") strXML.Append("" & arrLine(3).Replace(Chr(34), "") & "") strXML.Append("" & arrLine(4) & "") strXML.Append("" & arrLine(5) & "") strXML.Append("" & arrLine(6) & "") strXML.Append("" & arrLine(7) & "") strXML.Append("" & arrLine(8) & "") strXML.Append("" & arrLine(9) & "") strXML.Append("" & arrLine(10) & "") strXML.Append("" & arrLine(11).Replace(Chr(34), "") & "") strXML.Append("" & arrLine(12).Replace(Chr(34), "") & "") strXML.Append("" & arrLine(13) & "") strXML.Append("" & arrLine(14) & "") strXML.Append("" & arrLine(15).Replace(Chr(34), "") & "") strXML.Append("") Return strXML.ToString End Function
Figure 2: Download data
If you know the URL to pass, you can retrieve any values from Yahoo Web site. The thing is that I didn’t find any clear documentation on that. If you find some, please let me know! The easiest way to find the URL is to copy the URL from the “Download Data” button (see the bottom-right of figure 2) and removing the “&e=.csv” parameter.
Using this very same technique the downloadable demo also includes the retrieval of historical price and also the currency exchange rate.
Retrieving images
When using the WebRequest and the WebResponse classes, we are not limited to text data. Images are also available using this technique.
This is the main method that connects to a URL, retrieves the image and save it:
Private Function RequestWebImage(ByVal pstrURL As String) As String Dim objWReq As WebRequest Dim objWResp As WebResponse Dim strBuffer As String
'Contact the website objWReq = HttpWebRequest.Create(pstrURL) objWResp = objWReq.GetResponse() 'Read the answer from the website and store it into a stream Dim objStream As Stream objStream = objWResp.GetResponseStream Dim inBuf(100000) As Byte Dim bytesToRead As Integer = CInt(inBuf.Length) Dim bytesRead As Integer = 0 While bytesToRead > 0 Dim n As Integer = objStream.Read(inBuf, bytesRead, bytesToRead) If n = 0 Then Exit While End If bytesRead += n bytesToRead -= n End While Dim fstr As New FileStream("weather.jpg", FileMode.OpenOrCreate, FileAccess.Write) fstr.Write(inBuf, 0, bytesRead) objStream.Close() fstr.Close() objWResp.Close() Return "weather.jpg" End Function
Dim strFile As String strFile = RequestWebImage("http://image.weather.com/images/sat/canadasat_720x486.jpg") PictureBox1.Image = Image.FromFile(strFile)
Conclusion
As you can see, retrieving data from a Web site can be easy. Other sites that don’t give you that kind of returned string are much more difficult to handle. In a perfect world, Yahoo would offer a WebService but none is available for now!
I hope you appreciated the topic and see you next month.
Source code
