Rob Garrett - Blogs

Welcome to Rob Garrett - Blogs Sign in | Join | Help
in Search
Google

Software/Technology Discussion

Software and Technology Tid-bits

NOAA Weather Service and .NET

I have been fighting with the national weather web service, provided by the National Oceanic and Atmospheric Administration, today.  Like a lot of developers, who like to make their web sites fancy with the current weather forecast, I have been developing a component for the web site I am working on.

NOAA seems to be the norm for providing weather data in XML via the SOAP web service protocol.  Up until a couple of weeks ago, the C# code that I had to connect to the NOAA web service and pull weather data, was working  nicely....then it stopped - bah.  The proxy code (generated from the service WSDL by .NET) kept throwing an HTTP protocol violation exception:
Unhandled Exception: System.Net.WebException: Error getting response
stream (Rea dDone2): ReceiveFailure ---> System.Exception:
at System.Net.WebConnection.HandleError ()
at System.Net.WebConnection.ReadDone ()
at System.MulticastDelegate.invoke_void_IAsyncResult ()
at System.Net.Sockets.Socket+SocketAsyncResult.Complete ()
at System.Net.Sockets.Socket+Worker.Receive ()
at System.MulticastDelegate.invoke_void ()
in <0x000ad> System.Net.WebConnection:HandleError (WebExceptionStatus
st, System.Exception e, System.String where)--- End of inner exception
stack trace ---
in <0x00158> System.Net.HttpWebRequest:EndGetResponse (IAsyncResult
asyncResult)in <0x00047> System.Net.HttpWebRequest:GetResponse ()
in <0x0004d>
System.Web.Services.Protocols.WebClientProtocol:GetWebResponse (Sys
tem.Net.WebRequest request)
It seems that NOAA changed something involving their service protocol, because the response to a SOAP request is now being returned in separate chunked HTTP responses, which causes the .NET SOAP client to puke.

After some digging on the Internet (lots of digging in fact), I came across a post on a Mono newsgroup, which gave me a work around to the problem.  The trick is to change the web request in the SOAP proxy to use HTTP version 1.0 instead of 1.1.  Since HTTP 1.0 does not support chunked response, the NOAA server sends the response data as one block, which keeps the .NET SOAP client happy. 

Below is the code for my web service proxy, drop it into your project (assuming you use C#) and instantiate the WeatherProxy class from your code.  You'll not need to add a web reference to your project because this proxy class is attributed with the correct SOAP envelope information.

Share this post: Email it! | bookmark it! | digg it! | reddit!
Published Tuesday, January 24, 2006 4:24 PM by Rob Garrett

Comment Notification

If you would like to receive an email when updates are made to this post, please register here

Subscribe to this post's comments using RSS

Comments

 

Ed Mooers said:

Yikes! You're code formatter really munged up those XML comments! Still, it looks good. I'll have to give this a try over the weekend. Great job!

Ed

PS: Maybe you could create a little download link for code like this? ;-)
January 24, 2006 10:31 PM
 

Ed Mooers said:

And "you're" spell checker didn't stop me from looking like an idiot!
January 24, 2006 10:31 PM
 

Rob Garrett said:

Yeah, the code syntaxer is a little too smart for it's own good, I really need to find an alternative tool, or to fix it.

I was going to put the code on download but was in haste this afternoon (mainly because I'd spent all afternoon spitting at NOAA). I can put the file on line if you'd like it.
January 24, 2006 10:49 PM
 

TrackBack said:

February 14, 2006 2:36 PM
 

TrackBack said:

February 14, 2006 2:37 PM
 

Brian said:

Any idea how a lowly VB programmer can use this code? I've translated it into VB (using online tools) and can't seem to get it to parse and/or work.

GRRRRRRRRRRrrrrrrrrrrrrrrrrrrr is right!
February 17, 2006 4:40 AM
 

Rob Garrett said:

NOAA broke their service, see my latest post...

http://robgarrett.com/blogs/rant/archive/2006/02/14/1847.aspx

1. Is your VB code compiling?
2. I would use the Weather.com service on account of NOAA's service being unreliable.
3. If the translation of C# to VB.NET is your issue, can you not just create a class library with my code and call it from your VB.NET code?
February 17, 2006 10:58 AM
 

David Berman said:

Could you please provide the file with the source code for this? Do you have this working for ASP.NET 2.0 / Visual Studio 2005? Thank you for sharing this information.
March 8, 2006 12:16 PM
 

David Berman said:

I have this compiling now using Visual Studio 2005. The trick was to paste your code into a new class file, delete the line numbers from the left side, and add the following namespace references:

using System.Net;
using System.Xml.Serialization;
using System.Web.Services;
using System.Web.Services.Protocols;
using System.ComponentModel;

However, now instead of the protocol violation error I get a different error:
The request failed with HTTP status 502: Bad Gateway
This comes from line 62:
object[] results = this.Invoke("NDFDgenByDay", new object[] {

Any ideas?
March 8, 2006 12:32 PM
 

Rob Garrett said:

A word of warning - the last time I looked at NOAA's weather web service, they had broken the SOAP interface (http://robgarrett.com/blogs/rant/archive/2006/02/14/1847.aspx
). My advice would be to sign up with http://weather.com and use their service instead.


BTW, if you click "View Plain" in the top right of the code window you do not have to remove the line numbers from my code yourself.
March 8, 2006 2:10 PM
 

Mikhail Arkhipov (MSFT)'s WebLog said:

NOAA has been changing its weather forecast Web service and my Weather Report control has been broken...
June 13, 2006 4:50 PM
 

Stephen said:

This helped me out a ton!!!!  Had this same problem and were it not for you, those guys down at the NWS would have received some floggings.
August 8, 2006 1:31 PM
 

Jorge Bastos said:

How about this for winforms? can i do it? anyone has a converted example?
August 14, 2006 12:36 PM
 

Rob Garrett said:

Jorge,
   This code should work the same in a winforms application.  The code makes a web request to the SOAP weather service, which provides data as XML (when it is working).  As long as you add a reference to System.Web to your project (for the web client) you should be good to go.

R.
August 14, 2006 2:12 PM
 

Henry Bruce said:

Rob,

Nice work. I found that if you insert the HTTP1.0 override into the WSDL web reference generated code, it starts to work.

It strikes me that this is a more a .NET 2.0 missing feature that a NOAA problem.

Henry
August 15, 2006 3:00 PM
 

JacDev said:

Great tip! I took this a bit further,  incorporating inheritance. Here's my sollution (VB.NET)


Public Class WsCustomProxy
  Inherits WsNamespace.WSProxy
  Protected Overloads Overrides Function GetWebRequest(ByVal Uri As Uri) As System.Net.WebRequest
     Dim req As HttpWebRequest = CType(MyBase.GetWebRequest(Uri), HttpWebRequest)
     req.ProtocolVersion = HttpVersion.Version10
     Return req
  End Function
End Class
September 6, 2006 11:40 AM
 

Sebastian said:

I have the same problem that David, I receive a diferent error: 502 Bad Gateway.

Somebody fix this?
September 13, 2006 9:41 AM
 

Henry Bruce said:

Just tried to get the NDFDgen() method working using the same trick, but it generates the "Bad Gateway" exception. Not sure why NDFDgenByDay works and this doesn't...

Any ideas ?

Henry
September 18, 2006 4:41 PM
 

Sam said:

The service seems to return a 502 error when you pass it bad parameters. I was calling NDFDgenByDay and finally got it to work by passing the longitude value as a negative number. (In addition to using the http 1.0 workaround discussed above)
September 28, 2006 2:56 AM
 

Steven said:

Had any luck with NDFDgenAsync? I've got: Private Sub NOAA_Completed(ByVal sender As Object, ByVal e As NOAA.NDFDgenCompletedEventArgs) Handles xWeather.NDFDgenCompleted Try If e.Cancelled Then ElseIf Not e.Error Is Nothing Then Else sWeather = e.Result End If Catch ex As Exception ErrorMsg(ex) End Try End Sub But it never seems to fire off the event...
January 21, 2007 8:22 PM
 

Pooja said:

I am consuming the web service in windows app, Visual Studio 2005, C# and getting the exception - {"The server committed a protocol violation. Section=ResponseStatusLine"} What can I do to resolve this? Thanks Pooja.
January 24, 2007 1:26 PM
 

Ryan said:

As other people have said, throwing this in Reference.cs file(in the main class): Using System.Net; ............ protected override WebRequest GetWebRequest(Uri uri) { HttpWebRequest request = (HttpWebRequest)base.GetWebRequest(uri); request.ProtocolVersion = HttpVersion.Version10; return request; } Worked for us using 2.0
January 26, 2007 12:11 PM
 

Craig Scheets said:

I struggled with this for a bit and finally put together a (good) working solution... I finally figured this out using so many sources it's hard to give credit but this idea isn't mine, just the implementation. THE REAL ASP.NET 2.0 SOLUTION: I fought this for a while and finally got the right code to fix this problem in ASP.Net 2.0. As many of you know, ASP.Net 2.0 generates the proxy classes dynamically so you can't just simply edit them. The key is in a new feature, 'partial classes'. Below is everything you need to know to fix this in ASP.Net 2.0. First, my web reference declaration is: weather.ndfdXML The standard instantiation would then look like: weather.ndfdXML2 wxService = new weather.ndfdXML2(); With this setup, I ran into the exact same problem everyone else has written about here and rarely solved. First, add a new class and make it a partial class (C# code posted): using System; using System.Net; using System.Web.Services.Protocols; namespace weather { public partial class ndfdXML2 : weather.ndfdXML { protected override System.Net.WebRequest GetWebRequest(Uri uri) { //throw new Exception("Custom WebRequest override code hit!!"); System.Net.HttpWebRequest webRequest = (System.Net.HttpWebRequest)base.GetWebRequest(uri); webRequest.KeepAlive = false; webRequest.ProtocolVersion = HttpVersion.Version10; return webRequest; } } } You can optionally uncomment that new exception to verify that the code is indeed executing. Second, change you instantiation inside your code to the following: weather.ndfdXML2 wxService = new weather.ndfdXML2(); Now everything works great!!
February 2, 2007 2:23 PM
 

Eric said:

Thanks for the walkthrough code sample, Craig Scheets. I converted it to VB and I finally got it working. Everyone else, thanks for the discussion and learning points.
February 14, 2007 1:00 PM
 

Richard Clayton said:

Thank you so much for this solution. I was easily able to get the rest of the Methods working by pasting them from the Web Reference into the WeatherProxy class. Now you can access the Asynchronous download methods as well!
March 17, 2007 1:33 AM
 

Mayank Prajapati said:

Getting Error : Unable to read data from the transport connection I am using this same web service. NDFD service for wether data : http://www.weather.gov/forecasts/xml/ First i got error : underlaying connection closed. its resolved after i puts protected override System.Net.WebRequest GetWebRequest(Uri uri) { System.Net.HttpWebRequest webRequest = (System.Net.HttpWebRequest)base.GetWebRequest(uri); webRequest.ProtocolVersion = HttpVersion.Version10; webRequest.KeepAlive = false; return webRequest; } But now in some cases its gives me error: Unable to read data from the transport connection. Connection closed. Plz.. can any one help me on this? I m using ASP.Net 2.0
March 27, 2007 9:43 AM
 

Mark Chadwick said:

Thanks Craig for debugging that problem. I thought I'd post the VB. I added the web reference using http://www.weather.gov/forecasts/xml/SOAP_server/ndfdXMLserver.php?wsdl and then changed the default namespace VS2005 came up with from 'gov.weather.www 'to just 'weather'. Namespace weather Partial Class ndfdXML2 Inherits ndfdXML Protected Overrides Function GetWebRequest(ByVal uri As Uri) As System.Net.WebRequest Dim oWebReq As System.Net.HttpWebRequest oWebReq = MyBase.GetWebRequest(uri) oWebReq.KeepAlive = False oWebReq.ProtocolVersion = System.Net.HttpVersion.Version10 Return oWebReq End Function End Class End Namespace
May 22, 2007 7:55 PM
 

Consuming NOAA and Avoiding Protocol Violation in .Net at OmegaMan’s Musings said:

May 28, 2007 3:11 PM
 

OmegaMan said:

I recently consumed the NOAA web service and began receiving the problems you had. I wanted to continue to use the proxy class which was generated by Microsoft. I simply took your override of GetWebRequest and placed in the main class of my reference.cs hidden proxy file. I have blogged here ( http://www.omegacoder.com/?p=40 ) and gave this page as a link of thanks.
May 28, 2007 3:13 PM
 

Tommy Sundling said:

Just wanted to send a thank you, your code helped me with a similar problem I've been having the last couple of days :-) Warm regards from Stockholm, Sweden.
July 6, 2007 4:42 AM
 

Anne said:

When you say proxy, do you mean your service class that inherits from System.ServiceModel.ClientBase? If I put that override there, the compiler says that there is no suitable method to override. I know this solution will work, I just need to know where exactly to implement it.
August 10, 2007 4:33 PM
 

Stew said:

Anyone have a VB.NEt version of this? And how can I make this dynamic? Send the lat long to get the weather from different places? Thanks kindly for your time.
August 14, 2007 9:33 AM
 

Arturas said:

Thaks for Rob and JacDev, who write translated code. This helps me in one project. Greateengs from Lithuania
January 24, 2008 2:47 AM
 

Chris said:

Do you know if NOAA changed something recently? I was consuming their web service fine and then one day I started getting the "the server committed a protocol violation" error. I made the changes to the Reference.cs as above and it seemed to fix the problem. It is just odd that it worked yesterday and all the sudden it does not work today.
February 7, 2008 12:33 AM
 

a2z said:

Hello, Can you provide in classic asp the above code. http://www.a2zinterviewquestions.com
March 18, 2008 2:51 AM

Leave a Comment

(required) 
(optional)
(required) 
Submit

Blurb


Head Shot
Rob Garrett is a British Expat living in Maryland USA. Rob is a trained software engineer and experienced in Windows .NET development.

Rob enjoys listening to Rock music, posting to blogs, driving in the country with the sunroof open, beer (not in conjunction with country driving) and spending time with his family.

This Blog

Syndication

Powered by Community Server, by Telligent Systems