Thursday, November 29, 2012

Assessment of Hurricane Sandy Performance Back On - But With ZERO Independence...

According to a recent post on the Capital Weather Gang's blog, NOAA announced this afternoon that the assessment of how the National Weather Service (NWS) performed during the Hurricane Sandy event is back on, after being abruptly cancelled on November 15th.   Here is an excerpt of NOAA's statement from the blog post:

One of the most promising aspects of the original service assessment team was that for the first time it would have heavily involved members of the private sector as well as social scientists and other experts - not just members of the same government agencies that were responsible for issuing watches and warnings during the event.

I call your attention to the last line of today's statement, which suggests that non-government participation will be sorely lacking in the new assessment:

How unfortunate.

To be clear, I was in no way involved with the assembly of the original assessment team (I have received e-mail inquiries regarding that possibility based on my response to the suspension of the original group).  As a result, I harbor no "sour grapes" as to whether or not outside or private sector individuals should be involved in the process.

My frustration comes from being a meteorologist concerned with how public and commercial concerns react to an impending severe weather threat such as Sandy, and improving future forecast and warning products based on past experiences (including mistakes, which I clearly believe were made in this particular case).  Until we involve key members of the media, emergency management, social media experts, social scientists and private sector individuals in the assessment of these major events, I feel that we will potentially continue to let the public and other end users down in these situations.

Putting my concerns aside, NOAA's decision to proceed with the assessment without the involvement of individuals from outside of the government appears to show a lack of concern with point 6 of U.S. Rep. Paul Broun's letter to the Administrator of the agency, issued on November 20th:

It certainly appears that Rep. Broun was concerned that the new assessment team would remain "independent", as was promised by the assembly of the first team.  Unfortunately, it appears as though that won't be the case this time around.

I wonder what kind of response we can expect from Rep. Broun and his committee?  Will they accept NOAA's actions or demand that some level of independence be restored to the "official" examination of this historic weather event?

For now anyway, it appears as though the fox will continue to guard the hen house when it comes to reviewing government performance during extreme weather events in the U.S.

For more information from 'The Original Weather Blog', including shorter, more frequent posts during rapidly changing weather events, please be sure to follow Rob on facebook and twitter:


NWS employee said...

Your assertion that NOAA/NWS does not extend beyond the scope of inter-agency when it comes to service assessments, is inaccurate. Past service assessments have involved members from universities, the media, etc. No, they usually do not comprise the majority of team members. One would not expect that to be the case, given it is an internal evaluation. But, to quote from the assessment page, "Assessment teams, composed of experts from within and outside the NWS, evaluate activities before, during, and after events to determine the usefulness of NWS products and services.The team generates a report, which serves as an evaluative tool to identify and share best practices in operations and procedures, and identify and address service deficiencies. The goal of the activity is for the NWS to continuously improve its services to the nation." I work with members that have been on such service assessment teams, and can verify this is in fact the case. All service assessments clearly list out the assessment members. One can debate whether there is enough representation from beyond the agency. But it should not be portrayed that neither NWS or NOAA extends its assessments beyond the scope of the agency. Blogosphere's and personal rant pages tend to sometimes be seeds of misinformation and often times are not vetted - Respectfully, and NWS and NWSEO member.

Rob White said...

NWS Employee,

Thanks for your comment.

I find it unfortunate that you feel that anyone who challenges the status quo of NWS service assessments is engaging in a misinformed "personal rant." While I find it unfortunate, I don't necessarily find it surprising, which I think has a lot to do with the NWS's (or NOAA's) ultimate decision to terminate the original assessment team and establish a new one that will be lead, as usual, entirely by NOAA/NWS officials.

As I mentioned in my post, I have no sour grapes in this matter. I simply wish to see important lessons learned from this case so that we can make sure and repeat what worked in the Sandy event, and absolutely NOT repeat what didn't.

To be frank, I feel that many, particularly at the administrative level of the NWS are afraid to see what revelations may be brought about by "outsiders". It's kind of like getting bad news at home - wouldn't you rather receive it from Grandma rather than someone outside the family?

Unfortunately, when it comes to public service and public safety, we don't have the luxury of having Grandma give us all the bad news all of the time. Sometimes it needs to come from our neighbor Joe down the street in order for its full perspective to be appreciated and for meaningful changes to be made as a result.

I sincerely hope that the findings of the new assessment team are objective and that honest, hard recommendations are made. Unfortunately, I don't have much confidence in that scenario coming about as long as the fox is the one put in charge of guarding the hen house.