Thunderclap, the Newsletter of Rolling Thunder Computing

Volume 3, Number 2 Summer 2001

In this issue:

Feature Article: Web Services Drill -- Down: Use SOAP Extension for Encryption, Compression, or Logging
New Book: Introducing Microsoft .NET
Blatant Self-Promotion: New Training Class on .NET
The Internet Chuckle: Deirdre Flint
Contest with Prizes: How Can You Tell a Geek? 
Results of Last Contest: Baby Picture Caption
About Thunderclap
Subscription Information

Feature Article: Web Services Drill-Down -- Use SOAP Extensions for Encryption, Compression, or Logging

This article was inspired by part of a talk that Keith Ballinger, Web Services program manager at Microsoft,  gave at Tech Ed 2001 in Atlanta. It requires familiarity with .NET Web Services. My publisher and my creditors would love it if you would acquire this knowledge by buying and reading my new book on it. But if you haven't, you can get a quick and free overview by reading the online edition of my article in the February 2001 issue of MSDN magazine. 

Web Services make a big splash. In fact, when I teach classes on .NET, I start by demonstrating a simple Web Service and say, "This is what .NET is all about." The .NET Web Service architecture uses HTTP and XML to allows many different types of clients, .NET and non-.NET, even non-Microsoft, to access a .NET Object on a Windows 2000 server.  Everything else exists for the sole purpose of supporting this functionality. For example, a Web Service needs to stay up 24x7, therefore .NET contains automatic garbage collection to prevent failures due to memory leaks.  And the wizards in Visual Studio make a very cool demo, throwing together a rudimentary Web Service in less than 5 minutes. 

The problem is that, as with any piece of technology, the simple demo that you can throw together in 5 minutes doesn't handle all the complexity that a real-life application must deal with. Dotting all the t's and crossing all the i's in a project takes much longer, requires you to get your hands a lot dirtier than the wizard lets you think you'll get away with, and is absolutely necessary to produce a useful, industrial strength program. That's why this edition of ThunderClap will drill down into a Web Service and show you how to get important work done behind the scenes using an interesting piece of software called a SOAP Extension.  

One of the beauties of .NET is that it abstracts away the transport and mapping mechanism by which requests come from the client and are connected to methods on the server. You don't have to think about how incoming requests reach your Web Service; you just write an object and .NET figures out how to map the incoming calls as shown in the diagram below: 

Any high level of abstraction is bound to be both a blessing and a curse -- Visual Basic through version 6 being the classic example, although VB.NET is substantially better in this regard. Real-life industrial strength developers of Web Services often need access to the incoming and outgoing SOAP packets that underlie the Web Service. For example, during development, if you know that a client is trying to make calls to your Web Service, but somehow the calls aren't reaching your server-side objects, you would like to examine the SOAP packet the client is sending to ensure that it is properly formatted. It often isn't, particularly if the client isn't a .NET client built with the SDK's automatic proxy generator. Or you might want to display the incoming calls in a window, say, a .NET Web Service Spy program, or log them for examination by automated QA tools. Your code might also require access to the SOAP packets during run time as well. For example, you might want to compress large SOAP packets on the client side for more efficient network transmission, then decompress them on the server side before routing them to the correct Web Service method. Or you might want to encrypt the SOAP packets to ensure privacy before transmitting them over the network, then decrypt them on the server,  as shown in the diagram below. If I can think of four uses off the top of my head, I'm sure my collective readership can come up with 5 dozen. 

You could certainly do all these things in your Web Service methods.  For example, a client could encrypt pass an encrypted string as an ordinary parameter to a server method, which would then decrypt it and make sense of it, or both client and server methods could call functions that write to an audit log. But that approach would muddle your infrastructural code with your business logic, violating every principal of code organization in the book and making your program essentially impossible to debug. If you changed, say, your encryption algorithm, you would have to change every method, and I guarantee you'll forget some and screw up others. Also, operations such as audit logging exist to provide an independent check on your code, and combining them with the code (and worse, under the control of the programmer) they are supposed to be monitoring would defeat their entire purpose. We'd really like to separate infrastructural operations on the SOAP packets themselves from the business logic portion of our Web Service. 

The SOAP protocol used by .NET allows you to install a SOAP Extension on the server, client, or both sides of your Web Service.  A SOAP Extension is an object that you write that contains the code examines and optionally changes the incoming and outgoing SOAP packets that underlie every Web Service.  ASP.NET will call your SOAP Extension when a Web Service method call arrives on the server, before it gets delivered to the Web Service to which it is addressed. ASP.NET will also call your SOAP Extension when the result comes back from the Web Service object's method, before the result gets returned to the client. This process is shown in the following diagram:

If your client application uses a .NET proxy generated by the VS or SDK proxy generator, you can install and use a SOAP Extension on the client side as well. In this case, your SOAP Extension will be called when the request packet is ready to be sent to the server, and again when the response is received from the server but before it gets returned to the caller. This double-sided operation is necessary for such applications as encryption or compression, but not for such applications as logging or auditing. 

I've written a sample SOAP Extension that demonstrates encryption of SOAP packets to prevent eavesdropping over the wire. This application doesn't authenticate the client's identity, or the server's either. Neither side is really sure who the other is, but both sides want to make sure that no third party is listening in on their conversation. This model encompasses a  surprisingly large number of distributed applications, ranging from online Catholic confessionals (see www.absolution-online.com, with shopping cart, checkout, and all) to kinky sex chat rooms (you don't need me to tell you the URLs) . The sample application that accompanies this newsletter illustrates the former (sorry). Both client and server use a SOAP extension to encrypt data before sending it to their counterparty, and to decrypt it after receiving it from their counterparty. 

You can download the code by clicking here. I wrote it with the Beta 2 version of .NET and Visual Studio, released in June of 2001. The client is shown below. I'm hosting the server on my book's Web site, www.introducingmicrosoft.net. When you unpack the samples, simply run the client program, enter your sin, and click either button to find out what you have to do to receive absolution. If you'd like to run the server on a different machine, then move the RtWebConfessionalVB folder from the sample code into the  Inetpub\wwwroot directory on your server machine. Then enter the URL to this server into the edit control in the client.

We need to write a SOAP Extension that will apply our desired infrastructural logic, in this case, encryption and decryption. All SOAP Extensions must derive from the system base class System.Web.Services.Protocols.SoapExtension. You must override the methods of this base class and supply your own. The interesting method is ProcessMessage, which ASP.NET calls when it is your SOAP Extension's turn to look at the SOAP packet. The single parameter, here called "message", is of type System.Web.Services.Protocols.SoapMessage. If you look this up, you'll find it contains a member named Stage. Your SOAP Extension is called at 4 separate locations in the Web Service process. Rather than force you to write 4 separate handler functions, ASP.NET calls the same method every time, differentiating among them by means of this member variable. On input, the function is called before the incoming SOAP packet is deserialized from the wire, and after. On output, it is called before the Web Service method's return values are serialized into XML, and after. In this case, I use my own auxiliary function Decrypt in the BeforeDesrialize case, to undo the encryption that the client has done to ensure privacy over the wire. I use my own auxiliary function Encrypt in the AfterSerialize case to encrypt the return values just before they go on the wire to the client. The code looks like this:

Imports System
Imports System.Web.Services
Imports System.Web.Services.Protocols

Public Class MyOwnSoapExtension
   Inherits System.Web.Services.Protocols.SoapExtension

Public Overrides Sub ProcessMessage(ByVal message As System.Web.Services.Protocols.SoapMessage)

    Select Case message.Stage

        Case SoapMessageStage.BeforeDeserialize
            Decrypt()

        Case SoapMessageStage.AfterDeserialize
    
        Case SoapMessageStage.BeforeSerialize

        Case SoapMessageStage.AfterSerialize
            Encrypt()

    End Select

End Sub

Now that you've seen it, a few notes are required. Since we are modifying the data stream, we can't use the value passed in the SoapMessage parameter as it is read-only. Instead, Encrypt and Decrypt work on a different reference to the stream, which does allow writing. This reference is passed in a method called ChainStream, which I don't show here but which you will find in the sample code. There are also several other methods in the SoapExtension that must be overridden to get your code to work. Again, they aren't interesting enough to discuss in detail in this article, but the sample code contains working versions.

Obviously, for this architecture to work, both client and server must share the same encryption algorithm. For an encryption algorithm, I used a simple XOR operation to scramble and then unscramble the streams. This wouldn't last 5 minutes against a determined attack, maybe not even 5 seconds. I had hoped to use Pretty Good Privacy, but I couldn't find a shareware version and the company wanted big bucks for a developers' license. I next tried using the public key encryption functions that Microsoft provides in .NET. Because public key encryption is so computationally intensive, ballpark 1000 times more so than symmetric (non-public) key encryption, Microsoft's public key encryption functions will encrypt a maximum of 16 bytes at a time. So you can't easily do public key encryption for your whole message. What you can do, and what Pretty Good Privacy actually does, is to encrypt the message with a symmetrical key, then use public key encryption to encrypt the symmetrical key itself and send the encrypted symmetrical key along with the message. That's too much work for me on this simple newsletter, but you can see the power of the process. If you wanted really secure communication between client and server, you could use one-time pads for encryption. 

Now that we've written our SOAP Extension, we need to tell the server's ASP.NET to use it. We do that by marking each method in our Web Service for which we want to have the extension invoked with a special custom attribute, conceptually similar to <WebMethod()>, but invoking our new custom SOAP Extension. In order to do that, we need to create a new custom attribute, which must derive from the base class System.Web.Services.Protocols.SoapExtensionAttribute. We override the property named ExtensionType, writing code that responds with the .NET type name of our SOAP Extension. The Priority property specifies the order in which SOAP Extensions should be invoked if there are more than one. For example, you would like to ensure that your encryption algorithm ran after the compression code, not before. In this case, we hard-wire in a value of 0. The code looks like this: 

AttributeUsage(AttributeTargets.Method)> Public Class MyOwnSoapExtensionAttribute    
   Inherits System.Web.Services.Protocols.SoapExtensionAttribute


    Public Overrides ReadOnly Property ExtensionType() As System.Type
        Get
            Return GetType(MyOwnSoapExtension)
        End Get
    End Property

    Public Overrides Property Priority() As Integer
        Get
            Return 0
        End Get
        Set(ByVal Value As Integer)

        End Set
    End Property

End Class

In the Web Service itself, we mark the WebMethod( ) with the attribute that tells ASP.NET to use our new extension. The code of the Web Service looks like this:

Imports System.Web.Services

Public Class Service1
Inherits System.Web.Services.WebService

<WebMethod(), MyOwnSoapExtensionAttribute()> Public Function ConfessEncrypted (ByVal Sin As String) As String

    Dim penance as string

    penance =  <perform confession/penance matching logic>

  Return penance

End Function

End Class

The client side does the same thing. You can see it in the downloaded code. If this were a real program, you might make the client side available to your customers for a download. The sample client write the results to a log file, with the name "soapextensionclient.log".  Its contents will look something like this: 

That's all for this time. Want to learn more about .NET? Buy my book, then hire me to come teach it at your company. We'll continue our discussion of  .NET in the next issue of your newsletter. Enjoy playing with your new Web Service program. I can't wait to see what sins people confess to. Give me a call if I can help you out. Until next time,  as Red Green  would say, "Spare the duct tape, spoil the job."

New Book: Introducing Microsoft .NET

by David S. Platt, President of Rolling Thunder Computing

Published by Microsoft Press

ISBN 0-7356-1377-X 

I've done it again. I'm so busy teaching that I don't have time to sneeze (OK, I manage to squeeze it in occasionally). So what do I do? Like an idiot I sign up with Microsoft Press to do another book, this one on .NET. It's similar in content and style to my earlier Understanding COM+, and aimed at the same readership.   It is meant to be accessible to managers, while still providing enough information to be useful to programmers, both lightweight VB types and heavyweight C++ types.  The book is meant to be non-threatening, using a lot of pictures and diagrams, a small amount of simple VB code, and no C++ at all, which would scare off 80% of the potential readers. I write each chapter in a pyramidal structure, so managers can read the first three sections (Introduction, Problem Background, Solution Architecture). Ambitious managers and VB programmers can continue through the next section (Simplest Example). Heavier-duty developers can read the ends of the chapters, where I discuss more advanced elements of the topic.  The book's Web site is, naturally,  www.introducingmicrosoft.net. There you will find a free chapter on ADO.NET, as well as the code for all the book's samples, and a list of errata as soon as we find some. You'll also find a glowing review of it here (well, what other kind did you expect me to link to? I may be crazy, but I'm not stupid.) written by Manohar Kamath. Here's the table of contents: 

The book was officially released on May 11, and should be on shelves now. On June 11, its sales rank on amazon.com was #762, thumping Tom Clancy's latest hardcover and paperback, hopelessly mired around #1400. That tells you something about Amazon's customers -- they're all geeks.

Blatant Self Promotion

In-House Training Class on .NET

.NET is here, and it's hot. It changes everything in the software business, and you can't afford to be without it. Rolling Thunder Computing now offers in-house training classes on .NET, using my book as the text. We offer a one-day overview and a five-day programming class. You'll find the syllabi at http://www.rollthunder.com/DotNetClassFrame.htm. 

Public 5-day Class on .NET for Insurance coming this Fall

My work in the insurance industry has convinced me that it's different from other business areas. Not every company is large enough to afford an in-house class, or So I'm looking at putting together a 5-day intensive class aimed particularly at this industry. It will happen sometime in the fall, perhaps October, and probably be located in Boston or maybe Seattle. E-mail me, dplatt@rollthunder.com, for further information and to be kept abreast of the latest developments. 

In-House Training Class on XML and XML for Insurance

You've probably been hearing an enormous amount of hype these days about XML. It sounds like a great idea, and it is indeed a beautiful solution to certain classes of problems.  This four- or five-day training class covers the guts of XML, such as well-formed documents, validating documents with DTDs or schemas, programming XML using DOM and SAX, and transforming XML with XSL stylesheets. Most important, we talk about how to use XML in real-life applications: the advantages and drawbacks, the places where it fits well, and the places where you'd be trying to bang a square peg into a round hole. The insurance industry version of this class covers the ACORD XML standard, either for life insurance or P & C (or both).  You can read all about the new XML class on Rolling Thunder Computing's web site, http://www.rollthunder.com/xmlframe.htm. 

Public Three-Day Class on XML for Insurance in Hartford CT, August 20-22

I'll be presenting this 3-day training course is designed to educate the technical person on XML and the ACORD XML standards for both P&C and Life insurance. It gets beyond the hype and teaches programmers what they need to know about this technology standard. This one is sponsored by Traveler's Insurance in Hartford CT. The syllabus is at www.acord.org.  Register with gprescott@acord.org. 

Public 4-Day Class on .NET in Reykjavik, Iceland, Aug 27-30.

I'm going back to Iceland to teach a 4-day class on .NET. The dates are August 27-30 (finally I got them to schedule a class outside the dead of winter), at the University of Iceland in Reykjavik. It is open to the public, and will be taught in English. Information about the class can be found here.

Internet Chuckle: Deirdre Flint

I was prowling around the Web one day and came upon a concert entitled "Funny Songwriters." I'd heard of and enjoyed several of them, such as Don White, but Deirdre Flint was new to me. She has posted several songs publicly on MP3.com, which you can listen to and read about at http://artists.mp3s.com/artists/26/deirdre_flint.html. You can read her bio and upcoming shows on her own web site, which is www.deirdreflint.com. She's currently running a haiku contest on the topic of cheerleading. 

Her songs are funny while having a cynical, bitter edge that I greatly admire. For example, the lyrics to "Cheerleader" go "Cheerleaders drive Camaros / Cheerleaders don't date geeks," and also, "Some women say that it harms and demeans / Can you guess who didn't make their high school teams?"  Or "Listen", about a relationship where the love disappeared long ago, but the biological attraction remains. As a fan of good analogies (for example, using an in-proc COM server is like having unprotected sex, but using an .EXE server going through a proxy and stub is like wearing a condom), I love when she compares her relationship to "a narcoleptic hand model learning how to use a chipper." In "1-900-Score a Date" she imagines a way to give feedback after a first date without actually having to talk to the person again, containing the line "Live rejection's so passe / Get dumped the fiber optic way."  There's an Internet startup opportunity if I've ever heard one. I won't describe her song "Boob Fairy", other than to say that I think men and women will laugh equally hard, but at different parts of the song.  

Disclaimer: This section will often refer to commercial vendors of products or services. Rolling Thunder Computing receives no benefit from these vendors in any way, shape, form, or manner; and would decline any if offered. The sole criterion for mention in this section is that I laughed at it, and that I think people as sick as I am would do so as well. If you know a good one, send me the URL at chuckle@rollthunder.com. If I use it, I'll send you a Rolling Thunder Computing coffee mug, which makes an excellent specimen container when your company implements mandatory drug testing. 

Contest with Prizes: How Can You Tell a Geek?

Lord, send a man like Robbie Burns to sing the Song o' Steam! 
-- Rudyard Kipling, "McAndrew's Hymn", 1894

I'd settle for Dr Seuss, except he's dead.
-- David S. Platt, "Plattski's Musings", 2000

You see them on the street. You work alongside them. Since you subscribe to this newsletter, you probably are one yourself. Yes, I speak of geeks. The plastic shirt pocket protector used to be the giveaway, but now most of them (us!) wear T-shirts as part of the uniform, so that doesn't work any more.  How do you tell if someone is a geek? You tell me. For example:

If you say, "Hey, watch me download this!" when you chug a beer, you're a geek. 

If you tape down two of your child's fingers so she learns how to count in octal, you're a geek.

If you hear that a couple is going to lose their virginity live on the Web, and your first thought is, "Wow! Think of the load on their server," you're a geek. 

This contest was inspired by Jeff Foxworthy's classic routines about identifying rednecks ("If your three-year-old has a gun rack on his tricycle, you might be a redneck.")  As always, your entries must be computer-related and original. This isn't a family newsletter, so they don't have to be clean or politically correct; in fact, I'd just as soon they weren't. Submitting more than one increases your chances of winning. First prize is $100, second prize $50. Winners will be announced in the next issue of Thunderclap, and the judge's decision (mine) is final. In the event of duplicate or similar entries, the earliest wins. All entries become property of Rolling Thunder Computing.  All authors will be identified by first and last names, but not by company. Authors names will be withheld if you so request. Submit your entries via e-mail to contest@rollthunder.com.

Results of Last Contest: Picture Caption Contest

You were asked to submit a caption for the picture of me and Annabelle (then aged 4 months, now almost a year) shown above (I couldn't resist showing it again, aren't proud new fathers a pain in the ass?). As you'll notice, the majority of submissions deal with bodily emissions from one orifice or another. Those must be the ones from other parents.

First Prize, $100 to Phil MacArthur:

"My dump routine is completed. Please reboot."

Second Prize, $50 to Perry Riposte

"You can't fool me, Daddy. That's an Etch-a-Sketch, not a baby palmtop." 

And Honorable Mention to all:

"You mean programs barf too?"

"Soon I'll be puking worse than your code." 

    -- Peter Van Caeseele

"Daddykins, about that missing manuscript of your latest book... I admit it! I could not keep my fingers off it and those pieces of paper never tasted sooo good!"

    -- Le Nguyen

"Of course I'll smile for you Daddy!  I just pee-peed on your laptop!"

"I'm so happy!  I just downloaded something and we're taking it to Microsoft!"

"I thought Mommy wasn't supposed to use the digital camera during take-off and landing!"

"So I should smile like this and call Mr. Gates 'Uncle Bill' ?"  

    -- Barry Doyle

"What, you have a network of computers to introduce Microsoft to people?. (www.introducingmicrosoft.net )?"

    -- Sateesh Narahari

"More entertaining than Combat Flight Simulator, aren't I?"

"OK, I'll code that up for you on my Blackberry." 

    -- Phil MacArthur

"Lunch was delicious, thank you. And have I got a problem report for you!" 

    -- Hugh Blair-Smith

"The good news is I delivered my project on time, the bad news is now my pants are dirty." 

    --  Bill Johnson

"Looks like COM+. Fresh, Promising Technology in Hands of Experience. "

    -- Amit Limaye

"I don't care if you are Platinum!! Little Annabelle here gets me pre-boarding privileges!!!"

    -- Ian M. Layton

"The introduction of Try-Catch error handling in .NET has me drooling with anticipation"

    -- Shawn Richard

"Squeeze me again, Daddy! Maybe we can get that guy on the right hand side to move like we did the rest of 'em."

    --Nicker

"So that's how you got the lady to move to another seat!!!  Don't get me started"

    -- Mark Bishoff

About Thunderclap

This is the tenth issue of my (more or less) quarterly newsletters. Each will bring you a technical article on current topics in software development, which I hope you will find useful. Each will also bring you a contest, allowing you to show off your intelligence and creativity to win prizes, which I hope you will find funny (although some readers have reported the reverse.) In between you will find my own blatant self promotional material, telling you about the latest ways I've come up with to separate you from your money. (I could have said "carefully selected products and services that we feel might interest you", or other mealy-mouthed horsepuckey. You want the truth or you want me to waste my time and yours dressing it up?)

I'd like to hear what you think about this newsletter and what types of articles you'd like to see in the future.  Send your e-mail comments to newsletter@rollthunder.com.

This newsletter may be freely redistributed, provided that it is sent in its entirety. If you enjoyed it, can I ask you to please forward it to a friend who might also enjoy it? The subscription roll has grown to over 4000 for this issue.

Subscription Information

Thunderclap is free, and is distributed via e-mail only. We never rent, sell or give away our mailing list. Subscription and unsubscription are handled by a human operator reading e-mail messages. To subscribe or unsubscribe, jump to the Rolling Thunder Web site and fill in the subscription form.

Legal Notices

Thunderclap does not accept advertising; nor do we sell, rent, or give away our subscriber list. We will make every effort to keep the names of subscribers private; however, if served with a court order, we will sing like a whole flock of canaries. If this bothers you, don't subscribe.

Source code and binaries supplied via this newsletter are provided "as-is", with no warranty of functionality, reliability or suitability for any purpose.

This newsletter is Copyright © 2001 by Rolling Thunder Computing, Inc., Ipswich MA. It may be freely redistributed provided that it is redistributed in its entirety, and that absolutely no changes are made in any way, including the removal of these legal notices.

Thunderclap is a registered trademark ® of Rolling Thunder Computing, Inc., Ipswich MA. All other trademarks are owned by their respective companies.