Assumptions
This tutorial is written with the assumption that the reader has at least heard of the term
subclassing, and has some experience with using the windows API.
Requirements
You should be using Visual Basic 6.0, and have access to a Spy++ like program. I no longer use Spy++ after finding a better alternative on the internet. I will, however, be using it for this article since that is what most people using Visual Basic 6.0 have. If you are interested in using a different program or don't have access to Spy++ for whatever reason you can download the one I use
Winspector, or one of many other alternatives that can be found on the web. Here are a couple others:
MiniSpy,
WinSpy++.
Introduction
Subclassing is a way of gaining access to the most important part of any window, the window procedure. This procedure is the part of the window that deals with all the messages windows sends to it as things happen.
When using something like
SendMessage, or
PostMessage what you are doing is sending the message to the window procedure of a window. It is up to the window procedure to deal with each message in a way that it needs to, to accomplish its tasks. For instance when a user of a program clicks their mouse in a given window there are a series of messages sent to it by the operating system. It is up to the program, specifically the window procedure of the program, to process the messages.
Every window, as far as I know, that has an .hWnd property also has a window procedure that processes the messages for the window. Subclassing is a way of replacing this original window procedure with one of your own. By doing this, the operating system will then send all the messages to the procedure you have written, thereby allowing you to see and deal with any of the messages you want. After dealing with all the messages you are concerned with you should then pass each message along to the default window procedure, the one you replaced with your own, so it can handle the default processing of the message.
I know this can sound a little confusing to someone who is new to subclassing, but hopefully once you work through this article with me it will become clear.
Background
When writing a little notepad like program, I decided to switch to a RichTextBox instead of the standard TextBox control for a few reasons. First, I wanted to be able to work with large files, and the intrinsic TextBox control has a limit of somewhere around 32k. Second I thought that in the future I might want to make my text editor into a programming editor with syntax highlighting and to do that I needed to be able to change the colors of certain words which you can't do with the TextBox control.
Problem
While developing this editor I ran into a problem when pasting text into the RTB (RichTextBox). I originally had a menu item for the user to select when they wanted to paste some text and, of course, I assigned it the shortcut of "Ctrl+V" to be consistent with the windows shortcut conventions. The problem here was that when using the shortcut, it would not only paste the text in the RTB twice, but if the text on the clipboard was in RTF (Rich Text Format), then one copy that was pasted would show up with the formatting.
The first part of the problem can be summed up as follows. I programmed the shortcut into my program that took care of the paste action from the clipboard into the RTB. The problem is that the RTB already has this shortcut "Built in" and therefore it was being done once by my program and once by the RTB itself. This is a relatively straight forward problem to solve, by just taking out the shortcut of the menu item and changing that menu items .Caption property in the form load. Now my code would only execute when the menu item is actually clicked, and the RTB would take care of the shortcut. The menu item still looks like it has a shortcut to the user and they have no idea, nor do they need to, that there are two different pieces of code being executed depending on whether they use the shortcut or click the menu item.
Even though the first part of the problem is solved, there still remains the trouble of the RTF formatted text showing up in my RTB. This problem is a little more difficult to overcome. I tried everything I could possibly think of to take care of this problem with the built in functionality of Visual Basic but nothing I came up with fit in as cleanly as I wanted. I then decided to search the internet, thinking that someone must have come across this problem before me and hopefully they posted some code somewhere that I could use. I found a few solutions, but once again nothing that seemed "right" to me. One of these solutions was bound to cause problems with the program at some point, because it was doing quite a bit of work inside the KeyUp event. It would copy all the text from the RTB to a temporary string and then back into the RTB's .Text property. This, as you could imagine, would cause quite a bit of slowdown in my program when working with large files. I then decided it was time to take the leap, subclass the RTB and take care of the problem before it even occurs.
Time to Subclass
The first thing I did was setup the basic subclassing code in a blank project with just a RichTextBox (named "rtbMain") and a menu item with "Paste" under the top level item "Edit." Then I installed the subclassing code. To do that, create the blank project I mentioned and add a new standard module to the program by clicking the project menu and then "Add Module", and call it "modSubclass." Now in the module just copy and paste the following code:
Code:
Option Explicit
Public Declare Function CallWindowProc Lib "user32" Alias "CallWindowProcA" _
(ByVal lpPrevWndFunc As Long, _
ByVal hwnd As Long, _
ByVal msg As Long, _
ByVal wParam As Long, _
ByVal lParam As Long) As Long
Public Declare Function SetWindowLong Lib "user32" Alias "SetWindowLongA" _
(ByVal hwnd As Long, _
ByVal nIndex As Long, _
ByVal dwNewLong As Long) As Long
' A pointer to the old window procedure
Public pOldWindPoc As Long
Public Const GWL_WNDPROC& = (-4)
' Our new window procedure
Public Function WndProc(ByVal hwnd As Long, _
ByVal uMsg As Long, _
ByVal wParam As Long, _
ByVal lParam As Long) As Long
WndProc = CallWindowProc(pOldWindPoc, hwnd, uMsg, wParam, lParam)
End Function
This is the new window procedure, which will be put in place in a minute, as well as the API declares we need to get it running. There's really not much going on here, all the new window procedure does is pass all the messages it receives on to the default window procedure by using the API
CallWindowProc. The reason I do it like this is to be sure that everything is working correctly before I start to deal with any messages. This doesn't actually do any of that yet though because the window procedure needs to be installed first. To do that we need to add some code to the Form. The code that needs to be added goes into the Form_Load event and the Form_Unload, or the Form_QueryUnload whichever you prefer. Here is the code for that:
Code:
Option Explicit
Private Sub Form_Load()
' Redirect messages to our new handler in the module and
' save the pointer to the old handler
pOldWindPoc = SetWindowLong(rtbMain.hwnd, GWL_WNDPROC, AddressOf WndProc)
End Sub
Private Sub Form_Unload(Cancel As Integer)
' Restore the previously saved message handler
SetWindowLong rtbMain.hwnd, GWL_WNDPROC, pOldWindPoc
End Sub
It is very important to restore the old window procedure before the program terminates, and by doing it this way you don't have to worry about it later. Now you should save the program before running it because of the instability subclassing can add to your program. Once you have done this much you should always save your program before running it, and remember to never press the end button in the IDE. By saving your program every time you run it you won't lose anything if the program crashes, and by not pressing end you make sure that everything gets cleaned up properly in your Form_Unload or Form_QueryUnload event by restoring the old window procedure. Just to test that everything still works the way it should, go ahead and save it now and run the program. Test everything out, and paste some RTF text into the RTB so you can see an example of the problem we are about to solve.
Once I decided to subclass the window (RTB), it was time to figure out what message or messages I was interested in getting at. I remember seeing, somewhere on the web, that the message was WM_PASTE. So I turned to google using the search terms, in quotes, "Visual basic" and, not in quotes, WM_PASTE. Don't bother going through the trouble of trying this because the RTB doesn't get the WM_PASTE message when the user presses "Ctrl+V."
Continued in next post
Subclassing as a Method of Problem Solving (Part 1)
Linear Mode
