[PrOpHeT]

****************************************************************
***Removed because initial logic was flawed further confused the questions below***
****************************************************************

[jo0ls]

I think the confusion is just a typo:

    Private Sub TestOne(ByVal Var As String)
        _TestVar = "Bar" 'passed ByVal here, but _TestVar was modified
    End Sub

Here you are using _TestVar, not Var. That's why it is modified, it wouldn't be if you used Var.

The only complication with String is that it is immuatable. Every time you "alter" a String you are really creating a New one. And the difference in the behaviour of ByVal and ByRef with reference types occurs when you try assigning new objects to the variable in the method that you are calling.

Recap...

    Sub Sub1()
        Dim s As String = "Foo"
        Sub2(s)
    End Sub

    Sub Sub2(ByVal message As String)
        ' A
        message = "Bar" 
        ' B
    End Sub

At A we have passed a copy of the reference, so we have this situation:

variable name          Stack             Heap
                  .------------.       
                  | address    |       
s---------------->| of         |
                  | "Foo"      |-----------V
                  '------------'     .------------.
                  .------------.     |  "Foo"     |
                  | address    |     '------------'
message---------->| of         |-----------^
                  | "Foo"      |       
                  '------------'

At B, because Strings are immutable, we have not altered the "Foo" object but created a new object with the value "Bar".
The line message = "Bar" can be thought of as message = New String("Bar").

variable name          Stack             Heap
                  .------------.       
                  | address    |     .------------.
s---------------->| of         |---->|  "Foo"     |
                  | "Foo"      |     '------------'
                  '------------'     
                  .------------.     
                  | address    |     .------------.
message---------->| of         |---->|  "Bar"     |
                  | "Bar"      |     '------------'  
                  '------------'

If we use ByRef:

    Sub Sub1()
        Dim s As String = "Foo"
        Sub2(s)
    End Sub

    Sub Sub2(ByRef message As String)
        ' A
        message = "Bar"
        ' B
    End Sub

Then we are passing the same reference. At A:

variable name          Stack             Heap
                         
                         
s------------------------V
                  .------------.     
                  | address    |     .------------.
                  | of         |---->|  "Foo"     |
                  | "Foo"      |     '------------' 
                  '------------'    
message------------------^

Assigning a New String to message will alter the original, at B:

variable name          Stack             Heap
                         
                         
s------------------------V
                  .------------.     
                  | address    |     .------------.
                  | of         |---->|  "Bar"     |
                  | "Bar"      |     '------------' 
                  '------------'    
message------------------^

[PrOpHeT]

Ok, I see my typo, it was a test for the greater question, which though I had seemed to prove to myself what I was thinking, was a flawed test (maybe I typed what I wanted to see subconsciously )

I understand the difference where they are stored. and the only reason I have asked what should have been a simple question that I thought I understood is that I recently redid how I handle some of my events. I am inheriting eventargs and using eventhandler(of) instead of straight out declaring events. when I do this my reference to the raising class (sender as object) is passed ByVal not by ref as it was previously when I declared my events myself.

This led to the question and ultimately the above example which though I obviously made a mistake in expressing my question, still leaves me with the question I originally had.

I had used a string in the example because it was my understanding that a string is a reference type in .net. when a reference type, say an instance of a class that has a .DoWhatever method is passed by an event as (ByVal sender as object) it is indeed passing a reference to that object is it not?

Because DirectCast(Sender,MyObject).DoWhatever calls DoWhatever on the instance I passed (or at least appears to) so in that situation why is it passed ByVal (Done by the IDE, not me) and what would the difference be if it had been passed ByRef?

Let me use a more simple example (I'll test this one first)

    Private Sub Form1_MouseMove(ByVal sender As Object, ByVal e As System.Windows.Forms.MouseEventArgs) Handles Me.MouseMove
        DirectCast(sender, Form).Text = e.X & "/" & e.Y
    End Sub

This is what I was trying to create an example of, sender is passed ByVal, yet obviously is a pointer to an object because my instance of form1 gets its text altered.

If it is not a pointer how is this working, if it is why is it labeled as being passed ByVal? Or am I yet again missing something blatantly obvious?

Please bear with me here, I have been using this syntax for years and had always used ByRef and ByVal with my understanding of what they were without problem therefore I never looked closer at how the language internally was using them, and never really noticed how things like form events were being passed only casting the object and going on with life. Now that difference has made me question what I thought I understood.

[AtmaWeapon]

(I typed this post before seeing PrOpHeT's last post.)

jo0ls did an excellent job explaining. I explained it back in February too, but now I'm adding jo0ls' post to the FAQ page in my signature because the pictures speak louder than words.

You should read the post I linked, but I'm making an explanation that uses a different analogy using some other languages as examples. Stay tuned.

[AtmaWeapon]

The confusion you're experiencing is why I believe everyone should study their programming language carefully, and also why I feel like everyone should learn C/C++. The answer lies in how the language stores your variables, and how these variables are passed around.

First, I'm not a master of C/C++, so I'm sure someone can jump in and point out some places where I use "never" or "always" and I'm not correct. Additionally, I'm using some of C#'s syntax in order to save some lines of code, don't get pedantic with me about syntax. I'll probably forget to write C/C++ in a few cases and refer to it as just C++, but I believe the concepts I'm discussing are the same in both. I'm glossing over some of the technical details in order to illustrate a point.

In C/C++, everything is passed by value. No matter what the variable type, when you pass it to a function a copy will be made. There's some interesting consequences of this decision but they are unimportant to this discussion.

In C/C++, there is a concept of a pointer variable, and this is the key to passing things by reference. A pointer variable does not store a value; it stores the address of some memory that stores a value. So, for example, the following code declares an integer and stores it in a variable named intVal:
int intVal = 10;

In this case, 32 bits of memory are set aside to store the value 10, and the internal structure that manages variables understands that intVal corresponds to the 32 bits at this location. The following declares a pointer variable that can point to an integer:
int * pInt;

In this case, 32 bits of memory are set aside, but no value is stored in the memory. When you try to access pInt, you'll get an error from any compiler worth its salt because you haven't assigned an address to it yet. We can make pInt point to intVal with this syntax:
pInt = &intVal

In C/C++, & is the AddressOf operator; it returns a value that represents the location in memory of that variable. So let's say you combined the lines above into a program and wanted to print intVal using pInt:

int intVal = 10;
int * pInt = &intVal;
Console::WriteLine(pInt);

Your output will be apparently random, you'll likely see a value like -2172830. Why? Pointer variables store addresses, not values. The number that you see is actually the memory address of intVal; if you want to get the value that pInt points to, you have to use a special syntax to dereference it:
Console::WriteLine(*(pInt));

When you dereference a pointer variable, you tell the compiler to get the address from the pointer, then get the value of the memory at that address. So the above line of code would print 10. Pointers are how you pass things by reference in C/C++:
void ChangeInt(int * value)

You would call this function by passing it the address of an integer:
ChangeInt(&intVal); -OR- ChangeInt(pInt);

But wait, I said everything is passed by value in C++, what gives? Well, what happens when you call this function is the following:

1. The program allocates memory for a new pointer variable.
2. The address that is passed to the function is copied into the new variable.

So, what happens is you copy the pointer, not the actual variable, so when you work with the dereferenced pointer you are actually changing the variable outside the scope of the function.

So, why did I just spend a page discussing C++ on a VB forum? Get ready, this is very important to understand what's going on. I'm done discussing C++ syntax and internals though; the rest will be with respect to VB.

Despite what you might think, VB .NET uses pointers. They are hidden from you in almost all respects, but once you understand how pointers work you can see where the language is hiding pointers from you. When you pass something ByVal, a copy of the variable is passed to the function. When you pass something ByRef, a pointer to that variable is passed to the function. It'd be great if the abstraction of pointers ended there, but there's another case where you need to know what's going on under the covers.

Value-type variables behave exactly like non-pointer variables in C++. It's not even worth discussing, just read how C++ variables work and you have value types in .NET. This covers all primitive types such as Integer, and all Structure types as well.

Reference types behave exactly like pointer variables in C/C++, and this is why when you pass reference types ByVal you can still change the object: when a reference type is passed ByVal, .NET copies the pointer and passes that to the method.

Two interesting gotchas that are related:

1. ReadOnly properties that return reference types behave the same way as ByVal. If you really do not want the user to be able to modify the class instance that your type has, first make a copy of the instance, then return that from the ReadOnly property.
2. Arrays are always reference types. This is why one of the .NET design guidelines suggests never returning arrays from a property; there's no way you can prevent the user from changing the array.

[PrOpHeT]

Ok, I have read all three and I get this from it

Disregarding strings which are different...

ByVal when referring to a value type passes a copy not the actual object.
ByVal when referring to a reference type passes a copy of the pointer to the object? (So ByVal effectively becomes a ByRef anyway) <- this is the one that was confusing me.

ByRef when referring to a value type passes the pointer to the object.
ByRef when referring to a reference type passes a the pointer to the object?

So using ByVal or ByRef with a reference type always yields the pointer and will function the same either way?

If so that would lead me to question why would ByVal be the seemingly preferred method of passing it considering it is going by reference anyway

And providing all of this is indeed correct are there any benefits or drawbacks to using the copied pointer with a ByVal rather than THE pointer with a ByRef?

[AtmaWeapon]

This one's fun to discuss as well.

First, ByVal and ByRef represent a contract between the person that is calling the function (caller) and the person that wrote the function (developer). When the caller sees that a parameter is passed ByVal, they should be assured that the developer will NOT alter the state of the object that is passed in any way. When the caller sees that a parameter is ByRef, they are made aware that the developer is indicating that the object will be changed, and if the caller does not want this to happen then the caller must manually create a copy and pass that copy (Some people argue that ByRef isn't clear enough; in C# the caller is required to add out or ref before the parameter to indicate, "Yes, I understand this variable can be changed.) Likewise, when the developer is writing the method and specifies a ByVal parameter, the developer is agreeing that he will do nothing that alters the state of that parameter. The fact that this contract is not enforced by the compiler is woeful, but the next point is the reason why.

Passing variables by value is a performance risk. For primitive types it's no big deal, since they are small. But what if the object that's being passed contains a large amount of data, such as a Bitmap? If reference-types were always passed by value, then any method with a Bitmap parameter would cause an entirely new copy of the bitmap's data to be made every time the method was called. This would waste memory and time for the creation of what's basically a short-lived temporary variable. This is why reference-types are handled the way they are: the address is generally small compared to the size of the object, so it's more efficient to copy that instead. In C/C++, this is usually addressed by using pointer-type parameters even if you don't intend to modify the parameter, since it's understood that it's done for performance reasons. This is one reason why MSDN documentation has to label parameters as "[In]" or "[Out]"; sometimes a function uses pass-by-reference but does not alter the variable.

A consequence of this information is that you should rarely, if ever, define your own Structures. Structures are value types, and thus when passed ByVal they are copied. If your structure is larger than the size of an address on your system, you're going to be using extra memory at every method call, not to mention extra time to produce the copy. If the Structure has reference type members, they can't be copied, so their pointer is copied. Now you have two different value types with some members unique but others linked! This is confusing. Basically Structures don't have much value compared to classes outside of PInvoke scenarios.

Nitpicker's Corner
Technically, when you pass a reference type ByRef, what you pass is the address of the pointer to the object, that is, a pointer to a pointer. In C/C++, these "double pointers" are not uncommon but are still a pain to handle since they must be dereferenced twice. Let's play with it since I like pain:

int intVal = 10;
int * pIntVal = &intVal;
int ** ppIntVal = &pIntVal;

Console::WriteLine((*(*(pIntVal))));

Of course, this is slightly contrived, but when you become aware that arrays in C/C++ are pointers, the possibility of a double pointer becomes more sane. .NET is smart enough to determine that the reference type variable passed ByRef is a double pointer and handles it without you having to do any work.

Part of the feature I support has a few methods that return a pointer to a value where the address of an allocated handle resides; this is a pointer to a pointer to a pointer, the rare and dreaded "triple pointer". Don't ask.

[PrOpHeT]

Ok, since I am both learning and enjoying this conversation. I have to ask.

If it is standard accepted practice that by passing something ByVal means you should not change the state of the object passed then is it bad form to do something like the example I posted for the form text?

The sender is the object reference to Form1, I altered its state by settings its text property.

And if this is bad form, how should I have coded that scenario?

[AtmaWeapon]

Actually, it's a little different since it's an event handler. Event handlers aren't called by people, they are called by Windows in response to certain things happening. Because of this, the context is different.

If it were a normal method, I'd say make the first parameter ByRef and you're done. Since it's an event handler, sender has to be ByVal. However, in the context of event handlers, there's no real expectation that you'll leave the sender alone. In general, EventArgs parameters have ReadOnly properties, so unexpected change can't happen there; the common exception to this rule is the cancel-style events like Form.FormClosing, where you are expected to work with the Cancel property of the event args.

So, in the context of event handlers, it's OK to modify the object that is sent via the sender parameter. In other circumstances, it's best to make that parameter ByRef if you're going to change it.

[PrOpHeT]

Ok, enlightening to say the least. Are there any specific resources you suggest reading on this matter? Specifically one that explains some of these minutia such as when you should or should not do things it will let you do either way?

It all sounds like the i before e except after c or when sounding as a...

I have been trying to do more of my coding by accepted standards and less by "works".

[AtmaWeapon]

The best printed resource is the Framework Design Guidelines book; a 2nd edition seems to be scheduled for February 2009 but I don't trust Amazon's release dates. I'm pretty sure it's just a printed version of the MSDN Content, but it is nice because it's annotated with comments from some of the .NET dev team and various celebrity .NET developers. I also recommend Jeffery Richter's CLR via C#, as it reveals some neat internals of the framework that help you understand some of the bigger picture.

The rest of it is experience and spending an hour or so a day devoted to reading documentation or blog articles about coding. I'm honestly not that knowledgeable outside of .NET but I do try to find articles that I don't understand and try to work them out. Learn more than one language and you find that all of them teach you something. For example, I've never seen a book that explained value types and reference types with anything other than some stock diagrams that MS probably provided, but my experience with C/C++ made me immediately think "reference type = pointer" when I first encountered it.

[PrOpHeT]

Cool, reading the MSDN version now, ordered the book.

Thanks to you both for your input and clarification.

[Merrion]

I'm only going to make this worse because I've had the best part of a bottle of wine (the liquid contents) but...

Basically anything derived from object doesn't have a value...consider a Form - if I pass it ByVal we can't go and create a whole new copy form with all its properties because, for example, the hwnd has to be unique.

So - if it isn't a value type it can't be passed ByVal (in the VB6 sense of the word).

This example from a java page(?) makes sense of things...

Imports System

public class MainClass
   Shared Sub Main()
        Dim A As New MyObject()
        Dim B As MyObject = A
        A.X = 1
        Console.WriteLine("Initial state")
        Console.WriteLine("Are A and B the same? " + (A Is B).ToString())
        Console.WriteLine("A.x: " + A.X.ToString() + " B.x " + B.X.ToString())
        FunctionPassObjectByReference1(B)
        Console.WriteLine("After FobjByRef1")
        Console.WriteLine("Are A and B the same? " + (A Is B).ToString())
        Console.WriteLine("A.x: " + A.X.ToString() + " B.x " + B.X.ToString())
        A.X = 1
        B = A
        FunctionPassObjectByReference2(B)
        Console.WriteLine("After FobjByRef2")
        Console.WriteLine("Are A and B the same? " + (A Is B).ToString())
        Console.WriteLine("A.x: " + A.X.ToString() + " B.x " + B.X.ToString())
        A.X = 1
        B = A
        FunctionPassObjectByValue1(B)
        Console.WriteLine("After FobjByVal1")
        Console.WriteLine("Are A and B the same? " + (A Is B).ToString())
        Console.WriteLine("A.x: " + A.X.ToString() + " B.x " + B.X.ToString())
        A.X = 1
        B = A
        FunctionPassObjectByValue2(B)
        Console.WriteLine("After FobjByVal2")
        Console.WriteLine("Are A and B the same? " + (A Is B).ToString())
        Console.WriteLine("A.x: " + A.X.ToString() + " B.x " + B.X.ToString())
        
   End Sub

   Shared Public Sub FunctionPassObjectByReference1(ByRef Y As MyObject)
        Y.X = 5
    End Sub

   Shared Public Sub FunctionPassObjectByReference2(ByRef Y As MyObject)
        Y = New MyObject()
        Y.X = 5
    End Sub

   Shared Public Sub FunctionPassObjectByValue1(ByVal Y As MyObject)
        Y.X = 5
    End Sub

   Shared Public Sub FunctionPassObjectByValue2(ByVal Y As MyObject)
        Y = New MyObject()
        Y.X = 5
    End Sub

    Class MyObject
        Public X As Integer
    End Class

End Class

[PrOpHeT]

Hence the confusion, because by default this is how vb.net does pass them, it appears my understanding of ByVal and ByRef was in general correct, however my understanding of what that meant in the scope of the language and how it used them was generally flawed.

[Cas]

I don't know if you'll find this useful, it's an example that I came across the other day when trying to understand why an object unexpectedly went out of scope while I was sure I still had a valid reference to it.
It's VB6, but the important part should be identical in .Net, and it's somewhat contrived, but drives the point home all the better for it IMO.

Public Sub subCaller()
    Dim o As Object: Set o = 'some object
    Call subCallee(o, o, o)
End Sub

Public Sub subCallee(ByVal o1 As Object, ByRef o2 As Object, ByRef o3 As Object)
    
    Debug.Print (o1 Is Nothing) & "," & (o2 Is Nothing) & "," & (o3 Is Nothing) 'False,False,False
    Set o1 = Nothing
    Debug.Print (o1 Is Nothing) & "," & (o2 Is Nothing) & "," & (o3 Is Nothing) 'True,False,False
    Set o2 = Nothing
    Debug.Print (o1 Is Nothing) & "," & (o2 Is Nothing) & "," & (o3 Is Nothing) 'True,True,True
    
End Sub

[darkforcesjedi]

On the same topic, what really happens when you set an object to nothing?

Dim o, p As [SomeType]
o = New [huge object] ' A
p = New [huge object] ' B
o = p
o = Nothing

What actually happens to the new [huge object]s A and B when this code executes? Does "o = Nothing" delete B? How does "o = Nothing" affect p? Is p a null reference, or does it point to an object that doesn't exist? What happens if I change o's reference without clearing it ("o = p")?

[AtmaWeapon]

This is a garbage collector topic.

Objects are allocated on the heap, and the heap is maintained by the garbage collector. When you set a variable to Nothing, it's roughly equivalent to setting a pointer to null in C/C++: the object is still there and the variable doesn't reference anything. The difference in .NET is that is that this isn't a memory leak.

At certain intervals, the GC will run. It examines every object in the heap and tries to determine if any variables reference this object. If no accessible variables access the object, then the object is marked for collection and the garbage collector will free its memory.

This is why the Dispose pattern exists. There's no way to determine when the GC will run, so if your class is consuming a lot of resources it's going to continue using them for an indeterminate period of time. Implementing the Dispose pattern allows you to provide a way to dispose of the expensive resources; in general it's best for unmanaged resources since if your class references huge managed resources those are subject to garbage collection too.

The situation where you set o = p without setting o to Nothing first is similar. Now o and p point to the same object, and the object that o originally pointed to will remain in the heap until it is garbage collected.

[darkforcesjedi]

So at the end of execution, p still references the second object? What if I call GC.Collect()? Will the second object be collected?

[Cas]

Just think of object variables as the conceptual equivalent of API handles:

Dim o As Long, p As Long
o = CreateMenu
p = CreateMenu
o = p
o = 0

Nothing you do to the variable holding the handle value will have any effect on the entity to which the handle refers. To create and destroy the entities themselves, a different type of functionality is required, such as keywords (New), garbage collection, Create* and Delete* functions, etc.

[AtmaWeapon]

Originally Posted by darkforcesjedi

So at the end of execution, p still references the second object? What if I call GC.Collect()? Will the second object be collected?

No. So long as there is an in-scope variable that accesses the object, the GC will not collect it. To make clear what I mean by "in-scope", consider this code:

  1    Private _global As ReallyBigObject
  2    
  3    Public Sub Main
  4        Dim obj As new ReallyBigObject()
  5        Dim obj2 As ReallyBigObject = obj
  6        obj = New ReallyBigObject()
  7        
  8        ProcessObject(obj2)
  9        
 10        obj2 = Nothing
 11        GC.Collect()
 12    End Sub
 13    
 14    Sub ProcessObject(ByVal obj As ReallyBigObject)
 15        Dim obj3 As ReallyBigObject = obj
 16        Dim obj4 As New ReallyBigObject()
 17        Dim obj5 As New ReallyBigObject()
 18        _global = obj4
 19    End Sub

Let's walk through it line-by-line and discuss what happens.

On line 4, a new ReallyBigObject (1) is created on the heap, and obj points to it. On line 5, obj2 is created and made to point at the same object as obj; we have two variables and one object.

On line 6, a new ReallyBigObject (2) is created on the heap and obj is made to point at it. Now obj points at (2) and obj2 points at (1); we have two variables and two objects. If the GC runs at this point, nothing will be collected because the two objects are referenced by variables.

On line 8, ProcessObject is called, passing obj2 (1) as a parameter.

On line 15, obj3 is made to point at (1) (so now (1) has two variables pointing at it). On line 16, we create a new ReallyBigObject (3) and obj4 points at it. On line 17, we create a new ReallyBigObject (4) and make obj5 point to it. On line 18, we make _global point at the same thing as obj4 (3).

So, let's make a tally here as of line 19, before we exit ProcessObject. (1) is referenced by obj2 and obj3. (2) is referenced by obj. (3) is referenced by obj4 and _global. (4) is referenced by obj5.

After we exit ProcessObject and return to line 8, obj3, obj4, and obj5 are said to be "out of scope", since they only exist inside the ProcessObject. They no longer count to the GC, so our tally above needs to be adjusted:
(1) is referenced by obj2. (2) is referenced by obj. (3) is referenced by _global. (4) is no longer referenced by anything. At this point, if the GC runs, only (4) is eligible for collection.

On line 10, we make obj2 stop pointing at (1). Now nothing points at (1), so it's eligible for collection.

The GC is told to run on line 11, and (1) and (4) will likely be collected.

What happens if we exit Main (let's assume it was called by something else)? All of Main's variables will be out of scope, so (2) will no longer have obj referencing it. Since _global is always in scope as long as the class exists, it still references (3). After we exit Main the objects (1), (2), and (4) are eligible for garbage collection.

The fun final note is that the GC may not collect the objects when it runs. The GC's algorithm does its best to reclaim memory with minimum impact on the system. If it decides the system is CPU-starved, it may decide not to run at all. If the system is memory-starved, it tries harder to find things to collect. Only unmanaged resources can be immediately released since, by their nature, you are in complete control of their lifetime.