How to get full response with TCPClient?
How to get full response with TCPClient?
How to get full response with TCPClient?
How to get full response with TCPClient?
How to get full response with TCPClient?
How to get full response with TCPClient? How to get full response with TCPClient? How to get full response with TCPClient? How to get full response with TCPClient? How to get full response with TCPClient? How to get full response with TCPClient? How to get full response with TCPClient? How to get full response with TCPClient?
How to get full response with TCPClient? How to get full response with TCPClient?
How to get full response with TCPClient?
Go Back  Xtreme Visual Basic Talk > > > How to get full response with TCPClient?


Reply
 
Thread Tools Display Modes
  #1  
Old 06-27-2011, 01:49 PM
ggggggg ggggggg is offline
Newcomer
 
Join Date: Nov 2009
Posts: 2
Default How to get full response with TCPClient?


Hello, I'm having problems retrieving the full source code of a website (it only returns partial source) with TCPClient. My code is below if you could either correct it or lead me on the right path, it would make my day.

Sincerely, a vb.net beginner.

Code:
    Public Function GetHTTP(ByVal URL As String)
        On Error Resume Next

        Dim tcpClient As New System.Net.Sockets.TcpClient()
        Dim TheServer As String = Replace(URL, "http://", "")
        TheServer = GetLeftOf2(TheServer)
        TheServer = Replace(TheServer, "www.", "")
        Dim Packet As String = "GET http://www." & TheServer & GetURL(URL) & " HTTP/1.1" & vbCrLf & _
            "Host: www." & TheServer & vbCrLf & _
            "Accept: text/html" & vbCrLf & _
            "Connection: keep-alive" & vbCrLf & vbCrLf
        Dim Packet2 As String = "GET http://" & TheServer & GetURL(URL) & " HTTP/1.1" & vbCrLf & _
    "Host: www." & TheServer & vbCrLf & _
    "Accept: text/html" & vbCrLf & _
    "Connection: keep-alive" & vbCrLf & vbCrLf


        tcpClient.Connect(TheServer, 80)
        Dim networkStream As Sockets.NetworkStream = tcpClient.GetStream()
        If networkStream.CanWrite And networkStream.CanRead Then
            ' Do a simple write.
            Dim sendBytes As [Byte]() = Encoding.ASCII.GetBytes(Packet)
            networkStream.Write(sendBytes, 0, sendBytes.Length)
            

            ' Read the NetworkStream into a byte buffer.
            Dim bytes(tcpClient.ReceiveBufferSize) As Byte
            networkStream.Read(bytes, 0, CInt(tcpClient.ReceiveBufferSize))
            ' Output the data received from the host to the console.

            Dim returndata As String = Encoding.ASCII.GetString(bytes)

            If InStr(returndata, "301 Moved Permanently") Then
                Dim sendBytes2 As [Byte]() = Encoding.ASCII.GetBytes(Packet2)
                networkStream.Write(sendBytes2, 0, sendBytes2.Length)
                ' Read the NetworkStream into a byte buffer.

                Dim bytes2(tcpClient.ReceiveBufferSize) As Byte
                networkStream.Read(bytes2, 0, CInt(tcpClient.ReceiveBufferSize))
                ' Output the data received from the host to the console.

                Dim returndata2 As String = Encoding.ASCII.GetString(bytes2)
                If returndata2 <> "" Then GetHTTP = returndata2
                Exit Function
            Else
                If returndata <> "" Then GetHTTP = returndata
                Exit Function
            End If
        Else
            If Not networkStream.CanRead Then
                GetHTTP = "Invalid"
                tcpClient.Close()
            Else
                If Not networkStream.CanWrite Then
                    GetHTTP = "Invalid"
                    tcpClient.Close()
                End If
            End If
        End If
    End Function
Reply With Quote
  #2  
Old 06-27-2011, 04:07 PM
PlausiblyDamp's Avatar
PlausiblyDampHow to get full response with TCPClient? PlausiblyDamp is offline
Ultimate Contributor

Forum Leader
* Expert *
 
Join Date: Nov 2003
Location: Newport, Wales
Posts: 2,058
Default

One problem you have is
Code:
On Error Resume Next
, if anything is going wrong you are simply ignoring the problem and throwing away any error information.

On Error Resume next is really a throwback to VB6 (and was considered poor practice then), you really want to look at exception handling if you are writing a .Net application as that is the .Net way to do things.
__________________
Intellectuals solve problems; geniuses prevent them.
-- Albert Einstein

Posting Guidelines Forum Rules Use the code tags
Reply With Quote
  #3  
Old 06-28-2011, 03:31 AM
DrPunk's Avatar
DrPunkHow to get full response with TCPClient? DrPunk is offline
Senior Contributor

* Expert *
 
Join Date: Apr 2003
Location: Never where I want to be
Posts: 1,403
Default

Any particular reason you're using TCPClient for this?

Because there are WebRequest objects that make communicating with web servers much easier.

There's an example here -> http://msdn.microsoft.com/en-us/libr...v=VS.100).aspx
__________________
There are no computers in heaven!
Reply With Quote
  #4  
Old 06-28-2011, 09:57 AM
AtmaWeapon's Avatar
AtmaWeaponHow to get full response with TCPClient? AtmaWeapon is offline
Fabulous Florist

Forum Leader
* Guru *
 
Join Date: Feb 2004
Location: Austin, TX
Posts: 9,500
Default

NetworkStream.Read() only reads up to the size of its receive buffer, possibly less. You're expected to check the number of bytes it says it read and call it again if you need more data. How do you know if there should be more? That's why there's a Content-Length header in HTTP. If you don't have that, you have to take a less reliable approach: keep reading until nothing is read. There's plenty of examples in the documentation.
__________________
.NET Resources
My FAQ threads | Tutor's Corner | Code Library
I would bet money 2/3 of .NET questions are already answered in one of these three places.
Reply With Quote
  #5  
Old 06-28-2011, 03:50 PM
ggggggg ggggggg is offline
Newcomer
 
Join Date: Nov 2009
Posts: 2
Default

Quote:
Originally Posted by AtmaWeapon View Post
NetworkStream.Read() only reads up to the size of its receive buffer, possibly less. You're expected to check the number of bytes it says it read and call it again if you need more data. How do you know if there should be more? That's why there's a Content-Length header in HTTP. If you don't have that, you have to take a less reliable approach: keep reading until nothing is read. There's plenty of examples in the documentation.
Since half the sites I'm connecting to don't provide the content-length header I've decided to keep reading until nothing is read, but it just locks the app up.

Here's the code I used:
Code:
            Dim bytes(tcpClient.ReceiveBufferSize) As Byte
            Dim returndata As String
            While networkStream.CanRead
                Threading.Thread.Sleep(100)
                networkStream.Read(bytes, 0, CInt(tcpClient.ReceiveBufferSize))
                returndata += Encoding.ASCII.GetString(bytes)
            End While
What am I doing wrong?

@DrPunk, because Sockets seem to return the data faster than HTTPWebRequest.
Reply With Quote
  #6  
Old 06-28-2011, 04:04 PM
AtmaWeapon's Avatar
AtmaWeaponHow to get full response with TCPClient? AtmaWeapon is offline
Fabulous Florist

Forum Leader
* Guru *
 
Join Date: Feb 2004
Location: Austin, TX
Posts: 9,500
Default

Ugh that's right, Read()'s probably blocking. Brain was off for a second. In that case I think the solution is either use WebClient or bang on the people who make the websites to provide a content-length header. Under the covers, WebClient's using HttpWebRequest, which probably peeks at the TCP headers to get some idea of when everything's over. You could probably do it with sockets, but I doubt it'd be worth it. I doubt there's a big enough difference in speed to matter.
__________________
.NET Resources
My FAQ threads | Tutor's Corner | Code Library
I would bet money 2/3 of .NET questions are already answered in one of these three places.
Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Forum Jump

Advertisement:





Free Publications
The ASP.NET 2.0 Anthology
101 Essential Tips, Tricks & Hacks - Free 156 Page Preview. Learn the most practical features and best approaches for ASP.NET.
subscribe
Programmers Heaven C# School Book -Free 338 Page eBook
The Programmers Heaven C# School book covers the .NET framework and the C# language.
subscribe
Build Your Own ASP.NET 3.5 Web Site Using C# & VB, 3rd Edition - Free 219 Page Preview!
This comprehensive step-by-step guide will help get your database-driven ASP.NET web site up and running in no time..
subscribe
How to get full response with TCPClient?
How to get full response with TCPClient?
How to get full response with TCPClient? How to get full response with TCPClient?
How to get full response with TCPClient?
How to get full response with TCPClient?
How to get full response with TCPClient? How to get full response with TCPClient? How to get full response with TCPClient? How to get full response with TCPClient? How to get full response with TCPClient? How to get full response with TCPClient? How to get full response with TCPClient?
How to get full response with TCPClient?
How to get full response with TCPClient?
 
How to get full response with TCPClient?
How to get full response with TCPClient?
 
-->