Go Back  Xtreme Visual Basic Talk > Legacy Visual Basic (VB 4/5/6) > VBA / Office Integration > Excel > Excel VBA Internet Explorer Data Extraction


Reply
 
Thread Tools Display Modes
  #1  
Old 01-29-2010, 07:31 PM
Josh Hazel Josh Hazel is offline
Senior Contributor
 
Join Date: May 2008
Posts: 805
Default Excel VBA Internet Explorer Data Extraction


Here is the data from the webpage...
HTML Code:
<td>NAME 1</td>
<td>1111</td>
<td>&nbsp;&nbsp;SGF, V</td>
<td>&nbsp;&nbsp;46</td>
<td>&nbsp;&nbsp;STGA</td>
<td>&nbsp;&nbsp;</td>
</tr>
<tr>
<td>NAME 2</td>
<td>1234</td>
<td>&nbsp;&nbsp;GF, C</td>
<td>&nbsp;&nbsp;57</td>
<td>&nbsp;&nbsp;WR</td>
<td>&nbsp;&nbsp;</td>
</tr>
<tr>
<td>NAME 3</td>
<td>4213</td>
<td>&nbsp;&nbsp;SGF, V</td>
<td>&nbsp;&nbsp;46</td>
<td>&nbsp;&nbsp;SG</td>
<td>&nbsp;&nbsp;</td>
The user will provide the name, for example:
"NAME 1" and it will retrieve the 4 digit number "1111" for me from site
"NAME 2" and it will retrieve "1234"
"NAME 3" and it will retrieve "4321"

Code:
dim...
set ...
With ie
        .navigate "....."
        'What goes here to get that value?
End With
__________________
Josh

If Google = NoHelp Then PostHere = True
Reply With Quote
  #2  
Old 01-30-2010, 10:51 PM
mjclare mjclare is offline
Freshman
 
Join Date: Nov 2007
Posts: 29
Default

I took your HTML snippet and wrapped enough HTML around it to make it a web page and posted it here 'http:\\www.mulberrypatch.com\test1'

This code will open this page and loop through all the cells ("TD" tags) until it finds the one that contains 'NAME 1'. When it finds it, it displays a msgbox that contains the contents of the next cell.

Is this what you are looking for?

Mike Clare

Code:
Sub test1()
  Dim wflag As Boolean
  Dim s2 As String
  Dim i1 As Long
  Dim LOIE As Object
  Set LOIE = CreateObject("internetexplorer.application")
  With LOIE
    .Visible = True
    .navigate "http://www.mulberrypatch.com/test1.htm"
    wflag = lWait(LOIE)
    For i1 = 0 To LOIE.document.all.tags("TD").Length - 1
      s2 = LOIE.document.all.tags("TD").Item(i1).innertext
      If s2 = "NAME 1" Then
        MsgBox LOIE.document.all.tags("TD").Item(i1 + 1).innertext
        Exit For
      End If
    Next i1
  End With
End Sub
Function lWait(LOIE As Object) As Boolean
  Dim t1 As Long
  Dim t2 As Long
  On Error Resume Next
  t1 = Timer
  t2 = t1 + 60
  Do
    If Timer > t2 Then lWait = False: Exit Function
    If VarType(LOIE.document) = vbString Then Exit Do
  Loop
  Do
    If Timer > t2 Then lWait = False: Exit Function
    If LOIE.busy = False Then Exit Do
  Loop
  Do
    If Timer > t2 Then lWait = False: Exit Function
    If VarType(LOIE.document.readystate) = "C" Then Exit Do
  Loop
  Do
    If Timer > t2 Then lWait = False: Exit Function
    If LOIE.document.readystate = "complete" Then Exit Do
  Loop
  lWait = True
End Function
Reply With Quote
  #3  
Old 02-01-2010, 08:39 PM
Josh Hazel Josh Hazel is offline
Senior Contributor
 
Join Date: May 2008
Posts: 805
Default

That works great ... a bit slow, because theres a lot of data on the page but works great... any ideas on how to speed up the process? right now it prol takes about 5 seconds to come back with results (i have modified the code slightly to find anything with a specific string in it in case there are multiple matches - someone might search "beacon" and it will return Beacon St, Beacon NE, Beacon SW)

Code:
    For i1 = 0 To ie2.Document.all.tags("TD").Length - 1
      s2 = ie2.Document.all.tags("TD").Item(i1).innertext
      If InStr(LCase(s2), LCase(strStreetName)) Then
        a = a & vbNewLine & s2 & " - " & ie2.Document.all.tags("TD").Item(i1 + 1).innertext
        'Exit For
      End If
    Next i1

MsgBox "Possible Matches: " & a
Also whats the deal with the function, whats its purpose "function lwait()"
__________________
Josh

If Google = NoHelp Then PostHere = True
Reply With Quote
  #4  
Old 02-02-2010, 02:56 PM
mjclare mjclare is offline
Freshman
 
Join Date: Nov 2007
Posts: 29
Default

Here is a variation on the theme... it is looking at a more complicated and longer table...

The old code would look at each cell in every table in the webpage.

This new code will first examine the raw html code included in each table looking for the target string. If it finds it in the table it will then walk through the table a row at a time and examine the first cell on each row to see if it contains the target string. If it finds the target string it then copies each cell on that row to your spreadsheet.

In theory it should be faster by nature of its design (it doesn't look in every cell). But it should also be faster beacuse I made extensive use of 'with' statements.

Code:
Sub test1()
  Dim s0 As String
  Cells.Delete
  s0 = findit("Person")
End Sub
Function findit(searchtarget As String) As String
  Dim wflag As Boolean
  Dim s2 As String
  Dim i1 As Long
  Dim i2 As Long
  Dim i3 As Long
  Dim i4 As Long
  Dim LOIE As Object
  Set LOIE = CreateObject("internetexplorer.application")
  i4 = ActiveSheet.UsedRange.Rows.Count + 1
  With LOIE
    .Visible = True
    .navigate "http://quickfacts.census.gov/qfd/states/00000.html"
    wflag = lWait(LOIE)
    For i1 = 0 To LOIE.document.all.tags("TABLE").Length - 1
      With .document.all.tags("TABLE").Item(i1)
        If InStr(.outerhtml, searchtarget) > 0 Then
          For i2 = 0 To .all.tags("TR").Length - 1
            With .all.tags("TR").Item(i2).all
              If VarType(.tags("TD").Item(0)) <> 9 Then
                If InStr(.tags("TD").Item(0).innertext, searchtarget) > 0 Then
                  For i3 = 0 To .tags("TD").Length - 1
                    Cells(i4, i3 + 1).Value = .tags("TD").Item(i3).innertext
                    Cells.Columns.AutoFit
                  Next i3
                  i4 = i4 + 1
                End If
              End If
            End With
          Next i2
        End If
      End With
    Next i1
    .Quit
  End With
End Function
The lwait function is just a better way of making sure that the browser has finished painting the web page before it allows the code to continue. I found that a simple test of the busy flag or the readystate property would fail sometimes and cause errors. I also want to make sure that stuff works on platforms from IE6 to IE8 and the differences between them can be subtle.

Mike Clare
Reply With Quote
  #5  
Old 02-02-2010, 04:40 PM
Josh Hazel Josh Hazel is offline
Senior Contributor
 
Join Date: May 2008
Posts: 805
Default

I should give u a little more background info on my project here to see if u might have a possible solution to this ...

I currently have code that does the following:
Creates IE
Navigates to Page
Enters Zip Code in a box
Clicks the search button
-----> Here is the problem, if the zip code is not found - I get a popup box saying zip code not found... and no new window pops open
If the zip code is found a new window pops up, my code waits to grab hold of the new window, then closes the old window
Then your code goes here and it does the search for the street name from the new ie window

If no zip is found it doesnt pop up that new window, it just simply pops up a message box... and does nothing...

You seem quite versed in IE automation ... Im wondering if there is a way for my code to check if ie pops up that "zip cod not found" message box or not, that way i can stick in an Exit Sub if it does

Im currently using a timer to exit sub if the second window doesnt pop up (means that the msgbox did popup)
__________________
Josh

If Google = NoHelp Then PostHere = True

Last edited by Josh Hazel; 02-02-2010 at 08:27 PM.
Reply With Quote
  #6  
Old 02-02-2010, 09:27 PM
mjclare mjclare is offline
Freshman
 
Join Date: Nov 2007
Posts: 29
Default

I am confused... just a bit...

When you say it 'pops up a message box'... what exactly is the 'message box'? Is it a html file that it has retrieved from a remote server? or is it a true windows based msgbox like we would use in VBA?

Calling up a true 'msgbox' implies that the web page is accessing part of the windows API. I suppose it is possible... but it is generally not done. You would probably have to get there by having the user give permission to an activeX module. I have no experience in this area.

But... to track it down I would flip the debugging flag in IE options and then single step through the HTML/javascript code. Watch what happens when you get a negative search. It may set a javascript variable or a hidden field that you can interogate with your VBA code.

Mike Clare
Reply With Quote
  #7  
Old 02-02-2010, 10:02 PM
zyans22 zyans22 is offline
Newcomer
 
Join Date: Apr 2009
Posts: 1
Default Unable to create object of frame in IE

Hi,

I am trying to automate an IE.There I have three frames.right frame, main frame and left frame.I need to select Right frame and input one value.based on that value main frame fields get enabled.Now i need to input values on the main frame.

Please help me out.
Reply With Quote
  #8  
Old 02-03-2010, 12:32 PM
Josh Hazel Josh Hazel is offline
Senior Contributor
 
Join Date: May 2008
Posts: 805
Default

Quote:
Originally Posted by mjclare View Post
I am confused... just a bit...

When you say it 'pops up a message box'... what exactly is the 'message box'? Is it a html file that it has retrieved from a remote server? or is it a true windows based msgbox like we would use in VBA?

Mike Clare
Ahh, yes i may be using the wrong term ... like when i told ppl webpages used java rather than java script =p

Here is the code on the webpage (shortened for pasting)

HTML Code:
if ((zip_code == "123456") ||
	 (zip_code == "23456") ||
	 (zip_code == "45678") ||
	 (zip_code == "56789"))
{
	var pdf_window1 = window.open(fulllink,'pdf_window1');
}
else
{
	alert ("There is NO New York Fire Code information for the zip code you entered: "+zip_code);
__________________
Josh

If Google = NoHelp Then PostHere = True
Reply With Quote
  #9  
Old 02-03-2010, 02:48 PM
Josh Hazel Josh Hazel is offline
Senior Contributor
 
Join Date: May 2008
Posts: 805
Default

I just made a couple of modifications to meet my needs, but your help was instrumental, thank you very much.

The new code you just gave sped it up by 50&#37;, it only takes half the time now thats great!

I just removed the final loop and instead just specified the two items that i needed (street name and zip code) for the return value and sent the data to a listbox on another userform. This code is quite useful and I could probably use it for other tasks in the future, thanks!

Code:
    For i1 = 0 To LOIE.Document.all.tags("TABLE").Length - 1
      With .Document.all.tags("TABLE").Item(i1)
        If InStr(LCase(.outerHTML), LCase(searchtarget)) > 0 Then
          For i2 = 0 To .all.tags("TR").Length - 1
            With .all.tags("TR").Item(i2).all
              If VarType(.tags("TD").Item(0)) <> 9 Then
                If InStr(LCase(.tags("TD").Item(0).innertext), LCase(searchtarget)) > 0 Then
                  'For i3 = 0 To .tags("TD").Length - 1
                    'Cells(i4, i3 + 1).Value = .tags("TD").Item(i3).innertext
                    'Cells.Columns.AutoFit
(1).innertext
                    frmTerr.lstTerr.AddItem .tags("TD").Item(0).innertext & " - " & .tags("TD").Item(1).innertext
                  'Next i3
                  i4 = i4 + 1
                End If
              End If
            End With
          Next i2
        End If
      End With
    Next i1
    .Quit
__________________
Josh

If Google = NoHelp Then PostHere = True

Last edited by Josh Hazel; 02-03-2010 at 04:01 PM.
Reply With Quote
  #10  
Old 02-04-2010, 09:07 PM
mjclare mjclare is offline
Freshman
 
Join Date: Nov 2007
Posts: 29
Default

Josh,

This post continues the discussion about how to deal with the 'alert' message box that pops up...

A while back... I remembered reading a procedure that has a chance of working in your situation. See this thread: (scroll way down to the 'accepted solution')

http://www.experts-exchange.com/Prog..._20510962.html

This technique relies on the fact that when a user is looking at a webpage, the HTML code and any scripts contained within are actually physically residing in the memory address space of the local browser. I am thinking that with this procedure you could replace (on the fly) the javascript function that pops up the alert message with javascript that code that sets an property that your VBA code can test on.

It is not easy or simple and I have not actually tried it... but it should work.

Good luck,

Mike Clare
Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Forum Jump

Advertisement:





Free Publications
The ASP.NET 2.0 Anthology
101 Essential Tips, Tricks & Hacks - Free 156 Page Preview. Learn the most practical features and best approaches for ASP.NET.
subscribe
Programmers Heaven C# School Book -Free 338 Page eBook
The Programmers Heaven C# School book covers the .NET framework and the C# language.
subscribe
Build Your Own ASP.NET 3.5 Web Site Using C# & VB, 3rd Edition - Free 219 Page Preview!
This comprehensive step-by-step guide will help get your database-driven ASP.NET web site up and running in no time..
subscribe
 
 
-->