Extracting data from website using VBA


"Excel VBA Help"

 


Play Friv Now - Play Friv Games
Go Back   The Code Cage Forums > Microsoft Topics > Microsoft Excel Forum
Our Info Pages:-
The Code Cage - Microsoft Office help

Notices

Microsoft Excel Forum Members Post questions in this forum if they are related to using Microsoft Excel, this includes Worksheet Functions, Excel VBA Programming, Simple Macro Code & Charting, Excel formula etc.

New Reply
Thread Tools Rate Thread Display Modes
  #1  
Unread 27th March 2012, 17:57
Cutter's Avatar
Cutter Cutter is offline
Junior Member
MS Office Version: Excel 2010
MS Office Skill Level: Can use Formulae
 
Join Date: Mar 2012
Location: Ontario, Canada
Age: 62
Posts: 7
Posts Per Day: 0.01
My Top Tip Count: 0
Thanks to others: 3
Thanked 0 Times in 0 Posts
Reputation: 10 (3) Cutter is on a distinguished road
Canada
Default Extracting data from website using VBA
     Excel: VBA   


------ Register to get rid of these "In Post" ads! ------

Category: Excel: VBA   

Hi Folks
My second attempt to start this thread. Last night I kept receiving message to remove links even though there weren't any.
On to the problem - I attempted to follow the instructions given in a YouTube clip demonstrating how to extract specific data from a web site. The demo was for a site that provides the city, county and state when a zip code is inserted. Although I have no interest in that example I was hoping to be able to adapt the method for other sites. But of course the macro failed and I can't resolve it. VBA is not my forte!
I can't upload the file (forum rules re my noobiness here) nor can I provide the link to the clip (same forum rule). I'm hoping someone can spot an error in the code below:

VBA CODE:
Option Explicit 
 
Private Sub Worksheet_Change(ByVal Target As Range) 
    If Target.Row = Range("Zip_Code").Row And Target.Column = Range("Zip_Code").Column Then 
        Dim IE As New InternetExplorer 
         'IE.Visible = True
        IE.navigate "http://quickfacts.census.gov/cgi-bin/qfd/lookup?place=" & Range("Zip_Code").Value 
        Do 
            DoEvents 
        Loop Until IE.readyState = READYSTATE_COMPLETE 
        Dim doc As HTMLDocument 
        Set doc = IE.document 
        Dim sdd As String 
        sdd = Trim(doc.getElementsByTagName("dd")(1).innerText) 
        IE.Quit 
         'MsgBox sdd
        Dim add As Variant 
        add = Split(sdd, ", ") 
        Range("County").Value = add(1) 
        Range("City").Value = add(0) 
        Range("State").Value = add(2) 
    End If 
End Sub 

The macro throws this error on the highlighted line:
Run-time error '91':
Object variable or With block variable not set

Thanks for looking
Bill

The Code Cage Advertisment
Advertisement
  #2  
Unread 27th March 2012, 18:01
Simon Lloyd's Avatar
Simon Lloyd Simon Lloyd is offline
Administrator
MS Office Version: MS Office 97 - 2013
MS Office Skill Level: Can use most of MS Office
 
Join Date: Aug 2008
Location: Manchester, England
Age: 50
Posts: 11,684
Posts Per Day: 5.02
My Top Tip Count: 11
Thanks to others: 41
Thanked 427 Times in 412 Posts
Reputation: 6454 (187) Simon Lloyd has a reputation beyond reputeSimon Lloyd has a reputation beyond reputeSimon Lloyd has a reputation beyond reputeSimon Lloyd has a reputation beyond reputeSimon Lloyd has a reputation beyond reputeSimon Lloyd has a reputation beyond reputeSimon Lloyd has a reputation beyond reputeSimon Lloyd has a reputation beyond reputeSimon Lloyd has a reputation beyond reputeSimon Lloyd has a reputation beyond reputeSimon Lloyd has a reputation beyond repute
11 Top Tips
Wales
Default Re: Extracting data from website using VBA

------ Register to get rid of these "In Post" ads! ------


Cutter, you can upload a file!, you have forgotten that you have supplied url's in the vba code, this is what was preventing you from posting not the attachment
__________________
Regards,
Simon Lloyd
Microsoft Office Help

Want to say thanks? then hit this How to say thanks! button in any post you want to thank
If you find one of my posts helpful then hit this one Helpful Answer in the post that helped most!



The Following User Says Thank You to Simon Lloyd For This Useful Post:
Cutter (27th March 2012)
  #3  
Unread 27th March 2012, 18:05
norie's Avatar
norie norie is offline
Solution Centre Professional
MS Office Version: MS Office 2000/2010
MS Office Skill Level: Proficient with MS Office
 
Join Date: Aug 2009
Location: Yaren
Posts: 226
Posts Per Day: 0.12
My Top Tip Count: 0
Thanks to others: 0
Thanked 20 Times in 20 Posts
Reputation: 339 (11) norie is a jewel in the roughnorie is a jewel in the roughnorie is a jewel in the roughnorie is a jewel in the rough
Mexico
Default Re: Extracting data from website using VBA

------ Register to get rid of these "In Post" ads! ------


What is it you are trying to extract from the website?

I tried the URL with a city name, Albuquerque, instead of a zip and there were no 'dd' elements on the page.

Are you trying to get the footnotes below the table, or is it something else?

The Code Cage Advertisment
Advertisement
  #4  
Unread 27th March 2012, 18:21
Cutter's Avatar
Cutter Cutter is offline
Junior Member
MS Office Version: Excel 2010
MS Office Skill Level: Can use Formulae
 
Join Date: Mar 2012
Location: Ontario, Canada
Age: 62
Posts: 7
Posts Per Day: 0.01
My Top Tip Count: 0
Thanks to others: 3
Thanked 0 Times in 0 Posts
Reputation: 10 (3) Cutter is on a distinguished road
Canada
Default Re: Extracting data from website using VBA

------ Register to get rid of these "In Post" ads! ------


@Simon
You're right - I overlooked the url within the code. I'll attach the file now.

@Norie
Hopefully the file will explain it all. Thanks for looking.
Attached Files To view attachments in this forum your post count must be 2 or greater. You currently have 0 posts.

  #5  
Unread 27th March 2012, 18:35
norie's Avatar
norie norie is offline
Solution Centre Professional
MS Office Version: MS Office 2000/2010
MS Office Skill Level: Proficient with MS Office
 
Join Date: Aug 2009
Location: Yaren
Posts: 226
Posts Per Day: 0.12
My Top Tip Count: 0
Thanks to others: 0
Thanked 20 Times in 20 Posts
Reputation: 339 (11) norie is a jewel in the roughnorie is a jewel in the roughnorie is a jewel in the roughnorie is a jewel in the rough
Mexico
Default Re: Extracting data from website using VBA

------ Register to get rid of these "In Post" ads! ------


Change 1 to 0 - collections of elements are zero-based.

  #6  
Unread 27th March 2012, 19:39
Cutter's Avatar
Cutter Cutter is offline
Junior Member
MS Office Version: Excel 2010
MS Office Skill Level: Can use Formulae
 
Join Date: Mar 2012
Location: Ontario, Canada
Age: 62
Posts: 7
Posts Per Day: 0.01
My Top Tip Count: 0
Thanks to others: 3
Thanked 0 Times in 0 Posts
Reputation: 10 (3) Cutter is on a distinguished road
Canada
Default Re: Extracting data from website using VBA

------ Register to get rid of these "In Post" ads! ------


If you are referring to the (1) in the error line it's a 1 because it wants the second occurrence of "dd". If I change it to a 0 it will get the first occurrence which is not what is wanted.
This is described at the 5 minute mark of the YouTube clip.

Thanks

The Code Cage Advertisment
Advertisement

To stop seeing these ads and get other benefits check This page!

  #7  
Unread 27th March 2012, 20:07
norie's Avatar
norie norie is offline
Solution Centre Professional
MS Office Version: MS Office 2000/2010
MS Office Skill Level: Proficient with MS Office
 
Join Date: Aug 2009
Location: Yaren
Posts: 226
Posts Per Day: 0.12
My Top Tip Count: 0
Thanks to others: 0
Thanked 20 Times in 20 Posts
Reputation: 339 (11) norie is a jewel in the roughnorie is a jewel in the roughnorie is a jewel in the roughnorie is a jewel in the rough
Mexico
Default Re: Extracting data from website using VBA

------ Register to get rid of these "In Post" ads! ------


There is only one dd element for the zip code in the file and the one on the video.

That's why you get the error when you try to refer to a second one.



Here's your code with a few changes to show how many dd elements are on the page.

VBA CODE:
Private Sub Worksheet_Change(ByVal Target As Range) 
    If Target.Row = Range("Zip_Code").Row And Target.Column = Range("Zip_Code").Column Then 
         
        Dim IE As New InternetExplorer 
         
         'IE.Visible = True
         
        IE.navigate "http://quickfacts.census.gov/cgi-bin/qfd/lookup?place=" & Range("Zip_Code").Value 
         
        Do 
            DoEvents 
        Loop Until IE.readyState = READYSTATE_COMPLETE 
         
        Dim doc As HTMLDocument 
         
        Set doc = IE.document 
         
        Dim sdd As String 
         
        Dim dd As Object 
         
        Set dd = doc.getElementsByTagName("dd") 
         
         ' dd.Length tells you how many elements there are in the dd collection
         
        MsgBox "There " & IIf(dd.Length = 1, "is ", "are ") & dd.Length & " <dd> element" & IIf(dd.Length = 1, "", "s") & " on this page" 
         
        sdd = Trim(doc.getElementsByTagName("dd")(1).innerText) 
         
        IE.Quit 
         
        Dim add As Variant 
        add = Split(sdd, ", ") 
        Range("County").Value = add(1) 
        Range("City").Value = add(0) 
        Range("State").Value = add(2) 
    End If 
End Sub 

Did you try the code with the change I suggested?

It works for both the zip from the video and the zip from the file.

In fact I just had a look at the comments on the video, at least 2 people mention the same problem you are having and they find changing 1 to 0 worked for them.

PS Couldn't you have posted the link for the video here and explained what data you wanted?

Reply & Quote Reply


1 members found this post helpful.
The Following User Says Thank You to norie For This Useful Post:
Cutter (27th March 2012)
  #8  
Unread 27th March 2012, 20:33
snb's Avatar
snb snb is offline
Forum VIP
MS Office Version: > Office 97
MS Office Skill Level: Interested
 
Join Date: Feb 2011
Location: Europe
Posts: 1,261
Posts Per Day: 0.90
My Top Tip Count: 0
Thanks to others: 0
Thanked 284 Times in 277 Posts
Reputation: 4695 (62) snb has a reputation beyond reputesnb has a reputation beyond reputesnb has a reputation beyond reputesnb has a reputation beyond reputesnb has a reputation beyond reputesnb has a reputation beyond reputesnb has a reputation beyond reputesnb has a reputation beyond reputesnb has a reputation beyond reputesnb has a reputation beyond reputesnb has a reputation beyond repute
Default Re: Extracting data from website using VBA

------ Register to get rid of these "In Post" ads! ------


or simply:

VBA CODE:
Sub tst() 
    With New WinHttpRequest 
        .Open "get", "http://quickfacts.census.gov/cgi-bin/qfd/lookup?place=02119" 
        .send 
        msgbox Split(Split(.responseText, "<dd>")(1), ",")(0) 
    End With 
End Sub 

NB. reference to Winhttpservices. version 5.1

Reply & Quote Reply


1 members found this post helpful.
The Following User Says Thank You to snb For This Useful Post:
Cutter (27th March 2012)
  #9  
Unread 27th March 2012, 21:28
Cutter's Avatar
Cutter Cutter is offline
Junior Member
MS Office Version: Excel 2010
MS Office Skill Level: Can use Formulae
 
Join Date: Mar 2012
Location: Ontario, Canada
Age: 62
Posts: 7
Posts Per Day: 0.01
My Top Tip Count: 0
Thanks to others: 3
Thanked 0 Times in 0 Posts
Reputation: 10 (3) Cutter is on a distinguished road
Canada
Default Re: Extracting data from website using VBA

------ Register to get rid of these "In Post" ads! ------


@norie
I have implemented your code and changed the error producing (1) to a (0). I now see that the html code as seen in the video clip has changed and that is why the macro demonstrated in the clip fails. I didn't think to compare current html code with that shown in the video.
Thanks for getting me back on track.

@snb
Thanks for your code as well. I'll play with it, too.

The Code Cage Advertisment
Advertisement
  #10  
Unread 27th March 2012, 21:46
norie's Avatar
norie norie is offline
Solution Centre Professional
MS Office Version: MS Office 2000/2010
MS Office Skill Level: Proficient with MS Office
 
Join Date: Aug 2009
Location: Yaren
Posts: 226
Posts Per Day: 0.12
My Top Tip Count: 0
Thanks to others: 0
Thanked 20 Times in 20 Posts
Reputation: 339 (11) norie is a jewel in the roughnorie is a jewel in the roughnorie is a jewel in the roughnorie is a jewel in the rough
Mexico
Default Re: Extracting data from website using VBA

------ Register to get rid of these "In Post" ads! ------


Cutter

That's on of the things about doing this sort of thing, pages can be changed.

It might just be a small change which appears to have been done here.

I've seen some pages/sites totally rewritten so any existing code is basically useless and you have to go back to square on, might be able to keep the URL but that's not guaranteed either.

The Code Cage Advertisment
Advertisement
New Reply
The Code Cage at Facebookhe Code Cage at Twitter


Tags
data, extracting, vba, website

Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

The Code Cage Affilliates



http://www.thecodecage.com/forumz tested by McAfee Security http://www.thecodecage.com/forumz tested by McAfee Security

All times are GMT +1. The time now is 00:27.


Powered by vBulletin® Version 3.8.6
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Template-Modifications by TMS
No part of this board, functionality or creatives may be copied or reproduced either in part or full without the express permission of The Code Cage Team.
Copyright ©2008 - , Simon lloyd.
We are not associated with nor employed by Microsoft in any way, we simply provide resources!
All MS office icons are registered trademarks of the application they represent and are used with permission from Microsoft
UA-8135493-1
Cultural Forum | Study at Malaysian University