Get "href=link" from html page and navigate to that link using vba

Method ①

Use XHR to make initial request using question homepage URL; apply CSS selector to retrieve links and then pass those links to IE to navigate to

CSS selectors to select elements:

You want the href attribute of the element.You have been given an example already. You can use getAttribute, or, as pointed out by @Santosh, combine an href attribute CSS selector with other CSS selectors to target the elements.

CSS selector:

a.question-hyperlink[href] 

Looks for elements with parent a tag having class of question-hyperlink and an href attribute.

You then apply the CSS selector combination with the querySelectorAll method of document to gather a nodeList of the links.

XHR to get initial list of links:

I would issue this first as an XHR, as much faster, and gather your links into a collection/nodeList you can later loop with your IE browser.

Option Explicit Public Sub GetLinks() Dim sResponse As String, HTML As New HTMLDocument, linkList As Object, i As Long Const BASE_URL As String = "https://stackoverflow.com" With CreateObject("MSXML2.XMLHTTP") .Open "GET", "https://stackoverflow.com/questions", False .send sResponse = StrConv(.responseBody, vbUnicode) End With sResponse = Mid$(sResponse, InStr(1, sResponse, "<!DOCTYPE ")) With HTML .body.innerHTML = sResponse Set linkList = .querySelectorAll("a.question-hyperlink[href]") For i = 0 To linkList.Length - 1 Debug.Print Replace$(linkList.item(i), "about:", BASE_URL) Next i End With 'Code using IE and linkList End Sub 

In the above linkList is a nodeList holding all the matched elements from the homepage i.e. all the hrefs on the question landing page. You can loop the .Length of the nodeList and index into it to retrieve a particular href e.g. linkList.item(i). As the link returned is relative, you need to replace the relative about: part of the path with the protocol + domain i.e. "https://stackoverflow.com".

Now that you have quickly obtained that list, and can access items, you can pass any given updated href onto IE.Navigate.

Navigating to questions using IE and nodeList

For i = 0 To linkList.Length - 1 IE.Navigate Replace$(linkList.item(i).getAttribute("href"), "about:", BASE_URL) Next i 

Method ②

Use XHR to make initial request using GET request and searched for question title; apply CSS selector to retrieve links and then pass those links to IE to navigate to.

Option Explicit Public Sub GetLinks() Dim sResponse As String, HTML As New HTMLDocument, linkList As Object, i As Long Const BASE_URL As String = "https://stackoverflow.com" Const TARGET_QUESTION As String = "How to make the word invisible when it's checked without js" With CreateObject("MSXML2.XMLHTTP") .Open "GET", "https://stackoverflow.com/search?q=" & URLEncode(TARGET_QUESTION), False .send sResponse = StrConv(.responseBody, vbUnicode) End With sResponse = Mid$(sResponse, InStr(1, sResponse, "<!DOCTYPE ")) With HTML .body.innerHTML = sResponse Set linkList = .querySelectorAll("a.question-hyperlink[href]") For i = 0 To linkList.Length - 1 Debug.Print Replace$(linkList.item(i).getAttribute("href"), "about:", BASE_URL) Next i End With If linkList Is Nothing Then Exit Sub 'Code using IE and linkList End Sub 'https://stackoverflow.com/questions/218181/how-can-i-url-encode-a-string-in-excel-vba @Tomalak Public Function URLEncode( _ StringVal As String, _ Optional SpaceAsPlus As Boolean = False _ ) As String Dim StringLen As Long: StringLen = Len(StringVal) If StringLen > 0 Then ReDim result(StringLen) As String Dim i As Long, CharCode As Integer Dim Char As String, Space As String If SpaceAsPlus Then Space = "+" Else Space = "%20" For i = 1 To StringLen Char = Mid$(StringVal, i, 1) CharCode = Asc(Char) Select Case CharCode Case 97 To 122, 65 To 90, 48 To 57, 45, 46, 95, 126 result(i) = Char Case 32 result(i) = Space Case 0 To 15 result(i) = "%0" & Hex(CharCode) Case Else result(i) = "%" & Hex(CharCode) End Select Next i URLEncode = Join(result, "") End If End Function 

1. This If element.className = "question-hyperlink" Then is useless because it is always true because you getElementsByClassName("question-hyperlink") so all elements are definitely of class question-hyperlink. The If statement can be removed.

2. You have each link in the variable element so you don't need the count. Instead of html.getElementsByClassName("question-hyperlink")(count).innerText use element.innerText.

So it should look like this:

Set elements = html.getElementsByClassName("question-hyperlink") Dim erow As Long For Each element In elements erow = Worksheets("Exec").Cells(Rows.count, 1).End(xlUp).Offset(1, 0).Row Worksheets("Exec").Cells(erow, 1) = element.innerText Worksheets("Exec").Cells(erow, 2) = element.GetAttribute("href") 'this should give you the URL Next element