Vb.net Extract Links From Google-search Using Htmlagilitypack

I have now updated my code as a test I want to list all URLs that has the word index.php but it also displays other things. Here is my working code: Private Sub Button1_Click(send

Solution 1:

I would use Html Agility Pack to extract the links as below

Dim links AsNew List(OfString)()
Dim htmlDoc AsNew HtmlAgilityPack.HtmlDocument()
htmlDoc.LoadHtml(WebSource)
ForEach link As HtmlNode In htmlDoc.DocumentNode.SelectNodes("//a[@href]")
    Dim att As HtmlAttribute = link.Attributes("href")
    If att.Value.Contains("/forums/") Then
        links.Add(att.Value)
    EndIfNext

if it is google search result try something like below

ForEach link As HtmlNode In htmlDoc.DocumentNode.SelectNodes("//cite")
    If link.InnerText.Contains("index.php") Then
        links.Add(link.InnerText)
    EndIfNext