none
Download all the .bak files RRS feed

  • Question

  • I am trying to download all the .bak files from the webpage using credentials. But sometime, it is working fine and sometimes its not. Below is the script which I am using:

    $Username = 'xxxx'
    $Password = 'yyyyy'

    $webclient = New-Object System.Net.webclient
    $source = "https://somesite.com/"
    $destination = "E:\myfolder"
    #The following line returns the links in the webpage
    $WebClient.Credentials = New-Object System.Net.Networkcredential($Username, $Password)
    $testcode1=$webclient.downloadstring($source) -split "<a\s+" | %{ [void]($_ -match "^href=[`'`"]([^`'`">\s]*.bak)"); $matches[1]}


    foreach ($line in $testcode1) {
        #$Destination = "$destination\$line"
        #Create a new directory if it doesn't exist

        if (!(Test-Path $Destination))
        
        {
            New-Item $Destination -type directory -Force
        }
        
     $WebClient.Credentials = New-Object System.Net.Networkcredential($Username, $Password)  
        $webclient.downloadfile($source + $line , $destination + $line )
    }

    Everytime I am getting the below error while executing.

    Cannot index into a null array.
    At C:\Users\uswmnl09\Desktop\Download\Test.ps1:37 char:118
    + ... ">\s]*.bak)"); $matches[1]}
    +                    ~~~~~~~~~~~
        + CategoryInfo          : InvalidOperation: (:) [], RuntimeException
        + FullyQualifiedErrorId : NullArray
     
    Cannot index into a null array.
    At C:\Users\uswmnl09\Desktop\Download\Test.ps1:37 char:118
    + ... ">\s]*.bak)"); $matches[1]}
    +                    ~~~~~~~~~~~
        + CategoryInfo          : InvalidOperation: (:) [], RuntimeException
        + FullyQualifiedErrorId : NullArray
     
    Cannot index into a null array.
    At C:\Users\uswmnl09\Desktop\Download\Test.ps1:37 char:118
    + ... ">\s]*.bak)"); $matches[1]}
    +                    ~~~~~~~~~~~
        + CategoryInfo          : InvalidOperation: (:) [], RuntimeException
        + FullyQualifiedErrorId : NullArray

    Sometime I am getting the below error:

    Exception calling "DownloadString" with "1" argument(s): "The remote server returned an error: (407) Proxy Authentication Required."
    At C:\Users\uswmnl09\Desktop\Download\Test.ps1:37 char:1
    + $testcode1=$webclient.downloadstring($source) -split "<a\s+" | %{ [void]($_ -mat ...
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
        + FullyQualifiedErrorId : WebException

    Any help on this.

    • Moved by Bill_Stewart Wednesday, May 30, 2018 6:33 PM This is not "fix code to interface with some random unknown web server for me" forum
    Tuesday, March 20, 2018 10:30 PM

All replies

  • The line " $testcode1=$webclient.downloadstring($source) -split "<a\s+" | %{ [void]($_ -mat ..."

    Is not correct. 

    $testcode1=$webclient.downloadstring($source) -split "<a\s+" | %{ [void]($_ -match "^href=[`'`"]([^`'`">\s]*.bak)"); $matches[1]}

    Bet the links collection from the page and you won't have to parse the string.


    \_(ツ)_/

    • Proposed as answer by I.T Delinquent Wednesday, March 21, 2018 9:27 AM
    Tuesday, March 20, 2018 10:52 PM
  • So what will be the solution?

    I am new to powershell and I tried many ways.

    First time when  I run the script, it is giving the below error. When I try to manually login to the site and try to execute, the script is working fine.

    Exception calling "DownloadString" with "1" argument(s): "The remote server returned an error: (407) Proxy Authentication Required."
    At C:\Users\uswmnl09\Desktop\Download\Test.ps1:37 char:1
    + $testcode1=$webclient.downloadstring($source) -split "<a\s+" | %{ [void]($_ -mat ...
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
        + FullyQualifiedErrorId : WebException

    In which section I need to do the changes?

    Wednesday, March 28, 2018 10:24 PM
  • So what will be the solution?

    I am new to powershell and I tried many ways.

    First time when  I run the script, it is giving the below error. When I try to manually login to the site and try to execute, the script is working fine.

    Exception calling "DownloadString" with "1" argument(s): "The remote server returned an error: (407) Proxy Authentication Required."
    At C:\Users\uswmnl09\Desktop\Download\Test.ps1:37 char:1
    + $testcode1=$webclient.downloadstring($source) -split "<a\s+" | %{ [void]($_ -mat ...
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
        + FullyQualifiedErrorId : WebException

    In which section I need to do the changes?


    Wednesday, March 28, 2018 10:24 PM
  • So what will be the solution?

    I am new to powershell and I tried many ways.

    First time when  I run the script, it is giving the below error. When I try to manually login to the site and try to execute, the script is working fine.

    Exception calling "DownloadString" with "1" argument(s): "The remote server returned an error: (407) Proxy Authentication Required."
    At C:\Users\uswmnl09\Desktop\Download\Test.ps1:37 char:1
    + $testcode1=$webclient.downloadstring($source) -split "<a\s+" | %{ [void]($_ -mat ...
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
        + FullyQualifiedErrorId : WebException

    In which section I need to do the changes?



    Wednesday, March 28, 2018 10:24 PM
  • You have to get a correct link from the web page if it is possible.


    \_(ツ)_/

    Wednesday, March 28, 2018 10:33 PM
  • Validated and found that the link in the script is correct.

    Please let me know if anything needs to be changed in the script?

    Thursday, March 29, 2018 10:27 PM
  • Have you addressed the issue reported by the error message?

    \_(ツ)_/

    Thursday, March 29, 2018 11:50 PM
  • Nope What ever link I have used in the query, I was able to access manually. It would be helpful if you are able to tell me where is the issue occurring.
    Wednesday, April 4, 2018 10:25 PM
  • You need to authenticate on the proxy: "The remote server returned an error: (407) Proxy Authentication Required"


    \_(ツ)_/

    Wednesday, April 4, 2018 11:06 PM