Tag: PowerShell

Download S3 files using PowerShell

Download or copy S3 files

Use the below script to download the files from any S3 bucket to your local machine

$sourceBucket = '<bucket name from where you want to copy the files from>'
$profile = '<aws profile name>'
$Folder = '<Folder Path on local drive>'

$items = Get-S3Object -BucketName $sourceBucket -ProfileName $profile -Region 'us-east-1'
Write-Host "$($items.Length) objects to copy"
$index = 1
$items | % {
    Write-Host "$index/$($items.Length): $($_.Key)"
    $fileName = $Folder + ".\$($_.Key.Replace('/','\'))"
    Write-Host "$fileName"
    Read-S3Object -BucketName $sourceBucket -Key $_.Key -File $fileName -ProfileName $profile -Region 'us-east-1' > $null
    $index += 1
}


Disclosure: We use affiliate links to monetize our content.  We may receive a commission on products or services that you purchase through clicking on links within this blog.

Advertisement

PowerShell – Working with URLs

Invoke-WebRequest

This command is used to get the content from a web page by sending the HTTP or HTTPS requests to that page. In other words, you can simple parse or scrape a web page for images, links, etc.

In response, you will see a parsed result including status code, links, images, input fields, etc.

Invoke-WebRequest <Uri>

Example:

Invoke-WebRequest https://google.com

Output

SSL/TLS

By default, PowerShell uses TLS 1.0 for HTTP / HTTPS requests.

You can easily check the TLS version of any website.

Google Chrome – Click F12 > Go to Security Tab

tls

Mozilla Firefox – Click on the padlock icon on the left of address bar and then click on the right arrow as shown below.

fb

Click on more information and you will see a popup with TLS version.

tlsver

Internet Explorer – Open a website > Right click on page > Click on Properties

ie

Not all the websites use TLS 1.0 and it is not possible to establish a secure connection to a website using different TLS version.

error

So in order to make HTTP or HTTPS calls using Invoke-WebRequest, you have to force PowerShell to use a different version (eg. TLS 1.2)

# Forcing PowerShell to use TLS 1.2

[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12

Example:

1

If you are wondering, which versions you can use with PowerShell, then simply use the following cmdlet

[enum]::GetNames([Net.SecurityProtocolType])

ver

%d bloggers like this: