Loving Powershell

I have always loved Unix shell scripting and Unix shells in general. I especially like the shortcut keys in shell environment and you can choose different modes - it's emacs mode for me anytime. They are such productivity boost - it makes me cringe to see someone painstakingly pressing the back-arrow all the way to the front of the line just to fix a typo (and very often even in training videos). 😖

When it comes to Powershell, I am a newbie. I only started using it quite recently. However, I was pleasantly surprised on how powerful its construct is and how easy it is to manipulate objects in Powershell.

In Unix, the output of the commands are strings. So when you pipe them to the next command you are doing string manipulation - hence, sed, awk, cut... become very useful. However, in Powershell, they are objects - you can directly manipulate the objects themselves without having to rely on helper commands to locate the right fields all the time. 

Here are some examples to demonstrate it. In AWS, it is often required to assign security policies to users or roles, etc. However, those policy names are long and not easy to remember (for me anyway). So I end up querying those policy names and ARNs quite all the time. For example: search for S3 related security policies - 

1. using AWS-Tools for Powershell it is easy to filter based on the fields/attributes/properties and choose which one to display by directly manipulating the object's attributes:

PS C:\WINDOWS\system32> Get-IAMPolicies | Where-Object {$_.PolicyName -like "*S3*"} | Select-Object -Property PolicyName, arn

PolicyName                                              Arn                                                                    
----------                                              ---                                                                    
AmazonDMSRedshiftS3Role                                 arn:aws:iam::aws:policy/service-role/AmazonDMSRedshiftS3Role           
AmazonS3FullAccess                                      arn:aws:iam::aws:policy/AmazonS3FullAccess                             
QuickSightAccessForS3StorageManagementAnalyticsReadOnly arn:aws:iam::aws:policy/service-role/QuickSightAccessForS3StorageMan...
AmazonS3ReadOnlyAccess                                  arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess                         
AmazonS3OutpostsFullAccess                              arn:aws:iam::aws:policy/AmazonS3OutpostsFullAccess                     
S3StorageLensServiceRolePolicy                          arn:aws:iam::aws:policy/aws-service-role/S3StorageLensServiceRolePolicy
IVSRecordToS3                                           arn:aws:iam::aws:policy/aws-service-role/IVSRecordToS3                 
AmazonS3OutpostsReadOnlyAccess                          arn:aws:iam::aws:policy/AmazonS3OutpostsReadOnlyAccess                 



PS C:\WINDOWS\system32> 

See how neat the results are laid out! What I also like about PS is that its case insensitive - even using -like "*s3*" will return the same results. 👍

The huge complaint about AWS Tools for Powershell is its installation process - it took over 1 hour in one of my laptops over proxy, and about 5 minutes on another.

2. using AWS CLI native query capability

PS C:\WINDOWS\system32> aws iam list-policies --query 'Policies[?contains(PolicyName,`S3`)].[PolicyName, Arn]' --output text
AmazonDMSRedshiftS3Role	arn:aws:iam::aws:policy/service-role/AmazonDMSRedshiftS3Role
AmazonS3FullAccess	arn:aws:iam::aws:policy/AmazonS3FullAccess
QuickSightAccessForS3StorageManagementAnalyticsReadOnly	arn:aws:iam::aws:policy/service-role/QuickSightAccessForS3StorageManagementAnalyticsReadOnly
entAnalyticsReadOnly
AmazonS3ReadOnlyAccess	arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess
AmazonS3OutpostsFullAccess	arn:aws:iam::aws:policy/AmazonS3OutpostsFullAccess
S3StorageLensServiceRolePolicy	arn:aws:iam::aws:policy/aws-service-role/S3StorageLensServiceRolePolicy
IVSRecordToS3	arn:aws:iam::aws:policy/aws-service-role/IVSRecordToS3
AmazonS3OutpostsReadOnlyAccess	arn:aws:iam::aws:policy/AmazonS3OutpostsReadOnlyAccess

PS C:\WINDOWS\system32>

If it weren't for the syntax highlighter of this web page, I'd feel dizzy looking for the item that I want...😵  The AWS CLI is strictly case-sensitive and very unforgiving! 😠

3. using combination of AWS CLI query and awk

$ aws iam list-policies --query 'Policies[?contains(PolicyName, `S3`)]' --output text | awk '{print $9, $1}'
AmazonDMSRedshiftS3Role arn:aws:iam::aws:policy/service-role/AmazonDMSRedshiftS3Role
AmazonS3FullAccess arn:aws:iam::aws:policy/AmazonS3FullAccess
QuickSightAccessForS3StorageManagementAnalyticsReadOnly arn:aws:iam::aws:policy/service-role/QuickSightAccessForS3StorageManagementAnalyticsReadOnly
AmazonS3ReadOnlyAccess arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess
AmazonS3OutpostsFullAccess arn:aws:iam::aws:policy/AmazonS3OutpostsFullAccess
S3StorageLensServiceRolePolicy arn:aws:iam::aws:policy/aws-service-role/S3StorageLensServiceRolePolicy
IVSRecordToS3 arn:aws:iam::aws:policy/aws-service-role/IVSRecordToS3
AmazonS3OutpostsReadOnlyAccess arn:aws:iam::aws:policy/AmazonS3OutpostsReadOnlyAccess

Identical result as the previous one. As you can see, the awk command assumes the desired fields are the 9th and 1st fields. This makes the code fragile. 😱

I used WSL for the above command. If using Git Bash for Windows, then put the following before running any aws CLI commands:

$ export HOME=$USERPROFILE

This is needed for aws to locate the configuration and credential files.

4. using combination of AWS CLI and jq

$ aws iam list-policies --query 'Policies[?contains(PolicyName, `S3`)]' | jq '.[] | {PolicyName, Arn}'
{
  "PolicyName": "AmazonDMSRedshiftS3Role",
  "Arn": "arn:aws:iam::aws:policy/service-role/AmazonDMSRedshiftS3Role"
}
{
  "PolicyName": "AmazonS3FullAccess",
  "Arn": "arn:aws:iam::aws:policy/AmazonS3FullAccess"
}
{
  "PolicyName": "QuickSightAccessForS3StorageManagementAnalyticsReadOnly",
  "Arn": "arn:aws:iam::aws:policy/service-role/QuickSightAccessForS3StorageManagementAnalyticsReadOnly"
}
{
  "PolicyName": "AmazonS3ReadOnlyAccess",
  "Arn": "arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess"
}
{
  "PolicyName": "AmazonS3OutpostsFullAccess",
  "Arn": "arn:aws:iam::aws:policy/AmazonS3OutpostsFullAccess"
}
{
  "PolicyName": "S3StorageLensServiceRolePolicy",
  "Arn": "arn:aws:iam::aws:policy/aws-service-role/S3StorageLensServiceRolePolicy"
}
{
  "PolicyName": "IVSRecordToS3",
  "Arn": "arn:aws:iam::aws:policy/aws-service-role/IVSRecordToS3"
}
{
  "PolicyName": "AmazonS3OutpostsReadOnlyAccess",
  "Arn": "arn:aws:iam::aws:policy/AmazonS3OutpostsReadOnlyAccess"
}

or...

$ aws iam list-policies | jq '.Policies[] | select(.PolicyName | contains("S3")) | {PolicyName, Arn}'
{
  "PolicyName": "AmazonDMSRedshiftS3Role",
  "Arn": "arn:aws:iam::aws:policy/service-role/AmazonDMSRedshiftS3Role"
}
{
  "PolicyName": "AmazonS3FullAccess",
  "Arn": "arn:aws:iam::aws:policy/AmazonS3FullAccess"
}
{
  "PolicyName": "QuickSightAccessForS3StorageManagementAnalyticsReadOnly",
  "Arn": "arn:aws:iam::aws:policy/service-role/QuickSightAccessForS3StorageManagementAnalyticsReadOnly"
}
{
  "PolicyName": "AmazonS3ReadOnlyAccess",
  "Arn": "arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess"
}
{
  "PolicyName": "AmazonS3OutpostsFullAccess",
  "Arn": "arn:aws:iam::aws:policy/AmazonS3OutpostsFullAccess"
}
{
  "PolicyName": "S3StorageLensServiceRolePolicy",
  "Arn": "arn:aws:iam::aws:policy/aws-service-role/S3StorageLensServiceRolePolicy"
}
{
  "PolicyName": "IVSRecordToS3",
  "Arn": "arn:aws:iam::aws:policy/aws-service-role/IVSRecordToS3"
}
{
  "PolicyName": "AmazonS3OutpostsReadOnlyAccess",
  "Arn": "arn:aws:iam::aws:policy/AmazonS3OutpostsReadOnlyAccess"
}
$ aws iam list-policies | jq '.Policies[] | select(.PolicyName | contains("S3")) | .PolicyName, .Arn'
"AmazonDMSRedshiftS3Role"
"arn:aws:iam::aws:policy/service-role/AmazonDMSRedshiftS3Role"
"AmazonS3FullAccess"
"arn:aws:iam::aws:policy/AmazonS3FullAccess"
"QuickSightAccessForS3StorageManagementAnalyticsReadOnly"
"arn:aws:iam::aws:policy/service-role/QuickSightAccessForS3StorageManagementAnalyticsReadOnly"
"AmazonS3ReadOnlyAccess"
"arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess"
"AmazonS3OutpostsFullAccess"
"arn:aws:iam::aws:policy/AmazonS3OutpostsFullAccess"
"S3StorageLensServiceRolePolicy"
"arn:aws:iam::aws:policy/aws-service-role/S3StorageLensServiceRolePolicy"
"IVSRecordToS3"
"arn:aws:iam::aws:policy/aws-service-role/IVSRecordToS3"
"AmazonS3OutpostsReadOnlyAccess"
"arn:aws:iam::aws:policy/AmazonS3OutpostsReadOnlyAccess"