Friday 31 December 2021

Subtitles

 I was watching a French movie today and realised that the subtitle was coming up to early - about 1 second too early. It's a bit annoying. So I decided to fix it with a simple Javascript - it runs under Node.js.


//srt_update.js

function main() {
    // syntax: node srt_update.js filename
    // therefore the first argument is the 3rd word in the command line.
    const args = process.argv.slice(2);

    var filename = args[0];
    if (filename == null) {
        console.log("please specify file name.");
        return;
    }

    const readLine = require('readline');
    const f = require('fs');

    var rl = readLine.createInterface({
        input : f.createReadStream(filename),
        output : process.stdout,
        terminal: false
    });
    //const re=/(?<order>\d+)\n(?<start>[\d:,]+)\s+-{2}\>\s+(?<end>[\d:,]+)\n(?<text>[\s\S]*?(?=\n{2}|$))/gi;
    //const re=/(?<start>[\d:,]+)\s+-{2}\>\s+(?<end>[\d:,]+)/gi;
    const re=/(?<shour>[\d]+)[:](?<sminute>[\d]+)[:](?<ssecond>[\d]+)[,](?<smilli>[\d]+)\s+-{2}\>\s+(?<ehour>[\d]+)[:](?<eminute>[\d]+)[:](?<esecond>[\d]+)[,](?<emilli>[\d]+)/gi;
    rl.on('line', function (line) {
        var elements=re.exec(line);
        if (elements!=null) {
            var ssecond = parseInt( elements.groups.ssecond);
            if (ssecond!=59)
                ssecond=String(ssecond+1).padStart(2,'0');
            var esecond = parseInt(elements.groups.esecond);
            if (esecond!=59)
                esecond=String(esecond+1).padStart(2,'0');
            //console.log(`${line} => ${elements.groups.shour}-${elements.groups.sminute}-${ssecond}-${elements.groups.smilli}, ${elements.groups.ehour}-${elements.groups.eminute}-${esecond}-${elements.groups.emilli}`);
            console.log(`${elements.groups.shour}:${elements.groups.sminute}:${ssecond},${elements.groups.smilli} --> ${elements.groups.ehour}:${elements.groups.eminute}:${esecond},${elements.groups.emilli}`);
        } else
            console.log(line);
    });
}

main();
To run it in a command shell:

$ time node srt_update.js input.srt > output.srt

real    0m1.321s
user    0m0.701s
sys     0m0.062s

An alternative version by not using regular expression:

//srt_update2.js

function main() {
    // syntax: node srt_update.js filename
    // therefore the first argument is the 3rd word in the command line.
    const args = process.argv.slice(2);

    var filename = args[0];
    if (filename == null) {
        console.log("please specify file name.");
        return;
    }

    const readLine = require('readline');
    const f = require('fs');

    var rl = readLine.createInterface({
        input : f.createReadStream(filename),
        output : process.stdout,
        terminal: false
    });
   
    rl.on('line', function (line) {
        var elements = line.split(" --> ", 2);
        if (elements.length==2) {    
            var ssecond=parseInt(elements[0].substring(6, 8));
            var esecond=parseInt(elements[1].substring(6, 8));
            if (ssecond!=59)
                ssecond=String(ssecond+1).padStart(2,'0');
            if (esecond!=59)
                esecond=String(esecond+1).padStart(2,'0');

            console.log(`${elements[0].substring(0,6)}${ssecond}${elements[0].substring(8)} --> ${elements[1].substring(0,6)}${esecond}${elements[1].substring(8)}`);
        } else
            console.log(line);
    });
}

main();

Running result:


$ time node srt_update2.js input.srt > output.srt

real    0m1.304s
user    0m0.701s
sys     0m0.049s

The performance of the two versions seem to be very similar.

Thursday 23 December 2021

Merry Christmas 2021!

 It has been another great year. Merry Christmas and Happy New Year 2022!













Sunday 7 November 2021

Roses






















 














Saturday 18 September 2021

Oil Painting - Martha Jones

 













Recuerdos de la Alhambra - Take 3

Practising my favorite piece on my AU$120 Valencia while listening my favorite talk show on Youtube! Ah...life is good.


 

Monday 6 September 2021

Oil Painting - Nardole

 





Tuesday 31 August 2021

Oil Painting - 13th Doctor


 Did this one in the weekend. Still got some fixing to do.




Sunday 22 August 2021

Oil Painting - 9th Doctor

 This time I adopted Ben Lustenhouwer's palette and the 3 color zones practice. It really made things very easy!






Monday 16 August 2021

Oil Painting - 11th Doctor

 Did this one in the weekend. It's an experiment with slightly bolder colors, breaking rules.


Sunday 8 August 2021

Oil Painiting - the 10th Doctor

 More COVID lockdown, more oil painting! 'Finished' this one today. (Actually, I never finished them. There is always more tweaking to do).





Saturday 31 July 2021

Oil Painting - the 12th Doctor

 More lockdown, more time to paint! Finished this last weekend. Time to start another one.



Saturday 17 July 2021

Oil Painting

 The last oil painting I did was back in 2016. Work and frequent movement had sapped my energy to paint at home. The latest COVID lockdown gave me more incentive to pursue my hobbies at home, including oil painting. Just finished this one today.












我想躺平,但!

 居澳多年,早已融入当地社会,自然是过着与世无争的田园生活。自中年危机以来,重拾儿时嗜好 - 无非是琴棋书画,吃喝玩乐。细想起来,这不就是传说中的躺平主义吗?

然而,树欲静而风不止。。。

早年旅居新加坡时,有幸接触到凤凰卫视一清谈节目。一看便是十多年。然而,某年某月的某一天,节目突然遭取消。唉!不看也罢!

一个偶然的机会,在网上听到香港电台。顿觉耳目一新。一听又是五,六年。尤其是在午饭时间。然而,好景不长 - 好听的节目不是被取消,就是主持人被解雇。现在,港台连一个值得听的节目都不剩。

一时好奇,到网上查查。还好,我喜欢的主持人有在Youtube上继续。一看,哇!已经不是港台那样的温和节目了。唉!该怪谁呢?

我从小就远离政治,讨厌政客。但是,远在地球另一半的政治已无孔不入地渗透到身边,侵蚀着我的生活。我已躺平,但一只血淋淋的魔爪不停地抽走我床垫里的弹簧,使我不由自主地将身体侧向一边!






Saturday 3 July 2021

Recuerdos de la Alhambra - take 2

 2 more weeks of practice under COVID lock down.



Monday 14 June 2021

Recuerdos De La Alhambra

I got a 3/4 classical guitar that I bought years ago for my kids. But they never touched it. Since COVID I have been learning guitar playing with it. I just bought a full sized one a couple of weeks ago. It's so enjoyable to play any pieces by Tarrega and I especially love this one. Here is me butchering through the piece after practising for a few weeks.



Friday 2 April 2021

Like It or Not - Azure Function

 👍Here is what I like about Azure Functions:

  1. Resource Group - the concept of resource group is a good one. It allows me to see all the related resources in the same screen. In the Numbers Game example, it lists the related App Service Plan, Function App and Application Insights on the same screen (on Portal and VSCode). Also, if I want to delete the whole thing, I can just delete the resource group. That's just a few of the goodies that RG offers.
  2. Function App - this is another layer of the hierarchy that can group all the related functions under the same app. Settings like Keys, Logging, CORS can be specified at this level and apply to all the functions of the app. However, this is a double-edged sword (see below).
  3. Portal GUI is very intuitive
  4. Console and the other Development Tools - I can go into the console to see what's happening behind the scene. (OK, for serverless you shouldn't have to care about the server or what's behind the scene. But I am curious).
  5. Code is quite performant - speed of execution is pretty impressive - around 500ms - 800ms to solve the game. 
  6. Although not required for my function, I like the fact that Azure allows you to scale up by changing the App Service Plan. It would have been even nicer to be able to combine AWS' concurrency control with the scaling options here. 
👎What I don't quite like about Azure Functions:
  1. Portal GUI (and VSCode) loading of the screens are quite slow - loading of code takes a couple of seconds (with only 350 - 400 lines of code). However, the App Service Editor development tool from the portal is much quicker. (in fact, the Azure CLI/Powershell feel very slow in general).
  2. The language/Stack setting is at Function App level. Therefore, I am stuck with the same language for all functions underneath. In my example, the main code of solve game was developed a few years ago in Javascript, so it's natural to choose Node.js. However, the GetNumbersGameHtml function would be much easier to write in Python taking advantage of its """ multi-line string syntax. 
My overall impression about Azure Function is very positive albeit the tooling may still need some polishing and some missing features like data mapping and validation declaratively outside of the code.

Thursday 1 April 2021

Numbers Game on Azure Functions

Porting the Numbers Game onto Azure Function is so easy!
Just like in the AWS Lambda approach, I created two Azure Functions in the same Function App:
  • GetNumbersGameHtml - triggered by HTTP GET
  • SolveNumbersGame - triggered by HTTP POST
Both functions are configured with the same Function Key value.

Code for GetNumbersGameHtml:


module.exports = async function (context, req) {
    const key= req.headers["x-functions-key"] || req.query.code;

    context.res = {
        // status: 200, /* Defaults to 200 */
        headers: {
            "Content-Type": "text/html"
        },
        body: myHtml+key+myHtml2
    };
}

const myHtml = '<!DOCTYPE html>\n'+
'<html>\n'+...
Code for SolveNumbersGame:

module.exports = async function (context, req) {
    const e=req.body;
    var numbers=[e.n1,e.n2,e.n3,e.n4,e.n5,e.n6], target=e.target;
    context.log("solving: "+JSON.stringify(e));
	var answer=solveAllGames(numbers, target);
    
    context.res = {
        // status: 200, /* Defaults to 200 */
        body: JSON.stringify(answer)
    };
}
...
Although the speed of Azure's CLI/Powershell/Portal is very slow comparing to AWS, the Function execution is blazingly fast - between 500ms-800ms - this is on the lowest App Service Plan (F1). The AWS Lambda function can be 10x as slow, usually between 3-8s - I had to increase the timeout setting of the function. In fact, to cater for the long wait time on the web GUI, I downloaded and customised a nifty spinner like below. I kind of enjoy seeing it going 4-5 cycles before it disappears when the solution is displayed; but with Azure, I can hardly see it complete one cycle! (a bit disappointed 😋).

Please wait while getting the solutions...

100
25
5
4
7
9

AWS REST APIs offer fine-grained control such as Model, Mapping Template, which are missing in Azure Function. On the other hand, the way Azure uses Keys for the Function App and Function is very simple to use and effective.

Saturday 27 March 2021

Two Lambdas Are Better Than One

 To refactor the single Lambda function and API previously, I created two Lambda functions:

  1. getNumbersGameHtml - to return the HTML string, which is the GUI page of the Numbers Game single page web application.
  2. solveNumbersGame - to return the array of solutions (strings of math expressions)
The Python code for getNumbersGameHtml function:


def lambda_handler(event, context):
    return {
        'body': myHtml
    }
    
myHtml = ('<!DOCTYPE html>\n'
'<html>\n'
'<head profile=\"http://www.w3.org/2005/10/profile\"><link rel=... />...\n'
'<body>\n'
'...'
'	<input type="submit" value="Solve Game" onClick="if (validate_inputs()) {solve_game();} else alert(\'Please ensure all numbers and targets are filled in and with valid values.\\nHover mouse over input boxes for rules details.\');" />&nbsp;&nbsp;&nbsp;&nbsp;    \n'
'	<input type="button" value="Reset Game" onClick="reset_game()"/> \n'
'...')

# alternatively
myHtml="""<!DOCTYPE html>
<html>
<head profile=\"http://www.w3.org/2005/10/profile\"><link rel=... />...
<body>
...
	<input type="submit" value="Solve Game" onClick="if (validate_inputs()) {solve_game();} else alert(\'Please ensure all numbers and targets are filled in and with valid values.\\nHover mouse over input boxes for rules details.\');" />&nbsp;&nbsp;&nbsp;&nbsp;
	<input type="button" value="Reset Game" onClick="reset_game()"/>
...
"""
The web page looks like this:

    


Copyright (c) Romen Law. All rights reserved.

The Javascript code for solveNumbersGame function:

exports.handler = async (event) => {
	var e=event;
	var numbers=[e.n1,e.n2,e.n3,e.n4,e.n5,e.n6], target=e.target;
	var answer=solveAllGames(numbers, target);
	
    const response = {
        statusCode: 200,
        body: JSON.stringify(answer)
    };
    
    return response;
};
An API is created at the API Gateway with two methods implemented:
  • GET method invokes the getNumbersGameHtml function
    • at the GET - Integration Response, for response 200, 
      • add a Header Mappings entry with Response header of content-type and the Mapping value of 'text/html' (notice the single quote - it represents literal string in the mapping language).
      • at Mapping Templates, add an entry for Content-Type of text/html with template code of $input.path('body') (don't forget to press the grey Save button at the bottom before the blue Save button at the top).
    • at the GET - Method Response, for HTTP status 200 
      • add header of content-type
      • add Response model of Empty for content-type text/html
  • POST method invokes the solveNumbersGame function
    • at the POST - Method Request, Request Body
      • set the Request Validator field to Validate body and click the tick.
      • add NumbersInput model for application/json. This will enforce the input data format and some validation rules
    • at the POST - Integration Response, Mapping Templates: add template code $input.path('body') for content type application/json. This will extract the 'body' field from the response of the Lambda function and return it as the API's response.
The NumbersInput model:


{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "type": "object",
  "properties": {
    "n1": {
      "type": "integer", "minimum": 1, "maximum": 100
    },
    "n2": {
      "type": "integer", "minimum": 1, "maximum": 100
    },
    "n3": {
      "type": "integer", "minimum": 1, "maximum": 100
    },
    "n4": {
      "type": "integer", "minimum": 1, "maximum": 100
    },
    "n5": {
      "type": "integer", "minimum": 1, "maximum": 100
    },
    "n6": {
      "type": "integer", "minimum": 1, "maximum": 100
    },
    "target": {
      "type": "integer", "minimum": 100, "maximum": 999
    }
  },
  "required": [
    "n1",
    "n2",
    "n3",
    "n4",
    "n5",
    "n6",
    "target"
  ]
}
See also:

Sunday 21 March 2021

One Lambda to Serve Them All

 The really cheap way to host a dynamic web site on AWS is by using the combination of S3 and Lambda:

  • S3 to store the static contents such as HTML files, graphics, CSS, javascript files
  • Lambda and API Gateway to serve the dynamic contents by providing APIs to the javascripts of the web site.
I went one step further with my Numbers Game single page web application by serving everything from a Lambda function without having to use S3. This is because
  1. I want the web site to stay totally free. Lambda is perpetually free tier and I don't get charged as long as my volumes stay under the threshold. However, S3 is not free after the first 12 months.
  2. The static content for my web page is relatively simple - one HTML file actually. My own Javascripts are embedded in the page; 3rd-party contents (CSS, javascripts, icons) are pointed to their corresponding URLs so the browser will simply gets them from the internet.
  3. I don't want to enable CORS of my AWS API, so my web page (and its javascript) and the APIs must be served by the same domain. 
This is very easy to do with AWS Lambda: my Lambda function handles two types of requests - GET and POST (of course, you can refactor these to let the API layer do some mapping and use two Lambda functions... ). 
  • for GET, the Lambda function returns the HTML string. This string was prepared by feeding my static HTML file to some online converter to get the big string with all the double quotes (along with some other characters) escaped - the online tool's result was not perfect, I had to hand fix something that it missed. This HTML page basically allows the user to enter the input numbers and performs some validation according to the game's rules. Once submitted, it will invoke the API using POST
  • for POST, the Lambda function invokes the javascript functions to calculate the results and return them as a JSON string. The HTML simply displays the results on the same page.
The Lambda function:

exports.handler = async (event) => {
	console.log("event:"+JSON.stringify(event));
	var answer="no or unsupported event";
	var response;
	if(event.httpMethod == "GET") {
		response = {
	        statusCode: 200,
	        headers: {
	        	"content-type": "text/html"
	        },
	        body: myHtml,
	    };
	} else if (event.httpMethod == "POST" || event.routeKey.startsWith("POST")) { // supporting REST APIs and HTTP APIs
		if(event.body != null) {
			var e=JSON.parse(event.body);
			var numbers=[e.n1,e.n2,e.n3,e.n4,e.n5,e.n6], target=e.target;
			answer=solveAllGames(numbers, target);
			//answer=solveGame(numbers, target);
		}
	    response = {
	        statusCode: 200,
	        body: JSON.stringify(answer),
			headers: {
	        	"Access-Control-Allow-Origin": "*"
	        }
	    };
	}
    console.log(response);
    return response;
};
//... more code here ... where myHtml stores the big string converted from my static HTML file.

This approach works quite well. However, it is an abomination and wrong in so many places:
  • it's mixing network layer logic with business logic - in the code it's handling http methods which should be the responsibility of the API Gateway!
  • At the API, when only the ANY is mapped, the event passed into the Lambda is at a higher level - the Lambda function has to traverse into the 'body' to get the input parameters. 
  • The JSON.parse(event.body) is very cumbersome. I should be able to just get the input numbers from the event itself as a JSON object, not as a string. 
  • the lambda function cannot be used as-is for other triggering types (because it explicitly handles HTTP methods). e.g. cannot test in AWS console.
It's time to refactor... 💨

See also:

Monday 8 March 2021

Loving Powershell

I have always loved Unix shell scripting and Unix shells in general. I especially like the shortcut keys in shell environment and you can choose different modes - it's emacs mode for me anytime. They are such productivity boost - it makes me cringe to see someone painstakingly pressing the back-arrow all the way to the front of the line just to fix a typo (and very often even in training videos). 😖

When it comes to Powershell, I am a newbie. I only started using it quite recently. However, I was pleasantly surprised on how powerful its construct is and how easy it is to manipulate objects in Powershell.

In Unix, the output of the commands are strings. So when you pipe them to the next command you are doing string manipulation - hence, sed, awk, cut... become very useful. However, in Powershell, they are objects - you can directly manipulate the objects themselves without having to rely on helper commands to locate the right fields all the time. 

Here are some examples to demonstrate it. In AWS, it is often required to assign security policies to users or roles, etc. However, those policy names are long and not easy to remember (for me anyway). So I end up querying those policy names and ARNs quite all the time. For example: search for S3 related security policies - 

1. using AWS-Tools for Powershell it is easy to filter based on the fields/attributes/properties and choose which one to display by directly manipulating the object's attributes:
PS C:\WINDOWS\system32> Get-IAMPolicies | Where-Object {$_.PolicyName -like "*S3*"} | Select-Object -Property PolicyName, arn

PolicyName                                              Arn                                                                    
----------                                              ---                                                                    
AmazonDMSRedshiftS3Role                                 arn:aws:iam::aws:policy/service-role/AmazonDMSRedshiftS3Role           
AmazonS3FullAccess                                      arn:aws:iam::aws:policy/AmazonS3FullAccess                             
QuickSightAccessForS3StorageManagementAnalyticsReadOnly arn:aws:iam::aws:policy/service-role/QuickSightAccessForS3StorageMan...
AmazonS3ReadOnlyAccess                                  arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess                         
AmazonS3OutpostsFullAccess                              arn:aws:iam::aws:policy/AmazonS3OutpostsFullAccess                     
S3StorageLensServiceRolePolicy                          arn:aws:iam::aws:policy/aws-service-role/S3StorageLensServiceRolePolicy
IVSRecordToS3                                           arn:aws:iam::aws:policy/aws-service-role/IVSRecordToS3                 
AmazonS3OutpostsReadOnlyAccess                          arn:aws:iam::aws:policy/AmazonS3OutpostsReadOnlyAccess                 



PS C:\WINDOWS\system32> 

See how neat the results are laid out! What I also like about PS is that its case insensitive - even using -like "*s3*" will return the same results. 👍

The huge complaint about AWS Tools for Powershell is its installation process - it took over 1 hour in one of my laptops over proxy, and about 5 minutes on another.

2. using AWS CLI native query capability
PS C:\WINDOWS\system32> aws iam list-policies --query 'Policies[?contains(PolicyName,`S3`)].[PolicyName, Arn]' --output text
AmazonDMSRedshiftS3Role	arn:aws:iam::aws:policy/service-role/AmazonDMSRedshiftS3Role
AmazonS3FullAccess	arn:aws:iam::aws:policy/AmazonS3FullAccess
QuickSightAccessForS3StorageManagementAnalyticsReadOnly	arn:aws:iam::aws:policy/service-role/QuickSightAccessForS3StorageManagementAnalyticsReadOnly
entAnalyticsReadOnly
AmazonS3ReadOnlyAccess	arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess
AmazonS3OutpostsFullAccess	arn:aws:iam::aws:policy/AmazonS3OutpostsFullAccess
S3StorageLensServiceRolePolicy	arn:aws:iam::aws:policy/aws-service-role/S3StorageLensServiceRolePolicy
IVSRecordToS3	arn:aws:iam::aws:policy/aws-service-role/IVSRecordToS3
AmazonS3OutpostsReadOnlyAccess	arn:aws:iam::aws:policy/AmazonS3OutpostsReadOnlyAccess

PS C:\WINDOWS\system32>
If it weren't for the syntax highlighter of this web page, I'd feel dizzy looking for the item that I want...😵  The AWS CLI is strictly case-sensitive and very unforgiving! 😠

3. using combination of AWS CLI query and awk
$ aws iam list-policies --query 'Policies[?contains(PolicyName, `S3`)]' --output text | awk '{print $9, $1}'
AmazonDMSRedshiftS3Role arn:aws:iam::aws:policy/service-role/AmazonDMSRedshiftS3Role
AmazonS3FullAccess arn:aws:iam::aws:policy/AmazonS3FullAccess
QuickSightAccessForS3StorageManagementAnalyticsReadOnly arn:aws:iam::aws:policy/service-role/QuickSightAccessForS3StorageManagementAnalyticsReadOnly
AmazonS3ReadOnlyAccess arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess
AmazonS3OutpostsFullAccess arn:aws:iam::aws:policy/AmazonS3OutpostsFullAccess
S3StorageLensServiceRolePolicy arn:aws:iam::aws:policy/aws-service-role/S3StorageLensServiceRolePolicy
IVSRecordToS3 arn:aws:iam::aws:policy/aws-service-role/IVSRecordToS3
AmazonS3OutpostsReadOnlyAccess arn:aws:iam::aws:policy/AmazonS3OutpostsReadOnlyAccess
Identical result as the previous one. As you can see, the awk command assumes the desired fields are the 9th and 1st fields. This makes the code fragile. 😱

I used WSL for the above command. If using Git Bash for Windows, then put the following before running any aws CLI commands:
$ export HOME=$USERPROFILE
This is needed for aws to locate the configuration and credential files.

4. using combination of AWS CLI and jq

$ aws iam list-policies --query 'Policies[?contains(PolicyName, `S3`)]' | jq '.[] | {PolicyName, Arn}'
{
  "PolicyName": "AmazonDMSRedshiftS3Role",
  "Arn": "arn:aws:iam::aws:policy/service-role/AmazonDMSRedshiftS3Role"
}
{
  "PolicyName": "AmazonS3FullAccess",
  "Arn": "arn:aws:iam::aws:policy/AmazonS3FullAccess"
}
{
  "PolicyName": "QuickSightAccessForS3StorageManagementAnalyticsReadOnly",
  "Arn": "arn:aws:iam::aws:policy/service-role/QuickSightAccessForS3StorageManagementAnalyticsReadOnly"
}
{
  "PolicyName": "AmazonS3ReadOnlyAccess",
  "Arn": "arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess"
}
{
  "PolicyName": "AmazonS3OutpostsFullAccess",
  "Arn": "arn:aws:iam::aws:policy/AmazonS3OutpostsFullAccess"
}
{
  "PolicyName": "S3StorageLensServiceRolePolicy",
  "Arn": "arn:aws:iam::aws:policy/aws-service-role/S3StorageLensServiceRolePolicy"
}
{
  "PolicyName": "IVSRecordToS3",
  "Arn": "arn:aws:iam::aws:policy/aws-service-role/IVSRecordToS3"
}
{
  "PolicyName": "AmazonS3OutpostsReadOnlyAccess",
  "Arn": "arn:aws:iam::aws:policy/AmazonS3OutpostsReadOnlyAccess"
}
or...

$ aws iam list-policies | jq '.Policies[] | select(.PolicyName | contains("S3")) | {PolicyName, Arn}'
{
  "PolicyName": "AmazonDMSRedshiftS3Role",
  "Arn": "arn:aws:iam::aws:policy/service-role/AmazonDMSRedshiftS3Role"
}
{
  "PolicyName": "AmazonS3FullAccess",
  "Arn": "arn:aws:iam::aws:policy/AmazonS3FullAccess"
}
{
  "PolicyName": "QuickSightAccessForS3StorageManagementAnalyticsReadOnly",
  "Arn": "arn:aws:iam::aws:policy/service-role/QuickSightAccessForS3StorageManagementAnalyticsReadOnly"
}
{
  "PolicyName": "AmazonS3ReadOnlyAccess",
  "Arn": "arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess"
}
{
  "PolicyName": "AmazonS3OutpostsFullAccess",
  "Arn": "arn:aws:iam::aws:policy/AmazonS3OutpostsFullAccess"
}
{
  "PolicyName": "S3StorageLensServiceRolePolicy",
  "Arn": "arn:aws:iam::aws:policy/aws-service-role/S3StorageLensServiceRolePolicy"
}
{
  "PolicyName": "IVSRecordToS3",
  "Arn": "arn:aws:iam::aws:policy/aws-service-role/IVSRecordToS3"
}
{
  "PolicyName": "AmazonS3OutpostsReadOnlyAccess",
  "Arn": "arn:aws:iam::aws:policy/AmazonS3OutpostsReadOnlyAccess"
}
$ aws iam list-policies | jq '.Policies[] | select(.PolicyName | contains("S3")) | .PolicyName, .Arn'
"AmazonDMSRedshiftS3Role"
"arn:aws:iam::aws:policy/service-role/AmazonDMSRedshiftS3Role"
"AmazonS3FullAccess"
"arn:aws:iam::aws:policy/AmazonS3FullAccess"
"QuickSightAccessForS3StorageManagementAnalyticsReadOnly"
"arn:aws:iam::aws:policy/service-role/QuickSightAccessForS3StorageManagementAnalyticsReadOnly"
"AmazonS3ReadOnlyAccess"
"arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess"
"AmazonS3OutpostsFullAccess"
"arn:aws:iam::aws:policy/AmazonS3OutpostsFullAccess"
"S3StorageLensServiceRolePolicy"
"arn:aws:iam::aws:policy/aws-service-role/S3StorageLensServiceRolePolicy"
"IVSRecordToS3"
"arn:aws:iam::aws:policy/aws-service-role/IVSRecordToS3"
"AmazonS3OutpostsReadOnlyAccess"
"arn:aws:iam::aws:policy/AmazonS3OutpostsReadOnlyAccess"

Thursday 4 March 2021

Android Not-So-Smart TV

 On my TCL Android TV, the USB files are displayed on screen in reverse chronological order based on the file creation date, by default. To order by file name, I have to explicitly go to the menu and select it every time. This is really inconvenient. What's worse - the TV can only sort the file names as a string without considering numeric parts of the string - e.g. file 2.* will appear before 11.*. 

So for me to watch TV series or training videos, I 'd have to manually look for the next file to watch. This is not smart at all.

To get around the problem I recursively updated all my files' creation date to reversely align with their file names (with consideration of the numerical part of the file name).



$filepath="E:\1_TV\Show\[romen.com] - Block & Chain - A Complete Introduction"
ls -literalPath $filepath -Recurse | Sort-Object -descending { [regex]::Replace($_.Name, '\d+', { $args[0].Value.PadLeft(20) }) } | forEach-Object { 
    Write-Host "processing file" $_
    $_.CreationTime = Get-Date
    $_.LastWriteTime = Get-Date
    sleep 1
    }

Tuesday 2 March 2021

2c Worth of Notes

The following nuggets of knowledge costed me US2c (AUD3c) by trial and error playing with EC2 outside of the 12-month free tier.
  • after modifying (creating new versions of) launch templates make sure to make it default, or be specific of the version number when launching a new EC2 instance. Otherwise, the changes will not take effect.
  • It's better to copy and paste when creating User Data - I made the simple mistake of typing 'ec2_user' instead of 'ec2-user'. Also consult https://aws.amazon.com/premiumsupport/knowledge-center/execute-user-data-ec2/ to change the frequency of the execution of the user data.
  • the user data is run as root.
On another note, while I was playing with the EC2 launch template, my WSL played up - the DNS setting stopped working on the Linux VMs, and WSL kept overwriting the /etc/resolv.conf file even with /etc/wsl.conf created with 

[network]
generateResolvConf = false
It turns out that WSL has to be terminated to make the wls.conf take effect; alternatively, prevent the /etc/resolv.conf from being writable by
chattr +i /etc/resolve.conf

Thursday 11 February 2021

Happy New Year of the Ox!

Wednesday 10 February 2021

LWD

Amdox.x260
180679.99

Sunday 24 January 2021

Numbers Game - Part 6 - AWS Lambda

 Porting the numbers game to AWS Lambda is really simple. I created the Lambda function using AWS Console and added an API as trigger.

The AWS Lambda function for my numbers game is called getExprs. By adding a few lines of code in the generated index.js, and append my Javascript functions in the same file, it was working in no time. The index.js file:

exports.handler = async (event) => {
    console.log("event:"+JSON.stringify(event));
    var answer="no event";
    if(event.body != null) {
	var e=JSON.parse(event.body);
	var numbers=[e.n1,e.n2,e.n3,e.n4,e.n5,e.n6], target=e.target;
	answer=solveAllGames(numbers, target);
	//answer=solveGame(numbers, target);
    }
    const response = {
        statusCode: 200,
        body: JSON.stringify(answer),
        //body: answer,
    };
    console.log(answer);
    return response;
};
// my functions appended here...
Testing it with Postman:

 

Testing it in Python:
import requests

body={
  "n1": 75,
  "n2": 100,
  "n3": 2,
  "n4": 5,
  "n5": 9,
  "n6": 10,
  "target": 110
}
try:
    response = requests.post('https://xxxxxxxxxx.execute-api.ap-southeast-2.amazonaws.com/default/getExprs', json=body)
    response.raise_for_status()
    print(response.body.json());
except requests.exceptions.HTTPError as errh:
    print(errh)
except requests.exceptions.ConnectionError as errc:
    print(errc)
except requests.exceptions.Timeout as errt:
    print(errt)
except requests.exceptions.RequestException as err:
    print(err)

Results are:

======================= RESTART: C:\pcd_tmpl32\call.py =======================
['100+10', '2*5+100', '(2+9)*10', '(9-2)*5+75', '(75-10)+5*9', '5*9-(10-75)', '(100+5)+10/2', '(10-5)*2+100', '100*2-9*10', '100/10*(2+9)', '(100/5-9)*10', '100/5+9*10', '(5*9+10)*2', '((75+100)+5*9)/2', '(75-5)/(9-2)+100', '(75+9)/2*5-100', '((100-75)*9-5)/2', '75*2/(5+10)+100', '(75*2/10+100)-5', '75*2/10-(5-100)', '(75-5)+(100/2-10)', '(75/5+100)-10/2', '((75+5)/10+100)+2', '75/5-(10/2-100)', '(75+5)/(10-2)+100', '(75*10-100*2)/5', '((100*2-75)-5)-10', '100*2-((75+5)+10)', '(100*2-75)-(5+10)', '(100/2-(5-75))-10', '(75-100)*(5-9)+10', '(75-100)+(5+10)*9', '((75+5)+10)/9+100', '(((100-75)-5)-9)*10', '(100*9/75+10)*5', '(5*9+75)-100/10', '5*9-(100/10-75)', '75*2+(5-9)*10', '75*2-(9-5)*10', '(2-9)*(5-10)+75', '(100*2*5-10)/9', '100/5-10)*(2+9', '(100+9)+2*5/10', '(100+9)+10/2/5', '100+10)*(2*5-9', '(100/10+5*9)*2', '2*5-(9-10)*100', '(75+100)-((2+9)*5+10)', '((75+100)-(2+9)*5)-10', '(75+100)-(2*10+5*9)', '(75-100)+(2*10-5)*9', '((75+100)-(5-10)*9)/2', '(75+100)-((9-2)*10-5)', '((75+100)+(10-5)*9)/2', '(75*2+100*9)/10+5', '75*2+100/10*(5-9)', '75*2-100/10*(9-5)', '(75-5)-(100/2-9*10)', '75/5+(9*10+100)/2', '(75/5-9)*10+100/2', '((75+5)-10)/(9-2)+100', '(75*9-100)/5-10/2', '(((75-9)*2-100)-10)*5', '(((75+10)/5+100)+2)-9', '(75+10)/5-((9-100)-2)', '(100-75)-((2-9)-10)*5', '100*2*5*9/75-10', '100*2*9/75/5-10', '(100/2*9/75+5)*10', '(100*9/75*2+5)*10', '(5*9+75)/(2+10)+100']
>>>
One gripe I have about Lambda functions is that they are supposed to support multiple triggers - even the AWS Console shows so. However, the event that are passed into Lambda can have different structures depending on the channel/triggering type. For example, when integrated with a POST method of a REST API, the payload is the event - i.e. you can get the payload like n1=event.n1; however, when integrated with a POST method of a HTTP API, the payload is in the body - i.e. you can get the payload like n1=event.body.n1. This makes the Lambda function pretty much channel/trigger specific - you have to write another version of the Lambda function if you want to expose it to another trigger/channel.

See also: