According to Wikipedia event-driven programming "is a programming paradigm in which the flow of the program is determined by events"

This doesn't say much but once I saw a simple example and that brought a lot of clarity.Will help a lot to understand it and know its internals.

Node.js has a module that you can use to create fire and listen for your own events. The essence is that we will listen for specific events and register listeners for each event.Listeners are just functions. Then we can emit events and the registered listeners will be executed. Each event is identified by a string or symbol.

By default the most basic code will look like this:

const EventEmitter = require("events");const emitter = new EventEmitter();emitter.on("testEvent", () => {  console.log("test event");})emitter.emit("testEvent");

What was so good about it was the "on" method. I have seen it in https/http and in ethersjs provider.on and jquery, it's almost everywhere and much of the Node.js core API is built around it.

My first technical article was about how to use awk.For some reason, I like learning bash even if it doesn't always make sense.I posted an article on hacker news and somebody suggested using one-liners, Perl.I thought that was a good idea but with Python, not Perl.

After starting to work on it I realized by default python is not that fitted for this.Unfortunately, python is not that useful for command-line one-liner programs.As soon as you need an indented block (like if) there is no way around line break, but even so, I think it will be a good exercise.Awk also allows you to provide a file with the script so it might not be a big deal in the end.

Basic usage

The first thing you need is to run python with some input as code.This is easily achievable using -c

python -c 'print(1 + 1)'

The most common usage of awk is to print a few fields out of a file, for example:

$ cat file.inName, Population(2020), Population(2020), Land area(sq mi),Los Angeles, 3898747, 3792621, 469.49San Diego, 1386932, 1307402, 325.88San Jose, 1013240,945942, 178.26San Francisco, 873965, 805235, 46.91$ awk  -F, '{print $1 $3}' file.inName Population(2020)Los Angeles 3792621San Diego 1307402San Jose945942San Francisco 805235

The same thing in Python will look like:

$ python -c 'for line in open("file.in"): fields=line.split(","); print (fields[0].strip(),fields[3].strip())'Name    Land area(sq mi)Los Angeles    469.49San Diego    325.88San Jose    178.26San Francisco    46.91

Python looks a bit more complex and since it is longer it will be more complicated to type if this is not part of a script.Another inconvenience is that each field needs to be stripped.We also made use of ";" and wrote multiple instructions on the same line.

Begin end

Awk has begin and end actions.These are very useful if you need to do sum or averages.

$ seq 99 101 >numbers.txt$ awk '{sum=sum+$1} END {print "Sum: "sum" Avg: "sum/NR}' numbers.txSum: 300 Avg: 100

Python version will look like:

$ python -c 'sum=0; NR=0for line in open("numbers.tx"): sum+=int(line); NR+=1print("Sum:",sum," Avg:",sum/NR)'Sum: 300  Avg: 100.0

Awk can be written as a one-line while the python version will need multiple lines and initialization for variables.At this point, I started to think that even if I do these multi-line commands it will be better to have a small utility function that will deal with the parsing and initialize standard awk variables.

import sysdef arg_param_value(i):    if i < len(sys.argv):        return sys.argv[i]    else:        return ""def parse_params():    i = 1    while i < len(sys.argv):        arg = sys.argv[i]        if arg.startswith('-'):            i += 1            args[arg] = arg_param_value(i)        else:            inputs.append(sys.argv[i])        i += 1def code():    if '-f' in args:        with open(args['-f'], "r") as f:            return f.readlines()    else:        return inputs.pop(0)  # removes first element from the listdef process_file():    global NR    FNR = 0    with open(FILENAME, "r") as file:        for line in file:            line = line.strip()            if len(line) == 0:                continue            if FS == '':                fields = line.split()            else:                fields = line.split(FS)            NF = len(fields)            FNR += 1            NR += 1            exec(code)args = {}inputs = []parse_params()code = code()NR = 0FS = ''if '-F' in args: FS = args['-F']for FILENAME in inputs:    process_file()

And I run it like:

$ python main.py -F ";" "print(fields[2], fields[3])" in1.csv in2.csv

Of course, there is more functionality in awk, I will slowly add it to my script and hopefully, I can turn it into something usable.There won't be a $0 $1 $2 .. $n, instead, I will use line for $0 and fields[] for $1 $2 and all the others.

After doing all this I think python is useful for these types of tasks especially if you already use it in other places.The main advantage would be that you don't have to learn awk if you already know python.The GitHub with a more complete version is here: https://github.com/liviusd/pawk

Please let me know what you think about it!

A few weeks ago I read again about it, it was something about how to build a SaaS with 0$, and lambda was mentioned because you get 1 000 000 free calls.I liked the idea, seeing some benefits in it and I thought it would be useful to try all sorts of things, so I will need an easy deployment process that I could use to quickly run side projects.In order to follow this, you will need an AWS account and aws cli working.I will be using java but it should be easy to adapt to any other language because I will deploy it using Docker container.I did this using the command line because it is easier to replicate.

How lambda works

In theory, things are simple. You write a function that is ran based on some event.In this tutorial, the event is a REST call.Each such function will be executed in a container.This means lambda has its own isolated execution environment.The container is created when the function is executed for the first time.After some time the container is destroyed.Because of this, you might get into cold start issues.This happens because your container may take some time to start.Sometimes AWS will reuse your container so you won't notice it but if your function is rarely called you might see it takes a lot of time to execute.

Build the project

I used the IntelliJ Idea and created an empty maven project.I added the AWS lambda java core dependency to the project and build a simple method that handles a request.

The java code looks like this:

package bq;import com.amazonaws.services.lambda.runtime.Context;public class App {    public static Object handleRequest(Object arg, Context context) {        return arg;    }}

The maven pom to package this is:

<?xml version="1.0" encoding="UTF-8"?><project xmlns="http://maven.apache.org/POM/4.0.0"         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">    <modelVersion>4.0.0</modelVersion>    <groupId>bq</groupId>    <artifactId>bq</artifactId>    <version>1.0-SNAPSHOT</version>    <build>        <plugins>            <plugin>                <artifactId>maven-compiler-plugin</artifactId>                <version>3.5.1</version>                <configuration>                    <source>11</source>                    <target>11</target>                </configuration>            </plugin>                <plugin>                    <groupId>org.apache.maven.plugins</groupId>                    <artifactId>maven-dependency-plugin</artifactId>                    <version>3.1.2</version>                    <configuration>                        <includeScope>runtime</includeScope>                    </configuration>                    <executions>                        <execution>                            <id>copy-dependencies</id>                            <phase>package</phase>                            <goals>                                <goal>copy-dependencies</goal>                            </goals>                        </execution>                    </executions>                </plugin>        </plugins>    </build>    <dependencies>        <dependency>            <groupId>com.amazonaws</groupId>            <artifactId>aws-lambda-java-core</artifactId>            <version>1.2.1</version>        </dependency>    </dependencies></project>

We need maven-dependency-plugin because we will copy all the jar files into the docker container.This plugin will make sure we have them in the target/dependency directory. Maybe there is a way to push everything as a jar but this is good enough to start.

Make sure everything works

$ mvn package

This is the project structure:

. Dockerfile lambda-iam.json lambda.iml pom.xml src    main       java          bq              App.java       resources    test        java target     bq-1.0-SNAPSHOT.jar     classes        bq            App.class     dependency        aws-lambda-java-core-1.2.1.jar     generated-sources        annotations     maven-archiver         pom.properties    //removed the rest

Push the container

As previously mentioned the lambda will start a docker container.This uses AWS base images for Lambda.More details about it here https://gallery.ecr.aws/lambda/javaThe usage tab also shows you how to build your lambda container if you need more instructions.

The docker file:

FROM public.ecr.aws/lambda/java:11COPY target/classes ${LAMBDA_TASK_ROOT}COPY target/dependency/* ${LAMBDA_TASK_ROOT}/lib/CMD [ "bq.App::handleRequest" ]RUN chmod 644 $(find . -type f)RUN chmod 755 $(find . -type d)

Build the container

mvn packagedocker build . -t java-aws-echo:1

And tested it:

docker run  -p 9000:8080 java-aws-echo:1curl -XPOST "http://localhost:9000/2015-03-31/functions/function/invocations" -d '{}'

The docker image will be hosted in Elastic Container Registry (ecr)After I saw everything worked on localhost I created an ecr repository and pushed the image:

aws ecr create-repository --repository-name java-aws-echoaws ecr get-login-password --region us-west-2 | docker login --username AWS --password-stdin YOUR_AWS_ACCOUNT_ID.dkr.ecr.us-west-2.amazonaws.comdocker tag java-aws-echo:1 YOUR_AWS_ACCOUNT_ID.dkr.ecr.us-west-2.amazonaws.com/java-aws-echo:1docker push YOUR_AWS_ACCOUNT_ID.dkr.ecr.us-west-2.amazonaws.com/java-aws-echo:1

Create the lambda

Before we create the lambda we need to create a role that lambda will assume.This is the minimal policy I found somewhere in AWS docs:

{  "Version": "2012-10-17",  "Statement": [    {      "Effect": "Allow",      "Principal": {        "Service": "lambda.amazonaws.com"      },      "Action": "sts:AssumeRole"    }  ]}

aws iam create-role --role-name lambda --assume-role-policy-document file://lambda.json

Creating the lambda from the command line is fairly simple.

aws lambda create-function --function-name java-lambda --code ImageUri=YOUR_AWS_ACCOUNT_ID.dkr.ecr.us-west-2.amazonaws.com/java-aws-echo:1 --role arn:aws:iam::YOUR_AWS_ACCOUNT_ID:role/lambda --package-type Image

And voila that is all for your function. The only thing is that is not yet exposed on the internet. You can give it a try from AWS console UI and trigger a test.

Exposing the lambda into internet.

This was the part I wasn't expecting to be that complicated.Once you have your lambda deployed you are probably asking yourself, where do I find the URL?My assumption was that once you created it by default it will receive a default URL.It is not the case and you need to do a few more steps and your function will be exposed using the AWS API Gateway.Here is a bash script I created that will create an API GW for your lambda:

#!/usr/bin/env bashNAME="Lambda API GW"METHOD="POST"#you can find the arn if you go on Lambda > Functions > java-lambda ->Copy ARNLAMBDA_ARN="PROVIDE THE LAMBDA ARN!!!!"REGION="us-west-2"aws apigateway create-rest-api --name "$NAME" > apigateway-create-rest-api.tmpAPI_ID=$(cat apigateway-create-rest-api.tmp |jq ".id" -r)echo $API_IDROOT_ID=$(aws apigateway get-resources --rest-api-id $API_ID|jq '.items|first.id' -r)echo $ROOT_ID#requestaws apigateway put-method --rest-api-id $API_ID --resource-id $ROOT_ID --http-method $METHOD --authorization-type NONE --no-api-key-required#Responseaws apigateway put-method-response --rest-api-id $API_ID --resource-id $ROOT_ID --http-method $METHOD --status-code 200 --response-models "{\"application/json\": \"Empty\"}"echo "aws apigateway put-method --rest-api-id $API_ID --resource-id $ROOT_ID --http-method $METHOD"echo "Integrations"URI="arn:aws:apigateway:$REGION:lambda:path/2015-03-31/functions/$LAMBDA_ARN/invocations"echo $URIaws apigateway put-integration --rest-api-id $API_ID --resource-id $ROOT_ID --type AWS --http-method $METHOD --integration-http-method $METHOD --uri $URIaws apigateway put-integration-response --rest-api-id $API_ID --resource-id $ROOT_ID --http-method $METHOD --status-code 200 --content-handling CONVERT_TO_TEXTecho "Permissions"FUNCTION_NAME=$(aws lambda get-function --function-name $LAMBDA_ARN|jq '.Configuration.FunctionName' -r)ACCOUNT_ID=$(aws sts get-caller-identity|jq '.Account' -r)SOURCE_ARN="arn:aws:execute-api:$REGION:$ACCOUNT_ID:$API_ID/*/$METHOD/"echo "$FUNCTION_NAME $SOURCE_ARN"aws lambda add-permission --function-name $FUNCTION_NAME --statement-id STTT --action lambda:InvokeFunction --principal apigateway.amazonaws.com --source-arn $SOURCE_ARNaws apigateway create-deployment --rest-api-id $API_ID --stage-name atm1

I then searched for the lambda URL in the AWS console.Log into your account and select API Gateway.The URL for it is: https://us-west-2.console.aws.amazon.com/apigateway/main/apisThen click on the newly created API and it should show the Resources and the POST method.In the left menu, there should be "Stages", click on it and then select the atm1.
On a blue background, there should be "Invoke URL" in the stage editor.Give it a test:

curl -XPOST "Your Invoke URL" -d '{}'

If you followed this tutorial and you have any issues please let me know. Hope this helps!

A long time ago I learned the basics of sed and grep and this helped me a lot while doing support. Before having fancy tools like Kibana and CloudWatch knowing to grep was paying off. Even now if I need to debug something very complicated I prefer having the logs in a console.

Another thing that I still do in the console is some sanity checks between different sources. For example, I have a customer's file that I need to match to another file that has database records and see they are in sync. This is how I got to learn AWK and write about how I use it.

I could add a proper introduction to AWK here but I assume you have some experience with it so I'll keep it short. AWK is a text-processing language and it is used as a data extraction tool or other similar tasks.

How it works

Awk is a scripting language, requires no compilation, and has variables, numeric and string functions, and logical operators.

The basic thing to remember about it is that it deals with records and fields. Records are read one at a time and then they are split into fields. You can imagine it like reading a simple CSV file where a record is a line and a field is a cell.

Basic usage

The most basic usage of awk is to get just a few fields out of each line.

$ echo "a b c" | awk '{ print $2" "$3}'b c

A few things to remember from here. Each field has its own variable: $1, $2, $3, and so on ($0 represents the entire record). Other important to remember variables (not all of them):

• NR: Number of Records
• NF: Number of Fields
• FNR refers to the record number (typically the line number) in the current file different than NR refers to the total record number. This has an interesting usage because NR==FNR is true only for the first file, and is used a lot in file comparison

Separator

By default, awk will split records based on whitespace. If you have a different separator use -F. The delimiter can be a regular expression: -F'[,:]' so you can use multiple delimiters.

$ echo "a,b,c" | awk -F','  '{print $1" "$2}'a b

Input

Most probably you won't use awk with echo or commands from the standard input, reading from a file would look like this:

$ echo "a b c" > f1.txt$ awk '{ print $2" "$3}' f1.txt b c

Awk accepts multiple input files:

$  echo "1 2 3" > f2.txt$ awk '{ print $2" "$3}' f1.txt f2.txt b c2 3

Program source code can also be read from a file, for this we use the -f option:

$ echo '{ print $2" "$3}' > aw$ awk -f aw f1.txt b c

Selection Criteria

Before doing the processing for each record you can use selection criteria. This can be a pattern or a condition. This is useful if you want to process the lines which match the given pattern or respect condition.

awk '/manager/ {print}' f1.txt

Begin/End

You can also add begin and end actions.

$awk '      BEGIN {print "start"}{print $1" "$3}END {print "end"}' f1.txt starta cend

These are very useful if you need to do an average or print a sum.

$ seq 99 101 >numbers.txt$ awk '{sum=sum+$1}END {print "Sum: "sum" Avg: "sum/NR}' numbers.txt Sum: 300 Avg: 100

seq 99 101 >numbers.txt will create a sequence of numbers, one per line in a file

Matching 2 files

With these examples in mind, we can try the main task, match 2 files based on one of the fields. You can think of it as a database inner join.

There will be 2 files: names.txt and license.txt. In names.txt the first column is the id while in license.txt the second column is the id. The script will be in the matchScript file. We want to match each name to its license.

Awk will iterate over both files. As mentioned earlier, FNR is the record number in the current file while NR is the record number out of the total. The condition NR==FNR will hold true only while iterating records from the first files, this means details from the first file will be stored in a map. The map will be initialized on the first usage.Next will end current rerond processing moving to the next one.

The second part will check if the id is already in the map and print if the condition matched.

$ cat names.txt 1,Sleepy3,Dopey2,Grumpy$ cat license.txt BASIC,2BASIC,1PRO,3$ cat matchScript (NR==FNR){   names[$1]=$2;    next}($2 in names){   print names[$2]","$1}$ awk -f matchScript -F ',' names.txt license.txt Grumpy,BASICSleepy,BASICDopey,PRO

Other

If you need more info or you want to go into deep details this is the best place for it.

My hopes are I will keep a log of tech things I'm doing and trying. From time to time I will also post nontech things I found interesting.

Why BlockingQueue? I really like StackOverflow but all bug names domain are already taken, so I was left with a data structure

That's all for now, read me later :)