AWS

Amazon Web Services

Lambda Java samples:

• lambda samles

• sample apps java basic

User credentials

• root vs iam

• aws access keys best practices

• IAM best practices

Instead of sharing the credentials of the AWS account root user, create individual IAM users, granting each user only the permissions they require.

Follow the best practice of using the root user only to create your first IAM user.

There are two types of credentials:

• Root user credentials, allow full access to all resources in the AWS account.
• IAM credentials, control access to AWS services and resources for users in your AWS account

Serverless blog web application architecture

• https://github.com/aws-samples/lambda-refarch-webapp

• https://s3.amazonaws.com/aws-lambda-serverless-web-refarch/RefArch_BlogApp_Serverless.png

  • Amazon Route 53 (routes to specific places based on region)

  • Amazon CloudFront (deliver static content per region hosted inside S3)

  • Amazon Simple Storage Service (S3)
  • Amazon Cognito (Authentication and authorization)
  • Amazon API Gateway (routes requests to backend logic)
  • AWS Lambda (backend business logic)
  • AWS DynamoDB (managed DB)
  • AWS Identity and Access Management (IAM) - web service to control access to AWS resources

IPv6 info

• introducing ipv6 only subnets and ec2 instances

• using instance addressing amazon-dns

• using instance addressing #working with ipv6 addresses

Lambda authorizer (API gateway)

A lambda authorizer (API gateway) requires a resource based policy statement, with principalId and policy document.

• API gateway lambda authorizer output

JSON example

   1 {
   2   "principalId": "user|12345",
   3   "policyDocument": {
   4     "Version": "2012-10-17",
   5     "Statement": [
   6       {
   7         "Action": "execute-api:Invoke",
   8         "Effect": "Allow",
   9         "Resource": "arn:aws:execute-api:region:account-id:api-id/stage/METHOD/path"
  10       }
  11     ]
  12   },
  13   "context": {
  14     "stringKey": "value",
  15     "numberKey": 123,
  16     "booleanKey": true
  17   }
  18 }

Add inbound rule in RDS to a lambda function

   1 RDS_DB_PORT=5432
   2 RDS_SG=$(aws rds describe-db-instances --db-instance-identifier rds-instance-id \
   3     --query "DBInstances[0].VpcSecurityGroups[*].VpcSecurityGroupId" --output text)
   4 echo "RDS security group $RDS_SG"
   5 
   6 LAMBDA_SG=$(aws lambda get-function-configuration --function-name lambda-function-name \
   7     --query "VpcConfig.SecurityGroupIds[*]" --output text)
   8 echo "Lambda function security group $LAMBDA_SG"
   9 
  10 aws ec2 authorize-security-group-ingress --group-id $RDS_SG \
  11     --protocol tcp -port $RDS_DB_PORT --source-group $LAMBDA_SG

Stuff

   1 aws lambda list-functions
   2 aws ecs describe-task-definition -- task-definition 
   3 # a task definition sets a docker image for a task/container
   4 
   5 
   6 aws apigatewayv2 get-apis
   7 aws apigatewayv2 create-api --protocol-type WEBSOCKET
   8 aws apigatewayv2 create-authorizer

SQS vs AmazonMQ

• AmazonMQ supports JMS (queues and topics), AMPQ, MQTT and STOMP with websokcet support
• SQS is proprietary

AWS Elasticache supports

• memcache
  • key:value , multi threaded, pure cache
• valkey
  • pub/sub , producer/consumer , single thread , complex structures

ActiveMQ versus Valkey

• ActiveMQ
  • JMS , message persistence, latency ms
• Valkey
  • fire and forget, latency micro seconds. subscribe/publish multicast , lpush/rpop brpop unicast

Queue - unicast - producer/consumer

Topic - multicast - pub/sub

ECS service, task definition for python consumers with auto scaling. SIGTERM is sent to terminate politely the python consumers.

AWS EventBridge might run periodically lambda functions or ECS tasks.

Lakehouse and Datawarehouse (Kimball/Inmon)

Motoserver (mock a lakehouse/datawarehouse):

• S3
• Glue
• Athena (DuckDB)

Dimensional modelling, Athena star schema

Cheap storage of a data lake (S3)

High speed SQL (Athena/Redshift)

ETL, extract , transform in a staging area, load in the warehouse

ELT, extract raw data, load it into a data lake (S3), transform only when we need to query it

Top-down Inmon method, corporate data warehouse uisng 3rd normal form (3NF) (DB normalization)

• define schema/structure in AWS glue. enforce schema in all S3 buckets
• use Athena federated query to get raw data into bronze raw s3 (bronze layer)
• 3NF warehouse (silver layer) use Athena to transform the raw data into a normalized structure stored as a parquet files in S2. single version of the truth. Athena looks at glue catalog to turn parquet files into a 3NF table with rows and columns
• Data mart (gold layer) create athena views that use the 3NF warehouse optimized for each business unit

Bronze (raw) discovery and ingestion Silver (warehouse) strict 3NF, single version of truth Gold (Marts) star schema, business reporting, most similar to a cube

2010, SQL databases, MDX cubes

2026, use SQL that evolved to include "window functions" and "grouping sets"

Analytical SQL (OLAP) 2026

Standard SQL (OLTP)

Deployment strategies

Blue/Green deployment (all at once)

• blue is the current version
• green is the new version

Canary traffic shift strategy, incremental from 10% to 100%

Feature flag

New logic in code is wrapped in a if statement. Can also be implemented using class extension with beans with the same interface/contract. A router bean chooses between the older or newer code implementation instead of relying on an if statement.

Update Amazon linux using dnf and yum check-update

   1 docker pull amazonlinux:2023.12.20260817.0  
   2 docker run -it  amazonlinux:2023.12.20260817.0 bash 
   3 yum check-update
   4 yum update
   5 yum upgrade
   6 dnf upgrade --refresh
   7 # enable rsyslog
   8 dnf install -y rsyslog
   9 systemctl enable --now rsyslog 
  10 systemctl status rsyslog 
  11 logger "Test message"
  12 echo -e "\nForwardToSyslog=yes\n" >> /etc/systemd/journald.conf
  13 
  14 systemctl restart systemd-journald
  15 systemctl restart rsyslog
  16 logger "Other test"
  17 
  18 nano /root/run-dnf-update.sh
  19 
  20 crontab -e # each minute
  21 * * * * * /root/run-dnf-update.sh
  22 
  23 tail -f /var/log/messages
  24 
  25 crontab -e # each midnight
  26 0 0 * * * /root/run-dnf-update.sh

/root/run-dnf-update.sh

   1 #!/bin/sh
   2 RELEASEVER=$(/usr/bin/yum check-update 2>/dev/null | /usr/bin/grep releasever | /usr/bin/awk  -F '=' '//{print $2}')
   3 
   4 if [ -n "$TARGET_VER" ]; then
   5     # Pass the parsed version to dnf/yum
   6     /usr/bin/dnf upgrade --releasever="$RELEASEVER" -y
   7     /usr/bin/logger "RELEASE VER $RELEASEVER"
   8 else
   9     /usr/bin/dnf upgrade -y
  10     /usr/bin/logger "No RELEASE_VER"
  11 fi